Multi-Task Learning and Multimodal Fusion for Road Segmentation
The road segmentation task is to extract the road surface from the image at pixel level. In road segmentation for remote sensing images, deep learning-based methods have shown high-quality results in various scenarios. However, existing segmentation methods usually produce discontinuous roads, which...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9713858/ |
| _version_ | 1811162027204280320 |
|---|---|
| author | Bowen Cheng Miaomiao Tian Shuai Jiang Weiwei Liu Yalong Pang |
| author_facet | Bowen Cheng Miaomiao Tian Shuai Jiang Weiwei Liu Yalong Pang |
| author_sort | Bowen Cheng |
| collection | DOAJ |
| description | The road segmentation task is to extract the road surface from the image at pixel level. In road segmentation for remote sensing images, deep learning-based methods have shown high-quality results in various scenarios. However, existing segmentation methods usually produce discontinuous roads, which is not beneficial to applying practical scenarios. We propose a multi-task learning method of road segmentation, direction estimation and road edge learning to make our model connect roads reasonably. Moreover, we use the initial road segmentation results and the direction estimation results to make cascade inference to improve the model’s performance. We adopt the Canny operator to extract the edge information of images as the auxiliary modality fusion. We demonstrate our method’s effectiveness on two large-scale road segmentation datasets DeepGlobe and SpaceNet. |
| first_indexed | 2024-04-10T06:23:02Z |
| format | Article |
| id | doaj.art-a34d7bdb8f894c74b7e4909c0f371e47 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T06:23:02Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a34d7bdb8f894c74b7e4909c0f371e472023-03-02T00:00:41ZengIEEEIEEE Access2169-35362023-01-0111189471895910.1109/ACCESS.2022.31513729713858Multi-Task Learning and Multimodal Fusion for Road SegmentationBowen Cheng0https://orcid.org/0000-0003-1333-7182Miaomiao Tian1Shuai Jiang2https://orcid.org/0000-0002-4785-5132Weiwei Liu3Yalong Pang4Beijing Institute of Spacecraft System Engineering, Beijing, ChinaBeijing Institute of Spacecraft System Engineering, Beijing, ChinaBeijing Institute of Spacecraft System Engineering, Beijing, ChinaBeijing Institute of Spacecraft System Engineering, Beijing, ChinaBeijing Institute of Spacecraft System Engineering, Beijing, ChinaThe road segmentation task is to extract the road surface from the image at pixel level. In road segmentation for remote sensing images, deep learning-based methods have shown high-quality results in various scenarios. However, existing segmentation methods usually produce discontinuous roads, which is not beneficial to applying practical scenarios. We propose a multi-task learning method of road segmentation, direction estimation and road edge learning to make our model connect roads reasonably. Moreover, we use the initial road segmentation results and the direction estimation results to make cascade inference to improve the model’s performance. We adopt the Canny operator to extract the edge information of images as the auxiliary modality fusion. We demonstrate our method’s effectiveness on two large-scale road segmentation datasets DeepGlobe and SpaceNet.https://ieeexplore.ieee.org/document/9713858/Deep learningremote sensing imagemulti-task learningmultimodal fusioncascade inferenceHRNet |
| spellingShingle | Bowen Cheng Miaomiao Tian Shuai Jiang Weiwei Liu Yalong Pang Multi-Task Learning and Multimodal Fusion for Road Segmentation IEEE Access Deep learning remote sensing image multi-task learning multimodal fusion cascade inference HRNet |
| title | Multi-Task Learning and Multimodal Fusion for Road Segmentation |
| title_full | Multi-Task Learning and Multimodal Fusion for Road Segmentation |
| title_fullStr | Multi-Task Learning and Multimodal Fusion for Road Segmentation |
| title_full_unstemmed | Multi-Task Learning and Multimodal Fusion for Road Segmentation |
| title_short | Multi-Task Learning and Multimodal Fusion for Road Segmentation |
| title_sort | multi task learning and multimodal fusion for road segmentation |
| topic | Deep learning remote sensing image multi-task learning multimodal fusion cascade inference HRNet |
| url | https://ieeexplore.ieee.org/document/9713858/ |
| work_keys_str_mv | AT bowencheng multitasklearningandmultimodalfusionforroadsegmentation AT miaomiaotian multitasklearningandmultimodalfusionforroadsegmentation AT shuaijiang multitasklearningandmultimodalfusionforroadsegmentation AT weiweiliu multitasklearningandmultimodalfusionforroadsegmentation AT yalongpang multitasklearningandmultimodalfusionforroadsegmentation |