SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION
In this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-s...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2018-09-01
|
| Series: | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-1/29/2018/isprs-annals-IV-1-29-2018.pdf |
| _version_ | 1818529461056831488 |
|---|---|
| author | K. Chen K. Chen M. Weinmann X. Sun M. Yan S. Hinz B. Jutzi M. Weinmann |
| author_facet | K. Chen K. Chen M. Weinmann X. Sun M. Yan S. Hinz B. Jutzi M. Weinmann |
| author_sort | K. Chen |
| collection | DOAJ |
| description | In this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-scale extension of the Shuffling Convolutional Neural Network (SCNN) with deep supervision. Thereby, we take the advantage of the SCNN involving the shuffling operator to effectively upsample feature maps and then fuse multiscale features derived from the intermediate layers of the SCNN, which results in the Multi-scale Shuffling Convolutional Neural Network (MSCNN). Based on the MSCNN, we derive the DSCNN by introducing additional losses into the intermediate layers of the MSCNN. In addition, we investigate the impact of using different sets of hand-crafted radiometric and geometric features derived from the true orthophotos and the DSMs on the semantic segmentation task. For performance evaluation, we use a commonly used benchmark dataset. The achieved results reveal that both multi-scale fusion and deep supervision contribute to an improvement in performance. Furthermore, the use of a diversity of hand-crafted radiometric and geometric features as input for the DSCNN does not provide the best numerical results, but smoother and improved detections for several objects. |
| first_indexed | 2024-12-11T17:07:10Z |
| format | Article |
| id | doaj.art-2d6e428f313d4f329dced04a4cb83db1 |
| institution | Directory Open Access Journal |
| issn | 2194-9042 2194-9050 |
| language | English |
| last_indexed | 2024-12-11T17:07:10Z |
| publishDate | 2018-09-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-2d6e428f313d4f329dced04a4cb83db12022-12-22T00:57:39ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502018-09-01IV-1293610.5194/isprs-annals-IV-1-29-2018SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISIONK. Chen0K. Chen1M. Weinmann2X. Sun3M. Yan4S. Hinz5B. Jutzi6M. Weinmann7Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Institute of Electronics, Chinese Academy of Sciences, Beijing, P. R. ChinaUniversity of Chinese Academy of Sciences, Beijing, P. R. ChinaInstitute of Computer Science II, University of Bonn, Bonn, GermanyKey Laboratory of Technology in Geo-Spatial Information Processing and Application System, Institute of Electronics, Chinese Academy of Sciences, Beijing, P. R. ChinaKey Laboratory of Technology in Geo-Spatial Information Processing and Application System, Institute of Electronics, Chinese Academy of Sciences, Beijing, P. R. ChinaInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, Karlsruhe, GermanyInstitute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, Karlsruhe, GermanyIn this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-scale extension of the Shuffling Convolutional Neural Network (SCNN) with deep supervision. Thereby, we take the advantage of the SCNN involving the shuffling operator to effectively upsample feature maps and then fuse multiscale features derived from the intermediate layers of the SCNN, which results in the Multi-scale Shuffling Convolutional Neural Network (MSCNN). Based on the MSCNN, we derive the DSCNN by introducing additional losses into the intermediate layers of the MSCNN. In addition, we investigate the impact of using different sets of hand-crafted radiometric and geometric features derived from the true orthophotos and the DSMs on the semantic segmentation task. For performance evaluation, we use a commonly used benchmark dataset. The achieved results reveal that both multi-scale fusion and deep supervision contribute to an improvement in performance. Furthermore, the use of a diversity of hand-crafted radiometric and geometric features as input for the DSCNN does not provide the best numerical results, but smoother and improved detections for several objects.https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-1/29/2018/isprs-annals-IV-1-29-2018.pdf |
| spellingShingle | K. Chen K. Chen M. Weinmann X. Sun M. Yan S. Hinz B. Jutzi M. Weinmann SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION |
| title_full | SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION |
| title_fullStr | SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION |
| title_full_unstemmed | SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION |
| title_short | SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION |
| title_sort | semantic segmentation of aerial imagery via multi scale shuffling convolutional neural networks with deep supervision |
| url | https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-1/29/2018/isprs-annals-IV-1-29-2018.pdf |
| work_keys_str_mv | AT kchen semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT kchen semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT mweinmann semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT xsun semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT myan semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT shinz semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT bjutzi semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision AT mweinmann semanticsegmentationofaerialimageryviamultiscaleshufflingconvolutionalneuralnetworkswithdeepsupervision |