Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery
Land cover classification (LCC) of complex landscapes is attractive to the remote sensing community but poses great challenges. In complex open pit mining and agricultural development landscapes (CMALs), the landscape-specific characteristics limit the accuracy of LCC. The combination of traditional...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/11/22/2716 |
| _version_ | 1818321954254356480 |
|---|---|
| author | Xianju Li Zhuang Tang Weitao Chen Lizhe Wang |
| author_facet | Xianju Li Zhuang Tang Weitao Chen Lizhe Wang |
| author_sort | Xianju Li |
| collection | DOAJ |
| description | Land cover classification (LCC) of complex landscapes is attractive to the remote sensing community but poses great challenges. In complex open pit mining and agricultural development landscapes (CMALs), the landscape-specific characteristics limit the accuracy of LCC. The combination of traditional feature engineering and machine learning algorithms (MLAs) is not sufficient for LCC in CMALs. Deep belief network (DBN) methods achieved success in some remote sensing applications because of their excellent unsupervised learning ability in feature extraction. The usability of DBN has not been investigated in terms of LCC of complex landscapes and integrating multimodal inputs. A novel multimodal and multi-model deep fusion strategy based on DBN was developed and tested for fine LCC (FLCC) of CMALs in a 109.4 km<sup>2</sup> area of Wuhan City, China. First, low-level and multimodal spectral−spatial and topographic features derived from ZiYuan-3 imagery were extracted and fused. The features were then input into a DBN for deep feature learning. The developed features were fed to random forest and support vector machine (SVM) algorithms for classification. Experiments were conducted that compared the deep features with the softmax function and low-level features with MLAs. Five groups of training, validation, and test sets were performed with some spatial auto-correlations. A spatially independent test set and generalized McNemar tests were also employed to assess the accuracy. The fused model of DBN-SVM achieved overall accuracies (OAs) of 94.74% ± 0.35% and 81.14% in FLCC and LCC, respectively, which significantly outperformed almost all other models. From this model, only three of the twenty land covers achieved OAs below 90%. In general, the developed model can contribute to FLCC and LCC in CMALs, and more deep learning algorithm-based models should be investigated in future for the application of FLCC and LCC in complex landscapes. |
| first_indexed | 2024-12-13T10:49:06Z |
| format | Article |
| id | doaj.art-634b0fa12701475aa03b51adf93ee103 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-12-13T10:49:06Z |
| publishDate | 2019-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-634b0fa12701475aa03b51adf93ee1032022-12-21T23:49:58ZengMDPI AGRemote Sensing2072-42922019-11-011122271610.3390/rs11222716rs11222716Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 ImageryXianju Li0Zhuang Tang1Weitao Chen2Lizhe Wang3School of Computer Science, China University of Geosciences, Wuhan 430074, ChinaSchool of Computer Science, China University of Geosciences, Wuhan 430074, ChinaSchool of Computer Science, China University of Geosciences, Wuhan 430074, ChinaSchool of Computer Science, China University of Geosciences, Wuhan 430074, ChinaLand cover classification (LCC) of complex landscapes is attractive to the remote sensing community but poses great challenges. In complex open pit mining and agricultural development landscapes (CMALs), the landscape-specific characteristics limit the accuracy of LCC. The combination of traditional feature engineering and machine learning algorithms (MLAs) is not sufficient for LCC in CMALs. Deep belief network (DBN) methods achieved success in some remote sensing applications because of their excellent unsupervised learning ability in feature extraction. The usability of DBN has not been investigated in terms of LCC of complex landscapes and integrating multimodal inputs. A novel multimodal and multi-model deep fusion strategy based on DBN was developed and tested for fine LCC (FLCC) of CMALs in a 109.4 km<sup>2</sup> area of Wuhan City, China. First, low-level and multimodal spectral−spatial and topographic features derived from ZiYuan-3 imagery were extracted and fused. The features were then input into a DBN for deep feature learning. The developed features were fed to random forest and support vector machine (SVM) algorithms for classification. Experiments were conducted that compared the deep features with the softmax function and low-level features with MLAs. Five groups of training, validation, and test sets were performed with some spatial auto-correlations. A spatially independent test set and generalized McNemar tests were also employed to assess the accuracy. The fused model of DBN-SVM achieved overall accuracies (OAs) of 94.74% ± 0.35% and 81.14% in FLCC and LCC, respectively, which significantly outperformed almost all other models. From this model, only three of the twenty land covers achieved OAs below 90%. In general, the developed model can contribute to FLCC and LCC in CMALs, and more deep learning algorithm-based models should be investigated in future for the application of FLCC and LCC in complex landscapes.https://www.mdpi.com/2072-4292/11/22/2716fine classificationcomplex landscapedeep learningdbnfusionmultimodalziyuan-3machine learning algorithmremote sensing |
| spellingShingle | Xianju Li Zhuang Tang Weitao Chen Lizhe Wang Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery Remote Sensing fine classification complex landscape deep learning dbn fusion multimodal ziyuan-3 machine learning algorithm remote sensing |
| title | Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery |
| title_full | Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery |
| title_fullStr | Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery |
| title_full_unstemmed | Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery |
| title_short | Multimodal and Multi-Model Deep Fusion for Fine Classification of Regional Complex Landscape Areas Using ZiYuan-3 Imagery |
| title_sort | multimodal and multi model deep fusion for fine classification of regional complex landscape areas using ziyuan 3 imagery |
| topic | fine classification complex landscape deep learning dbn fusion multimodal ziyuan-3 machine learning algorithm remote sensing |
| url | https://www.mdpi.com/2072-4292/11/22/2716 |
| work_keys_str_mv | AT xianjuli multimodalandmultimodeldeepfusionforfineclassificationofregionalcomplexlandscapeareasusingziyuan3imagery AT zhuangtang multimodalandmultimodeldeepfusionforfineclassificationofregionalcomplexlandscapeareasusingziyuan3imagery AT weitaochen multimodalandmultimodeldeepfusionforfineclassificationofregionalcomplexlandscapeareasusingziyuan3imagery AT lizhewang multimodalandmultimodeldeepfusionforfineclassificationofregionalcomplexlandscapeareasusingziyuan3imagery |