A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection
Nonagriculturalization incidents are serious threats to local agricultural ecosystem and global food security. Remote sensing change detection (CD) can provide an effective approach for in-time detection and prevention of such incidents. However, existing CD methods are difficult to deal with the la...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9780164/ |
| _version_ | 1811246123298324480 |
|---|---|
| author | Mengxi Liu Zhuoqun Chai Haojun Deng Rong Liu |
| author_facet | Mengxi Liu Zhuoqun Chai Haojun Deng Rong Liu |
| author_sort | Mengxi Liu |
| collection | DOAJ |
| description | Nonagriculturalization incidents are serious threats to local agricultural ecosystem and global food security. Remote sensing change detection (CD) can provide an effective approach for in-time detection and prevention of such incidents. However, existing CD methods are difficult to deal with the large intraclass differences of cropland changes in high-resolution images. In addition, traditional CNN based models are plagued by the loss of long-range context information, and the high computational complexity brought by deep layers. Therefore, in this article, we propose a CNN-transformer network with multiscale context aggregation (MSCANet), which combines the merits of CNN and transformer to fulfill efficient and effective cropland CD. In the MSCANet, a CNN-based feature extractor is first utilized to capture hierarchical features, then a transformer-based MSCA is designed to encode and aggregate context information. Finally, a multibranch prediction head with three CNN classifiers is applied to obtain change maps, to enhance the supervision for deep layers. Besides, for the lack of CD dataset with fine-grained cropland change of interest, we also provide a new cropland change detection dataset, which contains 600 pairs of 512 × 512 bi-temporal images with the spatial resolution of 0.5–2m. Comparative experiments with several CD models prove the effectiveness of the MSCANet, with the highest F1 of 64.67% on the high-resolution semantic CD dataset, and of 71.29% on CLCD. |
| first_indexed | 2024-04-12T14:49:07Z |
| format | Article |
| id | doaj.art-a60e405cf1324f408a130b74f557aeae |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-04-12T14:49:07Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-a60e405cf1324f408a130b74f557aeae2022-12-22T03:28:32ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352022-01-01154297430610.1109/JSTARS.2022.31772359780164A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change DetectionMengxi Liu0https://orcid.org/0000-0001-5237-4758Zhuoqun Chai1Haojun Deng2https://orcid.org/0000-0002-7013-2450Rong Liu3https://orcid.org/0000-0002-4642-9086Guangdong Provincial Key Laboratory for Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou, ChinaGuangdong Provincial Key Laboratory for Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou, ChinaGuangdong Provincial Key Laboratory for Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou, ChinaGuangdong Provincial Key Laboratory for Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-sen University, Guangzhou, ChinaNonagriculturalization incidents are serious threats to local agricultural ecosystem and global food security. Remote sensing change detection (CD) can provide an effective approach for in-time detection and prevention of such incidents. However, existing CD methods are difficult to deal with the large intraclass differences of cropland changes in high-resolution images. In addition, traditional CNN based models are plagued by the loss of long-range context information, and the high computational complexity brought by deep layers. Therefore, in this article, we propose a CNN-transformer network with multiscale context aggregation (MSCANet), which combines the merits of CNN and transformer to fulfill efficient and effective cropland CD. In the MSCANet, a CNN-based feature extractor is first utilized to capture hierarchical features, then a transformer-based MSCA is designed to encode and aggregate context information. Finally, a multibranch prediction head with three CNN classifiers is applied to obtain change maps, to enhance the supervision for deep layers. Besides, for the lack of CD dataset with fine-grained cropland change of interest, we also provide a new cropland change detection dataset, which contains 600 pairs of 512 × 512 bi-temporal images with the spatial resolution of 0.5–2m. Comparative experiments with several CD models prove the effectiveness of the MSCANet, with the highest F1 of 64.67% on the high-resolution semantic CD dataset, and of 71.29% on CLCD.https://ieeexplore.ieee.org/document/9780164/Change detection (CD)croplanddeep learning (DL)remote sensingtransformer |
| spellingShingle | Mengxi Liu Zhuoqun Chai Haojun Deng Rong Liu A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Change detection (CD) cropland deep learning (DL) remote sensing transformer |
| title | A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection |
| title_full | A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection |
| title_fullStr | A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection |
| title_full_unstemmed | A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection |
| title_short | A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection |
| title_sort | cnn transformer network with multiscale context aggregation for fine grained cropland change detection |
| topic | Change detection (CD) cropland deep learning (DL) remote sensing transformer |
| url | https://ieeexplore.ieee.org/document/9780164/ |
| work_keys_str_mv | AT mengxiliu acnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT zhuoqunchai acnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT haojundeng acnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT rongliu acnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT mengxiliu cnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT zhuoqunchai cnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT haojundeng cnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection AT rongliu cnntransformernetworkwithmultiscalecontextaggregationforfinegrainedcroplandchangedetection |