Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset
As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mappi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9103196/ |
| _version_ | 1819150256427761664 |
|---|---|
| author | Chunping Qiu Xiaochong Tong Michael Schmitt Benjamin Bechtel Xiao Xiang Zhu |
| author_facet | Chunping Qiu Xiaochong Tong Michael Schmitt Benjamin Bechtel Xiao Xiang Zhu |
| author_sort | Chunping Qiu |
| collection | DOAJ |
| description | As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This article is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multilevel features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture, and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF as well as the So2Sat LCZ42 dataset. We also intensively investigated the influence of network depth and width, and the effectiveness of the design choices made for Sen2LCZ-Net-MF. This article will provide important baselines for future CNN-based algorithm developments for both LCZ classification and other urban land cover land use classification. Code and pretrained models are available at https://github.com/ChunpingQiu/benchmark-on-So2SatLCZ42-dataset-a-simple-tour. |
| first_indexed | 2024-12-22T14:14:37Z |
| format | Article |
| id | doaj.art-ddead96be9f64b9999c3ce959f88b707 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-22T14:14:37Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-ddead96be9f64b9999c3ce959f88b7072022-12-21T18:23:09ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352020-01-01132793280610.1109/JSTARS.2020.29957119103196Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 DatasetChunping Qiu0Xiaochong Tong1Michael Schmitt2https://orcid.org/0000-0002-0575-2362Benjamin Bechtel3Xiao Xiang Zhu4https://orcid.org/0000-0001-5530-3613Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM), Munich, GermanyInformation Engineering University, Zhengzhou, ChinaSignal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM), Munich, GermanyInstitute of Geography, Ruhr-University Bochum, Bochum, GermanyGerman Aerospace Center (DLR) and Signal Processing in Earth Observation (SiPEO), Remote Sensing Technology Institute (IMF), Technical University of Munich, Munich, GermanyAs a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This article is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multilevel features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture, and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF as well as the So2Sat LCZ42 dataset. We also intensively investigated the influence of network depth and width, and the effectiveness of the design choices made for Sen2LCZ-Net-MF. This article will provide important baselines for future CNN-based algorithm developments for both LCZ classification and other urban land cover land use classification. Code and pretrained models are available at https://github.com/ChunpingQiu/benchmark-on-So2SatLCZ42-dataset-a-simple-tour.https://ieeexplore.ieee.org/document/9103196/Benchmarkconvolutional neural networks (CNNs)local climate zones (LCZ)sentinel-2urban land cover |
| spellingShingle | Chunping Qiu Xiaochong Tong Michael Schmitt Benjamin Bechtel Xiao Xiang Zhu Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Benchmark convolutional neural networks (CNNs) local climate zones (LCZ) sentinel-2 urban land cover |
| title | Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset |
| title_full | Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset |
| title_fullStr | Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset |
| title_full_unstemmed | Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset |
| title_short | Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset |
| title_sort | multilevel feature fusion based cnn for local climate zone classification from sentinel 2 images benchmark results on the so2sat lcz42 dataset |
| topic | Benchmark convolutional neural networks (CNNs) local climate zones (LCZ) sentinel-2 urban land cover |
| url | https://ieeexplore.ieee.org/document/9103196/ |
| work_keys_str_mv | AT chunpingqiu multilevelfeaturefusionbasedcnnforlocalclimatezoneclassificationfromsentinel2imagesbenchmarkresultsontheso2satlcz42dataset AT xiaochongtong multilevelfeaturefusionbasedcnnforlocalclimatezoneclassificationfromsentinel2imagesbenchmarkresultsontheso2satlcz42dataset AT michaelschmitt multilevelfeaturefusionbasedcnnforlocalclimatezoneclassificationfromsentinel2imagesbenchmarkresultsontheso2satlcz42dataset AT benjaminbechtel multilevelfeaturefusionbasedcnnforlocalclimatezoneclassificationfromsentinel2imagesbenchmarkresultsontheso2satlcz42dataset AT xiaoxiangzhu multilevelfeaturefusionbasedcnnforlocalclimatezoneclassificationfromsentinel2imagesbenchmarkresultsontheso2satlcz42dataset |