Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset
In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise unde...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/1/151 |
| _version_ | 1797431266114535424 |
|---|---|
| author | Romain Wenger Anne Puissant Jonathan Weber Lhassane Idoumghar Germain Forestier |
| author_facet | Romain Wenger Anne Puissant Jonathan Weber Lhassane Idoumghar Germain Forestier |
| author_sort | Romain Wenger |
| collection | DOAJ |
| description | In the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise understanding of a given territory. Few studies have combined information from Sentinel-1 and Sentinel-2 imagery, but merging radar and optical imagery has been shown to have several benefits for a range of study cases, such as semantic segmentation or classification. For this study, we used a newly produced dataset, MultiSenGE, which provides a set of multitemporal and multimodal patches over the Grand-Est region in France. To merge these data, we propose a CNN approach based on spatio-temporal and spatio-spectral feature fusion, ConvLSTM+Inception-S1S2. We used a U-Net base model and ConvLSTM extractor for spatio-temporal features and an inception module for the spatio-spectral features extractor. The results show that describing an overrepresented class is preferable to map urban fabrics (UF). Furthermore, the addition of an Inception module on a date allowing the extraction of spatio-spectral features improves the classification results. Spatio-spectro-temporal method (ConvLSTM+Inception-S1S2) achieves higher global weighted <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>F</mi><msub><mn>1</mn><mrow><mi>S</mi><mi>c</mi><mi>o</mi><mi>r</mi><mi>e</mi></mrow></msub></mrow></semantics></math></inline-formula> than all other methods tested. |
| first_indexed | 2024-03-09T09:41:26Z |
| format | Article |
| id | doaj.art-26b85946bfee45dd8e6351de8d3b6177 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T09:41:26Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-26b85946bfee45dd8e6351de8d3b61772023-12-02T00:51:17ZengMDPI AGRemote Sensing2072-42922022-12-0115115110.3390/rs15010151Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE DatasetRomain Wenger0Anne Puissant1Jonathan Weber2Lhassane Idoumghar3Germain Forestier4LIVE UMR 7362 CNRS, University of Strasbourg, F-67000 Strasbourg, FranceLIVE UMR 7362 CNRS, University of Strasbourg, F-67000 Strasbourg, FranceIRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, FranceIRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, FranceIRIMAS UR 7499, University of Haute-Alsace, F-68100 Mulhouse, FranceIn the context of global change, up-to-date land use land cover (LULC) maps is a major challenge to assess pressures on natural areas. These maps also allow us to assess the evolution of land cover and to quantify changes over time (such as urban sprawl), which is essential for having a precise understanding of a given territory. Few studies have combined information from Sentinel-1 and Sentinel-2 imagery, but merging radar and optical imagery has been shown to have several benefits for a range of study cases, such as semantic segmentation or classification. For this study, we used a newly produced dataset, MultiSenGE, which provides a set of multitemporal and multimodal patches over the Grand-Est region in France. To merge these data, we propose a CNN approach based on spatio-temporal and spatio-spectral feature fusion, ConvLSTM+Inception-S1S2. We used a U-Net base model and ConvLSTM extractor for spatio-temporal features and an inception module for the spatio-spectral features extractor. The results show that describing an overrepresented class is preferable to map urban fabrics (UF). Furthermore, the addition of an Inception module on a date allowing the extraction of spatio-spectral features improves the classification results. Spatio-spectro-temporal method (ConvLSTM+Inception-S1S2) achieves higher global weighted <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>F</mi><msub><mn>1</mn><mrow><mi>S</mi><mi>c</mi><mi>o</mi><mi>r</mi><mi>e</mi></mrow></msub></mrow></semantics></math></inline-formula> than all other methods tested.https://www.mdpi.com/2072-4292/15/1/151multitemporalmultimodalSentinel-1Sentinel-2land useland cover |
| spellingShingle | Romain Wenger Anne Puissant Jonathan Weber Lhassane Idoumghar Germain Forestier Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset Remote Sensing multitemporal multimodal Sentinel-1 Sentinel-2 land use land cover |
| title | Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset |
| title_full | Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset |
| title_fullStr | Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset |
| title_full_unstemmed | Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset |
| title_short | Multimodal and Multitemporal Land Use/Land Cover Semantic Segmentation on Sentinel-1 and Sentinel-2 Imagery: An Application on a MultiSenGE Dataset |
| title_sort | multimodal and multitemporal land use land cover semantic segmentation on sentinel 1 and sentinel 2 imagery an application on a multisenge dataset |
| topic | multitemporal multimodal Sentinel-1 Sentinel-2 land use land cover |
| url | https://www.mdpi.com/2072-4292/15/1/151 |
| work_keys_str_mv | AT romainwenger multimodalandmultitemporallanduselandcoversemanticsegmentationonsentinel1andsentinel2imageryanapplicationonamultisengedataset AT annepuissant multimodalandmultitemporallanduselandcoversemanticsegmentationonsentinel1andsentinel2imageryanapplicationonamultisengedataset AT jonathanweber multimodalandmultitemporallanduselandcoversemanticsegmentationonsentinel1andsentinel2imageryanapplicationonamultisengedataset AT lhassaneidoumghar multimodalandmultitemporallanduselandcoversemanticsegmentationonsentinel1andsentinel2imageryanapplicationonamultisengedataset AT germainforestier multimodalandmultitemporallanduselandcoversemanticsegmentationonsentinel1andsentinel2imageryanapplicationonamultisengedataset |