DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing
In this article, we propose to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9756343/ |
| _version_ | 1818270832717201408 |
|---|---|
| author | Gaston Lenczner Adrien Chan-Hon-Tong Bertrand Le Saux Nicola Luminari Guy Le Besnerais |
| author_facet | Gaston Lenczner Adrien Chan-Hon-Tong Bertrand Le Saux Nicola Luminari Guy Le Besnerais |
| author_sort | Gaston Lenczner |
| collection | DOAJ |
| description | In this article, we propose to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely, these interactions are annotations representing the semantic labels. Our methodological contribution is twofold. First, we propose two interactive learning schemes to integrate user inputs into deep neural networks. The first one concatenates the annotations with the other network’s inputs. The second one uses the annotations as a sparse ground truth to retrain the network. Second, we propose an active learning (AL) strategy to guide the user toward the most relevant areas to annotate. To this purpose, we compare different state-of-the-art acquisition functions to evaluate the neural network uncertainty such as ConfidNet, entropy, or ODIN. Through experiments on three remote sensing datasets, we show the effectiveness of the proposed methods. Notably, we show that AL based on uncertainty estimation enables to quickly lead the user toward mistakes and that it is thus relevant to guide the user interventions. Code will be open-source and released in this repository.<sup>1</sup> |
| first_indexed | 2024-12-12T21:16:33Z |
| format | Article |
| id | doaj.art-1861963f2c144f1eab421864d09eed17 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-12T21:16:33Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-1861963f2c144f1eab421864d09eed172022-12-22T00:11:44ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352022-01-01153376338910.1109/JSTARS.2022.31665519756343DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote SensingGaston Lenczner0https://orcid.org/0000-0002-4697-8894Adrien Chan-Hon-Tong1Bertrand Le Saux2https://orcid.org/0000-0001-7162-6746Nicola Luminari3Guy Le Besnerais4Information Processing and Systems Department (DTIS), ONERA, Université Paris-Saclay, Palaiseau, FranceInformation Processing and Systems Department (DTIS), ONERA, Université Paris-Saclay, Palaiseau, FranceESA/ESRIN Φ-lab, Frascati, ItalyAlteia, Toulouse, FranceInformation Processing and Systems Department (DTIS), ONERA, Université Paris-Saclay, Palaiseau, FranceIn this article, we propose to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely, these interactions are annotations representing the semantic labels. Our methodological contribution is twofold. First, we propose two interactive learning schemes to integrate user inputs into deep neural networks. The first one concatenates the annotations with the other network’s inputs. The second one uses the annotations as a sparse ground truth to retrain the network. Second, we propose an active learning (AL) strategy to guide the user toward the most relevant areas to annotate. To this purpose, we compare different state-of-the-art acquisition functions to evaluate the neural network uncertainty such as ConfidNet, entropy, or ODIN. Through experiments on three remote sensing datasets, we show the effectiveness of the proposed methods. Notably, we show that AL based on uncertainty estimation enables to quickly lead the user toward mistakes and that it is thus relevant to guide the user interventions. Code will be open-source and released in this repository.<sup>1</sup>https://ieeexplore.ieee.org/document/9756343/Active learning (AL)deep learningearth observationinteractive segmentationsemantic segmentation |
| spellingShingle | Gaston Lenczner Adrien Chan-Hon-Tong Bertrand Le Saux Nicola Luminari Guy Le Besnerais DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Active learning (AL) deep learning earth observation interactive segmentation semantic segmentation |
| title | DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing |
| title_full | DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing |
| title_fullStr | DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing |
| title_full_unstemmed | DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing |
| title_short | DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing |
| title_sort | dial deep interactive and active learning for semantic segmentation in remote sensing |
| topic | Active learning (AL) deep learning earth observation interactive segmentation semantic segmentation |
| url | https://ieeexplore.ieee.org/document/9756343/ |
| work_keys_str_mv | AT gastonlenczner dialdeepinteractiveandactivelearningforsemanticsegmentationinremotesensing AT adrienchanhontong dialdeepinteractiveandactivelearningforsemanticsegmentationinremotesensing AT bertrandlesaux dialdeepinteractiveandactivelearningforsemanticsegmentationinremotesensing AT nicolaluminari dialdeepinteractiveandactivelearningforsemanticsegmentationinremotesensing AT guylebesnerais dialdeepinteractiveandactivelearningforsemanticsegmentationinremotesensing |