Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery
Cloud cover remains a significant limitation to a broad range of applications relying on optical remote sensing imagery, including crop identification/yield prediction, climate monitoring, and land cover classification. A common approach to cloud removal treats the problem as an inpainting task and...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/6/1342 |
| _version_ | 1797442793698754560 |
|---|---|
| author | Mikolaj Czerkawski Priti Upadhyay Christopher Davison Astrid Werkmeister Javier Cardona Robert Atkinson Craig Michie Ivan Andonovic Malcolm Macdonald Christos Tachtatzis |
| author_facet | Mikolaj Czerkawski Priti Upadhyay Christopher Davison Astrid Werkmeister Javier Cardona Robert Atkinson Craig Michie Ivan Andonovic Malcolm Macdonald Christos Tachtatzis |
| author_sort | Mikolaj Czerkawski |
| collection | DOAJ |
| description | Cloud cover remains a significant limitation to a broad range of applications relying on optical remote sensing imagery, including crop identification/yield prediction, climate monitoring, and land cover classification. A common approach to cloud removal treats the problem as an inpainting task and imputes optical data in the cloud-affected regions employing either mosaicing historical data or making use of sensing modalities not impacted by cloud obstructions, such as SAR. Recently, deep learning approaches have been explored in these applications; however, the majority of reported solutions rely on external learning practices, i.e., models trained on fixed datasets. Although these models perform well within the context of a particular dataset, a significant risk of spatial and temporal overfitting exists when applied in different locations or at different times. Here, cloud removal was implemented within an internal learning regime through an inpainting technique based on the deep image prior. The approach was evaluated on both a synthetic dataset with an exact ground truth, as well as real samples. The ability to inpaint the cloud-affected regions for varying weather conditions across a whole year with no prior training was demonstrated, and the performance of the approach was characterised. |
| first_indexed | 2024-03-09T12:47:14Z |
| format | Article |
| id | doaj.art-c08080e56c7446ebbf982ca6a2de8415 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T12:47:14Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-c08080e56c7446ebbf982ca6a2de84152023-11-30T22:11:22ZengMDPI AGRemote Sensing2072-42922022-03-01146134210.3390/rs14061342Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite ImageryMikolaj Czerkawski0Priti Upadhyay1Christopher Davison2Astrid Werkmeister3Javier Cardona4Robert Atkinson5Craig Michie6Ivan Andonovic7Malcolm Macdonald8Christos Tachtatzis9Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKDepartment of Electronic and Electrical Engineering, University of Strathclyde, Glasgow G1 1XW, UKCloud cover remains a significant limitation to a broad range of applications relying on optical remote sensing imagery, including crop identification/yield prediction, climate monitoring, and land cover classification. A common approach to cloud removal treats the problem as an inpainting task and imputes optical data in the cloud-affected regions employing either mosaicing historical data or making use of sensing modalities not impacted by cloud obstructions, such as SAR. Recently, deep learning approaches have been explored in these applications; however, the majority of reported solutions rely on external learning practices, i.e., models trained on fixed datasets. Although these models perform well within the context of a particular dataset, a significant risk of spatial and temporal overfitting exists when applied in different locations or at different times. Here, cloud removal was implemented within an internal learning regime through an inpainting technique based on the deep image prior. The approach was evaluated on both a synthetic dataset with an exact ground truth, as well as real samples. The ability to inpaint the cloud-affected regions for varying weather conditions across a whole year with no prior training was demonstrated, and the performance of the approach was characterised.https://www.mdpi.com/2072-4292/14/6/1342cloud removalSentinel-1Sentinel-2deep image priorinternal learningimage inpainting |
| spellingShingle | Mikolaj Czerkawski Priti Upadhyay Christopher Davison Astrid Werkmeister Javier Cardona Robert Atkinson Craig Michie Ivan Andonovic Malcolm Macdonald Christos Tachtatzis Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery Remote Sensing cloud removal Sentinel-1 Sentinel-2 deep image prior internal learning image inpainting |
| title | Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery |
| title_full | Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery |
| title_fullStr | Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery |
| title_full_unstemmed | Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery |
| title_short | Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery |
| title_sort | deep internal learning for inpainting of cloud affected regions in satellite imagery |
| topic | cloud removal Sentinel-1 Sentinel-2 deep image prior internal learning image inpainting |
| url | https://www.mdpi.com/2072-4292/14/6/1342 |
| work_keys_str_mv | AT mikolajczerkawski deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT pritiupadhyay deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT christopherdavison deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT astridwerkmeister deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT javiercardona deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT robertatkinson deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT craigmichie deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT ivanandonovic deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT malcolmmacdonald deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery AT christostachtatzis deepinternallearningforinpaintingofcloudaffectedregionsinsatelliteimagery |