Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods
Sensors onboard satellite platforms with short revisiting periods acquire frequent earth observation data. One limitation to the utility of satellite-based data is missing information in the time series of images due to cloud contamination and sensor malfunction. Most studies on gap-filling and clou...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/19/4692 |
| _version_ | 1827653311826755584 |
|---|---|
| author | Yidan Wang Xuewen Zhou Zurui Ao Kun Xiao Chenxi Yan Qinchuan Xin |
| author_facet | Yidan Wang Xuewen Zhou Zurui Ao Kun Xiao Chenxi Yan Qinchuan Xin |
| author_sort | Yidan Wang |
| collection | DOAJ |
| description | Sensors onboard satellite platforms with short revisiting periods acquire frequent earth observation data. One limitation to the utility of satellite-based data is missing information in the time series of images due to cloud contamination and sensor malfunction. Most studies on gap-filling and cloud removal process individual images, and existing multi-temporal image restoration methods still have problems in dealing with images that have large areas with frequent cloud contamination. Considering these issues, we proposed a deep learning-based method named content-sequence-texture generation (CSTG) network to generate gap-filled time series of images. The method uses deep neural networks to restore remote sensing images with missing information by accounting for image contents, textures and temporal sequences. We designed a content generation network to preliminarily fill in the missing parts and a sequence-texture generation network to optimize the gap-filling outputs. We used time series of Moderate-resolution Imaging Spectroradiometer (MODIS) data in different regions, which include various surface characteristics in North America, Europe and Asia to train and test the proposed model. Compared to the reference images, the CSTG achieved structural similarity (SSIM) of 0.953 and mean absolute errors (MAE) of 0.016 on average for the restored time series of images in artificial experiments. The developed method could restore time series of images with detailed texture and generally performed better than the other comparative methods, especially with large or overlapped missing areas in time series. Our study provides an available method to gap-fill time series of remote sensing images and highlights the power of the deep learning methods in reconstructing remote sensing images. |
| first_indexed | 2024-03-09T21:15:15Z |
| format | Article |
| id | doaj.art-b79fdb16d2e348b8a4cc322df20bf346 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T21:15:15Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-b79fdb16d2e348b8a4cc322df20bf3462023-11-23T21:36:58ZengMDPI AGRemote Sensing2072-42922022-09-011419469210.3390/rs14194692Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based MethodsYidan Wang0Xuewen Zhou1Zurui Ao2Kun Xiao3Chenxi Yan4Qinchuan Xin5School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, ChinaSchool of Atmospheric Sciences, Sun Yat-sen University, Zhuhai 519082, ChinaBeidou Research Institute, Faculty of Engineering, South China Normal University, Foshan 528000, ChinaSchool of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, ChinaSchool of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, ChinaSchool of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, ChinaSensors onboard satellite platforms with short revisiting periods acquire frequent earth observation data. One limitation to the utility of satellite-based data is missing information in the time series of images due to cloud contamination and sensor malfunction. Most studies on gap-filling and cloud removal process individual images, and existing multi-temporal image restoration methods still have problems in dealing with images that have large areas with frequent cloud contamination. Considering these issues, we proposed a deep learning-based method named content-sequence-texture generation (CSTG) network to generate gap-filled time series of images. The method uses deep neural networks to restore remote sensing images with missing information by accounting for image contents, textures and temporal sequences. We designed a content generation network to preliminarily fill in the missing parts and a sequence-texture generation network to optimize the gap-filling outputs. We used time series of Moderate-resolution Imaging Spectroradiometer (MODIS) data in different regions, which include various surface characteristics in North America, Europe and Asia to train and test the proposed model. Compared to the reference images, the CSTG achieved structural similarity (SSIM) of 0.953 and mean absolute errors (MAE) of 0.016 on average for the restored time series of images in artificial experiments. The developed method could restore time series of images with detailed texture and generally performed better than the other comparative methods, especially with large or overlapped missing areas in time series. Our study provides an available method to gap-fill time series of remote sensing images and highlights the power of the deep learning methods in reconstructing remote sensing images.https://www.mdpi.com/2072-4292/14/19/4692gap-fillingdata reconstructioncontent generationsequence-texture generationdeep learning |
| spellingShingle | Yidan Wang Xuewen Zhou Zurui Ao Kun Xiao Chenxi Yan Qinchuan Xin Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods Remote Sensing gap-filling data reconstruction content generation sequence-texture generation deep learning |
| title | Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods |
| title_full | Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods |
| title_fullStr | Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods |
| title_full_unstemmed | Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods |
| title_short | Gap-Filling and Missing Information Recovery for Time Series of MODIS Data Using Deep Learning-Based Methods |
| title_sort | gap filling and missing information recovery for time series of modis data using deep learning based methods |
| topic | gap-filling data reconstruction content generation sequence-texture generation deep learning |
| url | https://www.mdpi.com/2072-4292/14/19/4692 |
| work_keys_str_mv | AT yidanwang gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods AT xuewenzhou gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods AT zuruiao gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods AT kunxiao gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods AT chenxiyan gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods AT qinchuanxin gapfillingandmissinginformationrecoveryfortimeseriesofmodisdatausingdeeplearningbasedmethods |