Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method
Deep-learning-based radar echo extrapolation methods have achieved remarkable progress in the precipitation nowcasting field. However, they suffer from a common notorious problem—they tend to produce blurry predictions. Although some efforts have been made in recent years, the blurring problem is st...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/1/24 |
| _version_ | 1827667764595130368 |
|---|---|
| author | Yuan Hu Lei Chen Zhibin Wang Xiang Pan Hao Li |
| author_facet | Yuan Hu Lei Chen Zhibin Wang Xiang Pan Hao Li |
| author_sort | Yuan Hu |
| collection | DOAJ |
| description | Deep-learning-based radar echo extrapolation methods have achieved remarkable progress in the precipitation nowcasting field. However, they suffer from a common notorious problem—they tend to produce blurry predictions. Although some efforts have been made in recent years, the blurring problem is still under-addressed. In this work, we propose three effective strategies to assist deep-learning-based radar echo extrapolation methods to achieve more realistic and detailed prediction. Specifically, we propose a spatial generative adversarial network (GAN) and a spectrum GAN to improve image fidelity. The spatial and spectrum GANs aim at penalizing the distribution discrepancy between generated and real images from the spatial domain and spectral domain, respectively. In addition, a masked style loss is devised to further enhance the details by transferring the detailed texture of ground truth radar sequences to extrapolated ones. We apply a foreground mask to prevent the background noise from transferring to the outputs. Moreover, we also design a new metric termed the power spectral density score (PSDS) to quantify the perceptual quality from a frequency perspective. The PSDS metric can be applied as a complement to other visual evaluation metrics (e.g., LPIPS) to achieve a comprehensive measurement of image sharpness. We test our approaches with both ConvLSTM baseline and U-Net baseline, and comprehensive ablation experiments on the SEVIR dataset show that the proposed approaches are able to produce much more realistic radar images than baselines. Most notably, our methods can be readily applied to any deep-learning-based spatiotemporal forecasting models to acquire more detailed results. |
| first_indexed | 2024-03-10T03:23:52Z |
| format | Article |
| id | doaj.art-36e2c36370ae49038d79ad68df4281aa |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T03:23:52Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-36e2c36370ae49038d79ad68df4281aa2023-11-23T12:11:57ZengMDPI AGRemote Sensing2072-42922021-12-011412410.3390/rs14010024Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation MethodYuan Hu0Lei Chen1Zhibin Wang2Xiang Pan3Hao Li4DAMO Academy, Alibaba Group, Beijing 100102, ChinaDAMO Academy, Alibaba Group, Beijing 100102, ChinaDAMO Academy, Alibaba Group, Beijing 100102, ChinaDAMO Academy, Alibaba Group, Beijing 100102, ChinaDAMO Academy, Alibaba Group, Beijing 100102, ChinaDeep-learning-based radar echo extrapolation methods have achieved remarkable progress in the precipitation nowcasting field. However, they suffer from a common notorious problem—they tend to produce blurry predictions. Although some efforts have been made in recent years, the blurring problem is still under-addressed. In this work, we propose three effective strategies to assist deep-learning-based radar echo extrapolation methods to achieve more realistic and detailed prediction. Specifically, we propose a spatial generative adversarial network (GAN) and a spectrum GAN to improve image fidelity. The spatial and spectrum GANs aim at penalizing the distribution discrepancy between generated and real images from the spatial domain and spectral domain, respectively. In addition, a masked style loss is devised to further enhance the details by transferring the detailed texture of ground truth radar sequences to extrapolated ones. We apply a foreground mask to prevent the background noise from transferring to the outputs. Moreover, we also design a new metric termed the power spectral density score (PSDS) to quantify the perceptual quality from a frequency perspective. The PSDS metric can be applied as a complement to other visual evaluation metrics (e.g., LPIPS) to achieve a comprehensive measurement of image sharpness. We test our approaches with both ConvLSTM baseline and U-Net baseline, and comprehensive ablation experiments on the SEVIR dataset show that the proposed approaches are able to produce much more realistic radar images than baselines. Most notably, our methods can be readily applied to any deep-learning-based spatiotemporal forecasting models to acquire more detailed results.https://www.mdpi.com/2072-4292/14/1/24realistic radar echo extrapolationgenerative adversarial networksstyle losspower spectral density |
| spellingShingle | Yuan Hu Lei Chen Zhibin Wang Xiang Pan Hao Li Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method Remote Sensing realistic radar echo extrapolation generative adversarial networks style loss power spectral density |
| title | Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method |
| title_full | Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method |
| title_fullStr | Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method |
| title_full_unstemmed | Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method |
| title_short | Towards a More Realistic and Detailed Deep-Learning-Based Radar Echo Extrapolation Method |
| title_sort | towards a more realistic and detailed deep learning based radar echo extrapolation method |
| topic | realistic radar echo extrapolation generative adversarial networks style loss power spectral density |
| url | https://www.mdpi.com/2072-4292/14/1/24 |
| work_keys_str_mv | AT yuanhu towardsamorerealisticanddetaileddeeplearningbasedradarechoextrapolationmethod AT leichen towardsamorerealisticanddetaileddeeplearningbasedradarechoextrapolationmethod AT zhibinwang towardsamorerealisticanddetaileddeeplearningbasedradarechoextrapolationmethod AT xiangpan towardsamorerealisticanddetaileddeeplearningbasedradarechoextrapolationmethod AT haoli towardsamorerealisticanddetaileddeeplearningbasedradarechoextrapolationmethod |