A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting
Earth observation is a growing research area that can capitalize on the powers of artificial intelligence for short time forecasting, a now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been exp...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10285372/ |
| _version_ | 1797448891324432384 |
|---|---|
| author | Alabi Bojesomo Hasan AlMarzouqi Panos Liatsis |
| author_facet | Alabi Bojesomo Hasan AlMarzouqi Panos Liatsis |
| author_sort | Alabi Bojesomo |
| collection | DOAJ |
| description | Earth observation is a growing research area that can capitalize on the powers of artificial intelligence for short time forecasting, a now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of attention and the data-hungry training. To address these issues, we propose the use of video Swin-transformer (VST), coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed approach is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 h ahead future frames (4 per hour) from an hourly weather product sequence. The dataset was normalized to 0–1 to facilitate the use of the evaluation metrics across different datasets. The model results in an mse score of 0.4750 when provided with training data, and 0.4420 during transfer learning without using training data, respectively. |
| first_indexed | 2024-03-09T14:16:58Z |
| format | Article |
| id | doaj.art-9c6870079e214d79b2ccdaa8cc943bfd |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-09T14:16:58Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-9c6870079e214d79b2ccdaa8cc943bfd2023-11-29T00:00:54ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352024-01-0117455510.1109/JSTARS.2023.332372910285372A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather ForecastingAlabi Bojesomo0https://orcid.org/0000-0002-4685-148XHasan AlMarzouqi1https://orcid.org/0000-0002-2826-1515Panos Liatsis2https://orcid.org/0000-0002-5490-6030Department of Electrical Engineering and Computer Sciences, Khalifa University, Abu Dhabi, United Arab EmiratesDepartment of Electrical Engineering and Computer Sciences, Khalifa University, Abu Dhabi, United Arab EmiratesDepartment of Electrical Engineering and Computer Sciences, Khalifa University, Abu Dhabi, United Arab EmiratesEarth observation is a growing research area that can capitalize on the powers of artificial intelligence for short time forecasting, a now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of attention and the data-hungry training. To address these issues, we propose the use of video Swin-transformer (VST), coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed approach is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 h ahead future frames (4 per hour) from an hourly weather product sequence. The dataset was normalized to 0–1 to facilitate the use of the evaluation metrics across different datasets. The model results in an mse score of 0.4750 when provided with training data, and 0.4420 during transfer learning without using training data, respectively.https://ieeexplore.ieee.org/document/10285372/Encoder–decoder video architecturenow-castingshifted window cross attentionvideo Swin-transformer (VST)weather forecasting |
| spellingShingle | Alabi Bojesomo Hasan AlMarzouqi Panos Liatsis A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Encoder–decoder video architecture now-casting shifted window cross attention video Swin-transformer (VST) weather forecasting |
| title | A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting |
| title_full | A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting |
| title_fullStr | A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting |
| title_full_unstemmed | A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting |
| title_short | A Novel Transformer Network With Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting |
| title_sort | novel transformer network with shifted window cross attention for spatiotemporal weather forecasting |
| topic | Encoder–decoder video architecture now-casting shifted window cross attention video Swin-transformer (VST) weather forecasting |
| url | https://ieeexplore.ieee.org/document/10285372/ |
| work_keys_str_mv | AT alabibojesomo anoveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting AT hasanalmarzouqi anoveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting AT panosliatsis anoveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting AT alabibojesomo noveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting AT hasanalmarzouqi noveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting AT panosliatsis noveltransformernetworkwithshiftedwindowcrossattentionforspatiotemporalweatherforecasting |