Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis
Remote sensing images play a crucial role in remote sensing target detection and 3D remote sensing modeling, and the enhancement of resolution holds significant application implications. The task of remote sensing target detection requires a substantial amount of high-resolution remote sensing image...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/16/3920 |
| _version_ | 1797583380468989952 |
|---|---|
| author | Junwei Lv Jiayi Guo Yueting Zhang Xin Zhao Bin Lei |
| author_facet | Junwei Lv Jiayi Guo Yueting Zhang Xin Zhao Bin Lei |
| author_sort | Junwei Lv |
| collection | DOAJ |
| description | Remote sensing images play a crucial role in remote sensing target detection and 3D remote sensing modeling, and the enhancement of resolution holds significant application implications. The task of remote sensing target detection requires a substantial amount of high-resolution remote sensing images, while 3D reconstruction tasks generate denser models from diverse view perspectives. However, high-resolution remote sensing images are often limited due to their high acquisition costs, a scarcity of acquisition views, and restricted view perspective variations, which pose challenges for remote sensing tasks. In this paper, we propose an advanced method for a high-resolution remote sensing novel view synthesis by integrating attention mechanisms with neural radiance fields to address the scarcity of high-resolution remote sensing images. To enhance the relationships between sampled points and rays and to improve the 3D implicit model representation capability of the network, we introduce a point attention module and batch attention module into the proposed framework. Additionally, a frequency-weighted position encoding strategy is proposed to determine the significance of each frequency for position encoding. The proposed method is evaluated on the LEVIR-NVS dataset and demonstrates superior performance in quality assessment metrics and visual effects compared to baseline NeRF (Neural Radiance Fields) and ImMPI (Implicit Multi-plane Images). Overall, this work presents a promising approach for a remote sensing novel view synthesis by leveraging attention mechanisms and frequency-weighted position encoding. |
| first_indexed | 2024-03-10T23:38:09Z |
| format | Article |
| id | doaj.art-f22e4c04a4ba4bdd9e6fd6ef08e07df3 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T23:38:09Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-f22e4c04a4ba4bdd9e6fd6ef08e07df32023-11-19T02:51:59ZengMDPI AGRemote Sensing2072-42922023-08-011516392010.3390/rs15163920Neural Radiance Fields for High-Resolution Remote Sensing Novel View SynthesisJunwei Lv0Jiayi Guo1Yueting Zhang2Xin Zhao3Bin Lei4Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaRemote sensing images play a crucial role in remote sensing target detection and 3D remote sensing modeling, and the enhancement of resolution holds significant application implications. The task of remote sensing target detection requires a substantial amount of high-resolution remote sensing images, while 3D reconstruction tasks generate denser models from diverse view perspectives. However, high-resolution remote sensing images are often limited due to their high acquisition costs, a scarcity of acquisition views, and restricted view perspective variations, which pose challenges for remote sensing tasks. In this paper, we propose an advanced method for a high-resolution remote sensing novel view synthesis by integrating attention mechanisms with neural radiance fields to address the scarcity of high-resolution remote sensing images. To enhance the relationships between sampled points and rays and to improve the 3D implicit model representation capability of the network, we introduce a point attention module and batch attention module into the proposed framework. Additionally, a frequency-weighted position encoding strategy is proposed to determine the significance of each frequency for position encoding. The proposed method is evaluated on the LEVIR-NVS dataset and demonstrates superior performance in quality assessment metrics and visual effects compared to baseline NeRF (Neural Radiance Fields) and ImMPI (Implicit Multi-plane Images). Overall, this work presents a promising approach for a remote sensing novel view synthesis by leveraging attention mechanisms and frequency-weighted position encoding.https://www.mdpi.com/2072-4292/15/16/3920novel view synthesisneural radiance fieldsremote sensingattentionvolume rendering |
| spellingShingle | Junwei Lv Jiayi Guo Yueting Zhang Xin Zhao Bin Lei Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis Remote Sensing novel view synthesis neural radiance fields remote sensing attention volume rendering |
| title | Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis |
| title_full | Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis |
| title_fullStr | Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis |
| title_full_unstemmed | Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis |
| title_short | Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis |
| title_sort | neural radiance fields for high resolution remote sensing novel view synthesis |
| topic | novel view synthesis neural radiance fields remote sensing attention volume rendering |
| url | https://www.mdpi.com/2072-4292/15/16/3920 |
| work_keys_str_mv | AT junweilv neuralradiancefieldsforhighresolutionremotesensingnovelviewsynthesis AT jiayiguo neuralradiancefieldsforhighresolutionremotesensingnovelviewsynthesis AT yuetingzhang neuralradiancefieldsforhighresolutionremotesensingnovelviewsynthesis AT xinzhao neuralradiancefieldsforhighresolutionremotesensingnovelviewsynthesis AT binlei neuralradiancefieldsforhighresolutionremotesensingnovelviewsynthesis |