Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets
The lack of adequate stereo coverage and where available, lengthy processing time, various artefacts, and unsatisfactory quality and complexity of automating the selection of the best set of processing parameters, have long been big barriers for large-area planetary 3D mapping. In this paper, we pro...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/15/2877 |
| _version_ | 1827686434273755136 |
|---|---|
| author | Yu Tao Siting Xiong Susan J. Conway Jan-Peter Muller Anthony Guimpier Peter Fawdon Nicolas Thomas Gabriele Cremonese |
| author_facet | Yu Tao Siting Xiong Susan J. Conway Jan-Peter Muller Anthony Guimpier Peter Fawdon Nicolas Thomas Gabriele Cremonese |
| author_sort | Yu Tao |
| collection | DOAJ |
| description | The lack of adequate stereo coverage and where available, lengthy processing time, various artefacts, and unsatisfactory quality and complexity of automating the selection of the best set of processing parameters, have long been big barriers for large-area planetary 3D mapping. In this paper, we propose a deep learning-based solution, called MADNet (Multi-scale generative Adversarial u-net with Dense convolutional and up-projection blocks), that avoids or resolves all of the above issues. We demonstrate the wide applicability of this technique with the ExoMars Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) 4.6 m/pixel images on Mars. Only a single input image and a coarse global 3D reference are required, without knowing any camera models or imaging parameters, to produce high-quality and high-resolution full-strip Digital Terrain Models (DTMs) in a few seconds. In this paper, we discuss technical details of the MADNet system and provide detailed comparisons and assessments of the results. The resultant MADNet 8 m/pixel CaSSIS DTMs are qualitatively very similar to the 1 m/pixel HiRISE DTMs. The resultant MADNet CaSSIS DTMs display excellent agreement with nested Mars Reconnaissance Orbiter Context Camera (CTX), Mars Express’s High-Resolution Stereo Camera (HRSC), and Mars Orbiter Laser Altimeter (MOLA) DTMs at large-scale, and meanwhile, show fairly good correlation with the High-Resolution Imaging Science Experiment (HiRISE) DTMs for fine-scale details. In addition, we show how MADNet outperforms traditional photogrammetric methods, both on speed and quality, for other datasets like HRSC, CTX, and HiRISE, without any parameter tuning or re-training of the model. We demonstrate the results for Oxia Planum (the landing site of the European Space Agency’s Rosalind Franklin ExoMars rover 2023) and a couple of sites of high scientific interest. |
| first_indexed | 2024-03-10T09:10:06Z |
| format | Article |
| id | doaj.art-5e479f910cf040c0b6bf7f31d2e5373e |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T09:10:06Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-5e479f910cf040c0b6bf7f31d2e5373e2023-11-22T06:05:29ZengMDPI AGRemote Sensing2072-42922021-07-011315287710.3390/rs13152877Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-NetsYu Tao0Siting Xiong1Susan J. Conway2Jan-Peter Muller3Anthony Guimpier4Peter Fawdon5Nicolas Thomas6Gabriele Cremonese7Mullard Space Science Laboratory, Imaging Group, Department of Space and Climate Physics, University College London, Holmbury St Mary, Surrey RH5 6NT, UKCollege of Civil and Transportation Engineering, Shenzhen University, Shenzhen 518060, ChinaLaboratoire de Planétologie et Géodynamique, CNRS, UMR 6112, Universités de Nantes, 44300 Nantes, FranceMullard Space Science Laboratory, Imaging Group, Department of Space and Climate Physics, University College London, Holmbury St Mary, Surrey RH5 6NT, UKLaboratoire de Planétologie et Géodynamique, CNRS, UMR 6112, Universités de Nantes, 44300 Nantes, FranceSchool of Physical Sciences, Open University, Walton Hall, Milton Keynes MK7 6AA, UKPhysikalisches Institut, Universität Bern, Sidlerstrasse 5, 3012 Bern, SwitzerlandINAF, Osservatorio Astronomico di Padova, 35122 Padova, ItalyThe lack of adequate stereo coverage and where available, lengthy processing time, various artefacts, and unsatisfactory quality and complexity of automating the selection of the best set of processing parameters, have long been big barriers for large-area planetary 3D mapping. In this paper, we propose a deep learning-based solution, called MADNet (Multi-scale generative Adversarial u-net with Dense convolutional and up-projection blocks), that avoids or resolves all of the above issues. We demonstrate the wide applicability of this technique with the ExoMars Trace Gas Orbiter Colour and Stereo Surface Imaging System (CaSSIS) 4.6 m/pixel images on Mars. Only a single input image and a coarse global 3D reference are required, without knowing any camera models or imaging parameters, to produce high-quality and high-resolution full-strip Digital Terrain Models (DTMs) in a few seconds. In this paper, we discuss technical details of the MADNet system and provide detailed comparisons and assessments of the results. The resultant MADNet 8 m/pixel CaSSIS DTMs are qualitatively very similar to the 1 m/pixel HiRISE DTMs. The resultant MADNet CaSSIS DTMs display excellent agreement with nested Mars Reconnaissance Orbiter Context Camera (CTX), Mars Express’s High-Resolution Stereo Camera (HRSC), and Mars Orbiter Laser Altimeter (MOLA) DTMs at large-scale, and meanwhile, show fairly good correlation with the High-Resolution Imaging Science Experiment (HiRISE) DTMs for fine-scale details. In addition, we show how MADNet outperforms traditional photogrammetric methods, both on speed and quality, for other datasets like HRSC, CTX, and HiRISE, without any parameter tuning or re-training of the model. We demonstrate the results for Oxia Planum (the landing site of the European Space Agency’s Rosalind Franklin ExoMars rover 2023) and a couple of sites of high scientific interest.https://www.mdpi.com/2072-4292/13/15/2877DTMdigital terrain modeldeep learning3D mapping3D reconstructionreal-time 3D |
| spellingShingle | Yu Tao Siting Xiong Susan J. Conway Jan-Peter Muller Anthony Guimpier Peter Fawdon Nicolas Thomas Gabriele Cremonese Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets Remote Sensing DTM digital terrain model deep learning 3D mapping 3D reconstruction real-time 3D |
| title | Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets |
| title_full | Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets |
| title_fullStr | Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets |
| title_full_unstemmed | Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets |
| title_short | Rapid Single Image-Based DTM Estimation from ExoMars TGO CaSSIS Images Using Generative Adversarial U-Nets |
| title_sort | rapid single image based dtm estimation from exomars tgo cassis images using generative adversarial u nets |
| topic | DTM digital terrain model deep learning 3D mapping 3D reconstruction real-time 3D |
| url | https://www.mdpi.com/2072-4292/13/15/2877 |
| work_keys_str_mv | AT yutao rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT sitingxiong rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT susanjconway rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT janpetermuller rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT anthonyguimpier rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT peterfawdon rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT nicolasthomas rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets AT gabrielecremonese rapidsingleimagebaseddtmestimationfromexomarstgocassisimagesusinggenerativeadversarialunets |