Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage
A key necessity for the safe and autonomous flight of Unmanned Aircraft Systems (UAS) is their reliable perception of the environment, for example, to assess the safety of a landing site. For visual perception, Machine Learning (ML) provides state-of-the-art results in terms of performance, but the...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/4/1144 |
| _version_ | 1797296977329782784 |
|---|---|
| author | Joachim Rüter Theresa Maienschein Sebastian Schirmer Simon Schopferer Christoph Torens |
| author_facet | Joachim Rüter Theresa Maienschein Sebastian Schirmer Simon Schopferer Christoph Torens |
| author_sort | Joachim Rüter |
| collection | DOAJ |
| description | A key necessity for the safe and autonomous flight of Unmanned Aircraft Systems (UAS) is their reliable perception of the environment, for example, to assess the safety of a landing site. For visual perception, Machine Learning (ML) provides state-of-the-art results in terms of performance, but the path to aviation certification has yet to be determined as current regulation and standard documents are not applicable to ML-based components due to their data-defined properties. However, the European Union Aviation Safety Agency (EASA) published the first usable guidance documents that take ML-specific challenges, such as data management and learning assurance, into account. In this paper, an important concept in this context is addressed, namely the Operational Design Domain (ODD) that defines the limitations under which a given ML-based system is designed to operate and function correctly. We investigated whether synthetic data can be used to complement a real-world training dataset which does not cover the whole ODD of an ML-based system component for visual object detection. The use-case in focus is the detection of humans on the ground to assess the safety of landing sites. Synthetic data are generated using the methods proposed in the EASA documents, namely augmentations, stitching and simulation environments. These data are used to augment a real-world dataset to increase ODD coverage during the training of Faster R-CNN object detection models. Our results give insights into the generation techniques and usefulness of synthetic data in the context of increasing ODD coverage. They indicate that the different types of synthetic images vary in their suitability but that augmentations seem to be particularly promising when there is not enough real-world data to cover the whole ODD. By doing so, our results contribute towards the adoption of ML technology in aviation and the reduction of data requirements for ML perception systems. |
| first_indexed | 2024-03-07T22:14:34Z |
| format | Article |
| id | doaj.art-04a0491348e74b3fb856b90773b8a9b4 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-07T22:14:34Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-04a0491348e74b3fb856b90773b8a9b42024-02-23T15:33:41ZengMDPI AGSensors1424-82202024-02-01244114410.3390/s24041144Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain CoverageJoachim Rüter0Theresa Maienschein1Sebastian Schirmer2Simon Schopferer3Christoph Torens4German Aerospace Center (DLR), Institute of Flight Systems, 38108 Braunschweig, GermanyGerman Aerospace Center (DLR), Institute of Flight Systems, 38108 Braunschweig, GermanyGerman Aerospace Center (DLR), Institute of Flight Systems, 38108 Braunschweig, GermanyGerman Aerospace Center (DLR), Institute of Flight Systems, 38108 Braunschweig, GermanyGerman Aerospace Center (DLR), Institute of Flight Systems, 38108 Braunschweig, GermanyA key necessity for the safe and autonomous flight of Unmanned Aircraft Systems (UAS) is their reliable perception of the environment, for example, to assess the safety of a landing site. For visual perception, Machine Learning (ML) provides state-of-the-art results in terms of performance, but the path to aviation certification has yet to be determined as current regulation and standard documents are not applicable to ML-based components due to their data-defined properties. However, the European Union Aviation Safety Agency (EASA) published the first usable guidance documents that take ML-specific challenges, such as data management and learning assurance, into account. In this paper, an important concept in this context is addressed, namely the Operational Design Domain (ODD) that defines the limitations under which a given ML-based system is designed to operate and function correctly. We investigated whether synthetic data can be used to complement a real-world training dataset which does not cover the whole ODD of an ML-based system component for visual object detection. The use-case in focus is the detection of humans on the ground to assess the safety of landing sites. Synthetic data are generated using the methods proposed in the EASA documents, namely augmentations, stitching and simulation environments. These data are used to augment a real-world dataset to increase ODD coverage during the training of Faster R-CNN object detection models. Our results give insights into the generation techniques and usefulness of synthetic data in the context of increasing ODD coverage. They indicate that the different types of synthetic images vary in their suitability but that augmentations seem to be particularly promising when there is not enough real-world data to cover the whole ODD. By doing so, our results contribute towards the adoption of ML technology in aviation and the reduction of data requirements for ML perception systems.https://www.mdpi.com/1424-8220/24/4/1144operational design domain (ODD)environment perceptionobject detectionmachine learningsynthetic imagesgame engine |
| spellingShingle | Joachim Rüter Theresa Maienschein Sebastian Schirmer Simon Schopferer Christoph Torens Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage Sensors operational design domain (ODD) environment perception object detection machine learning synthetic images game engine |
| title | Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage |
| title_full | Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage |
| title_fullStr | Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage |
| title_full_unstemmed | Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage |
| title_short | Filling the Gaps: Using Synthetic Low-Altitude Aerial Images to Increase Operational Design Domain Coverage |
| title_sort | filling the gaps using synthetic low altitude aerial images to increase operational design domain coverage |
| topic | operational design domain (ODD) environment perception object detection machine learning synthetic images game engine |
| url | https://www.mdpi.com/1424-8220/24/4/1144 |
| work_keys_str_mv | AT joachimruter fillingthegapsusingsyntheticlowaltitudeaerialimagestoincreaseoperationaldesigndomaincoverage AT theresamaienschein fillingthegapsusingsyntheticlowaltitudeaerialimagestoincreaseoperationaldesigndomaincoverage AT sebastianschirmer fillingthegapsusingsyntheticlowaltitudeaerialimagestoincreaseoperationaldesigndomaincoverage AT simonschopferer fillingthegapsusingsyntheticlowaltitudeaerialimagestoincreaseoperationaldesigndomaincoverage AT christophtorens fillingthegapsusingsyntheticlowaltitudeaerialimagestoincreaseoperationaldesigndomaincoverage |