THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS
In 2021 EuroSDR initiated a benchmark study with the aim to evaluate the geometric quality of real-world survey data generated from state-of-practice commercial Remotely Piloted Aircraft System (RPAS) photogrammetry (including DJI P4 RTK and DJI P1) and lidar (including DJI L1 and Riegl MiniVUX). Th...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2023-10-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/107/2023/isprs-archives-XLVIII-1-W3-2023-107-2023.pdf |
| _version_ | 1797655731450675200 |
|---|---|
| author | J. P. Mills M. V. Peppa A. Alma'Amari L. Davidson J. Goodyear N. T. Penna |
| author_facet | J. P. Mills M. V. Peppa A. Alma'Amari L. Davidson J. Goodyear N. T. Penna |
| author_sort | J. P. Mills |
| collection | DOAJ |
| description | In 2021 EuroSDR initiated a benchmark study with the aim to evaluate the geometric quality of real-world survey data generated from state-of-practice commercial Remotely Piloted Aircraft System (RPAS) photogrammetry (including DJI P4 RTK and DJI P1) and lidar (including DJI L1 and Riegl MiniVUX). The particular benchmark focus was on achievable data quality from real-world network configurations in the absence of ground control, on-the-fly Real Time Kinematic (RTK) corrections, and/or local GNSS base station information. Successive custom datasets were released to registered benchmark participants who submitted individual outputs that were independently evaluated against reference surveys. Without the inclusion of any supporting ground information, DJI P4 RTK and DJI P1 RPAS solutions were found to deliver m- and dm-level accuracies, respectively, in both plan and height. RTK solutions were found to provide cm-level precisions and accuracies, with some outliers. The introduction of ground control points resulted in similar planimetric accuracy to the RTK solutions, but with slight improvements in height. In terms of lidar datasets, the Riegl MiniVUX solution, using corrections from a local base station, was found to provide smaller discrepancies than the DJI L1 RTK solution, when independently compared against terrestrial laser scanning surveys. This paper provides various quality statistics and demonstrates multiple ways of assessing the geometric quality of RPAS data. The EuroSDR RPAS benchmark datasets are now openly available online in order to support and facilitate further investigation by the community. |
| first_indexed | 2024-03-11T17:18:50Z |
| format | Article |
| id | doaj.art-ab7e0d73623145cd9cc83d8dd67d816e |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-03-11T17:18:50Z |
| publishDate | 2023-10-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-ab7e0d73623145cd9cc83d8dd67d816e2023-10-19T18:03:15ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342023-10-01XLVIII-1-W3-202310711410.5194/isprs-archives-XLVIII-1-W3-2023-107-2023THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTSJ. P. Mills0M. V. Peppa1A. Alma'Amari2L. Davidson3J. Goodyear4N. T. Penna5School of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomSchool of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomSchool of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomSchool of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomSchool of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomSchool of Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, United KingdomIn 2021 EuroSDR initiated a benchmark study with the aim to evaluate the geometric quality of real-world survey data generated from state-of-practice commercial Remotely Piloted Aircraft System (RPAS) photogrammetry (including DJI P4 RTK and DJI P1) and lidar (including DJI L1 and Riegl MiniVUX). The particular benchmark focus was on achievable data quality from real-world network configurations in the absence of ground control, on-the-fly Real Time Kinematic (RTK) corrections, and/or local GNSS base station information. Successive custom datasets were released to registered benchmark participants who submitted individual outputs that were independently evaluated against reference surveys. Without the inclusion of any supporting ground information, DJI P4 RTK and DJI P1 RPAS solutions were found to deliver m- and dm-level accuracies, respectively, in both plan and height. RTK solutions were found to provide cm-level precisions and accuracies, with some outliers. The introduction of ground control points resulted in similar planimetric accuracy to the RTK solutions, but with slight improvements in height. In terms of lidar datasets, the Riegl MiniVUX solution, using corrections from a local base station, was found to provide smaller discrepancies than the DJI L1 RTK solution, when independently compared against terrestrial laser scanning surveys. This paper provides various quality statistics and demonstrates multiple ways of assessing the geometric quality of RPAS data. The EuroSDR RPAS benchmark datasets are now openly available online in order to support and facilitate further investigation by the community.https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/107/2023/isprs-archives-XLVIII-1-W3-2023-107-2023.pdf |
| spellingShingle | J. P. Mills M. V. Peppa A. Alma'Amari L. Davidson J. Goodyear N. T. Penna THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS |
| title_full | THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS |
| title_fullStr | THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS |
| title_full_unstemmed | THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS |
| title_short | THE EUROSDR RPAS BENCHMARK: OPEN DATASET DESCRIPTION AND SUMMARY OF KEY RESULTS |
| title_sort | eurosdr rpas benchmark open dataset description and summary of key results |
| url | https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/107/2023/isprs-archives-XLVIII-1-W3-2023-107-2023.pdf |
| work_keys_str_mv | AT jpmills theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT mvpeppa theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT aalmaamari theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT ldavidson theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT jgoodyear theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT ntpenna theeurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT jpmills eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT mvpeppa eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT aalmaamari eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT ldavidson eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT jgoodyear eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults AT ntpenna eurosdrrpasbenchmarkopendatasetdescriptionandsummaryofkeyresults |