Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights
Machine Learning-based workflows are being progressively used for the automatic detection of archaeological objects (intended as below-surface sites) in remote sensing data. Despite promising results in the detection phase, there is still a lack of a standard set of measures to evaluate the performa...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/7/1694 |
| _version_ | 1827622577115234304 |
|---|---|
| author | Marco Fiorucci Wouter B. Verschoof-van der Vaart Paolo Soleni Bertrand Le Saux Arianna Traviglia |
| author_facet | Marco Fiorucci Wouter B. Verschoof-van der Vaart Paolo Soleni Bertrand Le Saux Arianna Traviglia |
| author_sort | Marco Fiorucci |
| collection | DOAJ |
| description | Machine Learning-based workflows are being progressively used for the automatic detection of archaeological objects (intended as below-surface sites) in remote sensing data. Despite promising results in the detection phase, there is still a lack of a standard set of measures to evaluate the performance of object detection methods, since buried archaeological sites often have distinctive shapes that set them aside from other types of objects included in mainstream remote sensing datasets (e.g., Dataset of Object deTection in Aerial images, DOTA). Additionally, archaeological research relies heavily on geospatial information when validating the output of an object detection procedure, a type of information that is not normally considered in regular machine learning validation pipelines. This paper tackles these shortcomings by introducing two novel automatic evaluation measures, namely ‘centroid-based’ and ‘pixel-based’, designed to encode the salient aspects of the archaeologists’ thinking process. To test their usability, an experiment with different object detection deep neural networks was conducted on a LiDAR dataset. The experimental results show that these two automatic measures closely resemble the semi-automatic one currently used by archaeologists and therefore can be adopted as fully automatic evaluation measures in archaeological remote sensing detection. Adoption will facilitate cross-study comparisons and close collaboration between machine learning and archaeological researchers, which in turn will encourage the development of novel human-centred archaeological object detection tools. |
| first_indexed | 2024-03-09T11:28:21Z |
| format | Article |
| id | doaj.art-48df27f3aefd4b6eb4b004bdd2aeb4d6 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T11:28:21Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-48df27f3aefd4b6eb4b004bdd2aeb4d62023-11-30T23:57:36ZengMDPI AGRemote Sensing2072-42922022-03-01147169410.3390/rs14071694Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and InsightsMarco Fiorucci0Wouter B. Verschoof-van der Vaart1Paolo Soleni2Bertrand Le Saux3Arianna Traviglia4Center for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, ItalyFaculty of Archaeology, Leiden University, P.O. Box 9514, 2300 RA Leiden, The NetherlandsCenter for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, ItalyESA/ESRIN, ϕ-Lab, 00044 Frascati, ItalyCenter for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, ItalyMachine Learning-based workflows are being progressively used for the automatic detection of archaeological objects (intended as below-surface sites) in remote sensing data. Despite promising results in the detection phase, there is still a lack of a standard set of measures to evaluate the performance of object detection methods, since buried archaeological sites often have distinctive shapes that set them aside from other types of objects included in mainstream remote sensing datasets (e.g., Dataset of Object deTection in Aerial images, DOTA). Additionally, archaeological research relies heavily on geospatial information when validating the output of an object detection procedure, a type of information that is not normally considered in regular machine learning validation pipelines. This paper tackles these shortcomings by introducing two novel automatic evaluation measures, namely ‘centroid-based’ and ‘pixel-based’, designed to encode the salient aspects of the archaeologists’ thinking process. To test their usability, an experiment with different object detection deep neural networks was conducted on a LiDAR dataset. The experimental results show that these two automatic measures closely resemble the semi-automatic one currently used by archaeologists and therefore can be adopted as fully automatic evaluation measures in archaeological remote sensing detection. Adoption will facilitate cross-study comparisons and close collaboration between machine learning and archaeological researchers, which in turn will encourage the development of novel human-centred archaeological object detection tools.https://www.mdpi.com/2072-4292/14/7/1694evaluation measuresmachine learningobject detectionarchaeologyLiDAR |
| spellingShingle | Marco Fiorucci Wouter B. Verschoof-van der Vaart Paolo Soleni Bertrand Le Saux Arianna Traviglia Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights Remote Sensing evaluation measures machine learning object detection archaeology LiDAR |
| title | Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights |
| title_full | Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights |
| title_fullStr | Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights |
| title_full_unstemmed | Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights |
| title_short | Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights |
| title_sort | deep learning for archaeological object detection on lidar new evaluation measures and insights |
| topic | evaluation measures machine learning object detection archaeology LiDAR |
| url | https://www.mdpi.com/2072-4292/14/7/1694 |
| work_keys_str_mv | AT marcofiorucci deeplearningforarchaeologicalobjectdetectiononlidarnewevaluationmeasuresandinsights AT wouterbverschoofvandervaart deeplearningforarchaeologicalobjectdetectiononlidarnewevaluationmeasuresandinsights AT paolosoleni deeplearningforarchaeologicalobjectdetectiononlidarnewevaluationmeasuresandinsights AT bertrandlesaux deeplearningforarchaeologicalobjectdetectiononlidarnewevaluationmeasuresandinsights AT ariannatraviglia deeplearningforarchaeologicalobjectdetectiononlidarnewevaluationmeasuresandinsights |