Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping
Simultaneous Localization and Mapping (SLAM) is being deployed in real-world applications, however many state-of-the-art solutions still struggle in many common scenarios. A key necessity in progressing SLAM research is the availability of high-quality datasets and fair and transparent benchmarking....
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Journal article |
| Langue: | English |
| Publié: |
Institute of Electrical and Electronics Engineers
2022
|
| _version_ | 1826309483148607488 |
|---|---|
| author | Zhang, L Helmberger, M Fu, LFT Wisth, D Camurri, M Scaramuzza, D Fallon, M |
| author_facet | Zhang, L Helmberger, M Fu, LFT Wisth, D Camurri, M Scaramuzza, D Fallon, M |
| author_sort | Zhang, L |
| collection | OXFORD |
| description | Simultaneous Localization and Mapping (SLAM) is being deployed in real-world applications, however many state-of-the-art solutions still struggle in many common scenarios. A key necessity in progressing SLAM research is the availability of high-quality datasets and fair and transparent benchmarking. To this end, we have created the Hilti-Oxford Dataset, to push state-of-the-art SLAM systems to their limits. The dataset has a variety of challenges ranging from sparse and regular construction sites to a 17th century neoclassical building with fine details and curved surfaces. To encourage multi-modal SLAM approaches, we designed a data collection platform featuring a lidar, five cameras, and an IMU (Inertial Measurement Unit). With the goal of benchmarking SLAM algorithms for tasks where accuracy and robustness are paramount, we implemented a novel ground truth collection method that enables our dataset to accurately measure SLAM pose errors with millimeter accuracy. To further ensure accuracy, the extrinsics of our platform were verified with a micrometer-accurate scanner, and temporal calibration was managed online using hardware time synchronization. The multi-modality and diversity of our dataset attracted a large field of academic and industrial researchers to enter the second edition of the Hilti SLAM challenge, which concluded in June 2022. The results of the challenge show that while the top three teams could achieve an accuracy of 2cm or better for some sequences, the performance dropped off in more difficult sequences. |
| first_indexed | 2024-03-07T07:36:29Z |
| format | Journal article |
| id | oxford-uuid:706c3ba5-4a4b-4dfd-b7ce-ecec5bcfbe0b |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:36:29Z |
| publishDate | 2022 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | oxford-uuid:706c3ba5-4a4b-4dfd-b7ce-ecec5bcfbe0b2023-03-10T14:33:20ZHilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mappingJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:706c3ba5-4a4b-4dfd-b7ce-ecec5bcfbe0bEnglishSymplectic ElementsInstitute of Electrical and Electronics Engineers2022Zhang, LHelmberger, MFu, LFTWisth, DCamurri, MScaramuzza, DFallon, MSimultaneous Localization and Mapping (SLAM) is being deployed in real-world applications, however many state-of-the-art solutions still struggle in many common scenarios. A key necessity in progressing SLAM research is the availability of high-quality datasets and fair and transparent benchmarking. To this end, we have created the Hilti-Oxford Dataset, to push state-of-the-art SLAM systems to their limits. The dataset has a variety of challenges ranging from sparse and regular construction sites to a 17th century neoclassical building with fine details and curved surfaces. To encourage multi-modal SLAM approaches, we designed a data collection platform featuring a lidar, five cameras, and an IMU (Inertial Measurement Unit). With the goal of benchmarking SLAM algorithms for tasks where accuracy and robustness are paramount, we implemented a novel ground truth collection method that enables our dataset to accurately measure SLAM pose errors with millimeter accuracy. To further ensure accuracy, the extrinsics of our platform were verified with a micrometer-accurate scanner, and temporal calibration was managed online using hardware time synchronization. The multi-modality and diversity of our dataset attracted a large field of academic and industrial researchers to enter the second edition of the Hilti SLAM challenge, which concluded in June 2022. The results of the challenge show that while the top three teams could achieve an accuracy of 2cm or better for some sequences, the performance dropped off in more difficult sequences. |
| spellingShingle | Zhang, L Helmberger, M Fu, LFT Wisth, D Camurri, M Scaramuzza, D Fallon, M Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title | Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title_full | Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title_fullStr | Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title_full_unstemmed | Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title_short | Hilti-Oxford Dataset: a millimeter-accurate benchmark for simultaneous localization and mapping |
| title_sort | hilti oxford dataset a millimeter accurate benchmark for simultaneous localization and mapping |
| work_keys_str_mv | AT zhangl hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT helmbergerm hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT fulft hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT wisthd hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT camurrim hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT scaramuzzad hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping AT fallonm hiltioxforddatasetamillimeteraccuratebenchmarkforsimultaneouslocalizationandmapping |