Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures
Despite the growing availability of self-contained augmented reality head-mounted displays (AR HMDs) based on optical see-through (OST) technology, their potential applications across highly challenging medical and industrial settings are still hampered by the complexity of the display calibration r...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9291377/ |
| _version_ | 1819158731283234816 |
|---|---|
| author | Xue Hu Ferdinando Rodriguez Y Baena Fabrizio Cutolo |
| author_facet | Xue Hu Ferdinando Rodriguez Y Baena Fabrizio Cutolo |
| author_sort | Xue Hu |
| collection | DOAJ |
| description | Despite the growing availability of self-contained augmented reality head-mounted displays (AR HMDs) based on optical see-through (OST) technology, their potential applications across highly challenging medical and industrial settings are still hampered by the complexity of the display calibration required to ensure the locational coherence between the real and virtual elements. The calibration of commercial OST displays remains an open challenge due to the inaccessibility of the user's perspective and the limited hardware information available to the end-user. State-of-the-art calibrations usually comprise both offline and online stages. The offline calibration at a generic viewpoint provides a starting point for the subsequent refinements and it is crucial. Current offline calibration methods either heavily rely on the user-alignment or require complicated hardware calibrations, making the overall procedure subjective and/or tedious. To address this problem, in this work we propose two fully alignment-free calibration methods with less complicated hardware calibration procedures compared with state-of-the-art solutions. The first method employs an eye-replacement camera to compute the rendering camera's projection matrix based on photogrammetry techniques. The second method controls the rendered object position in a tracked 3D space to compensate for the parallax-related misalignment for a generic viewpoint. Both methods have been tested on Microsoft HoloLens 1. Quantitative results show that the average overlay misalignment is fewer than 4 pixels (around 1.5 mm or 9 arcmin) when the target stays within arm's reach. The achieved misalignment is much lower than the HoloLens default interpupillary distance (IPD)-based correction, and equivalent but with lower variance than the Single Point Active Alignment Method (SPAAM)-based calibration. The two proposed methods offer strengths in complementary aspects and can be chosen according to the user's needs. We also provide several update schemes for the two methods that can be integrated for an on-line viewpoint-dependent refinement of the calibration parameters. Both methods have been integrated into a Unity3D-based framework and can be directly applied to Unity-assisted devices. |
| first_indexed | 2024-12-22T16:29:19Z |
| format | Article |
| id | doaj.art-2b88ed57cc1a48a0826c9456d6d9f1e0 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T16:29:19Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-2b88ed57cc1a48a0826c9456d6d9f1e02022-12-21T18:20:06ZengIEEEIEEE Access2169-35362020-01-01822366122367410.1109/ACCESS.2020.30441849291377Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified ProceduresXue Hu0https://orcid.org/0000-0003-4554-061XFerdinando Rodriguez Y Baena1https://orcid.org/0000-0002-5199-9083Fabrizio Cutolo2https://orcid.org/0000-0001-6773-3741Mechatronics in Medicine Laboratory, Imperial College London, London, U.KMechatronics in Medicine Laboratory, Imperial College London, London, U.KDepartment of Information Engineering, University of Pisa, Pisa, ItalyDespite the growing availability of self-contained augmented reality head-mounted displays (AR HMDs) based on optical see-through (OST) technology, their potential applications across highly challenging medical and industrial settings are still hampered by the complexity of the display calibration required to ensure the locational coherence between the real and virtual elements. The calibration of commercial OST displays remains an open challenge due to the inaccessibility of the user's perspective and the limited hardware information available to the end-user. State-of-the-art calibrations usually comprise both offline and online stages. The offline calibration at a generic viewpoint provides a starting point for the subsequent refinements and it is crucial. Current offline calibration methods either heavily rely on the user-alignment or require complicated hardware calibrations, making the overall procedure subjective and/or tedious. To address this problem, in this work we propose two fully alignment-free calibration methods with less complicated hardware calibration procedures compared with state-of-the-art solutions. The first method employs an eye-replacement camera to compute the rendering camera's projection matrix based on photogrammetry techniques. The second method controls the rendered object position in a tracked 3D space to compensate for the parallax-related misalignment for a generic viewpoint. Both methods have been tested on Microsoft HoloLens 1. Quantitative results show that the average overlay misalignment is fewer than 4 pixels (around 1.5 mm or 9 arcmin) when the target stays within arm's reach. The achieved misalignment is much lower than the HoloLens default interpupillary distance (IPD)-based correction, and equivalent but with lower variance than the Single Point Active Alignment Method (SPAAM)-based calibration. The two proposed methods offer strengths in complementary aspects and can be chosen according to the user's needs. We also provide several update schemes for the two methods that can be integrated for an on-line viewpoint-dependent refinement of the calibration parameters. Both methods have been integrated into a Unity3D-based framework and can be directly applied to Unity-assisted devices.https://ieeexplore.ieee.org/document/9291377/Augmented realitydisplay-relative calibrationOST-HMD calibration |
| spellingShingle | Xue Hu Ferdinando Rodriguez Y Baena Fabrizio Cutolo Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures IEEE Access Augmented reality display-relative calibration OST-HMD calibration |
| title | Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures |
| title_full | Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures |
| title_fullStr | Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures |
| title_full_unstemmed | Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures |
| title_short | Alignment-Free Offline Calibration of Commercial Optical See-Through Head-Mounted Displays With Simplified Procedures |
| title_sort | alignment free offline calibration of commercial optical see through head mounted displays with simplified procedures |
| topic | Augmented reality display-relative calibration OST-HMD calibration |
| url | https://ieeexplore.ieee.org/document/9291377/ |
| work_keys_str_mv | AT xuehu alignmentfreeofflinecalibrationofcommercialopticalseethroughheadmounteddisplayswithsimplifiedprocedures AT ferdinandorodriguezybaena alignmentfreeofflinecalibrationofcommercialopticalseethroughheadmounteddisplayswithsimplifiedprocedures AT fabriziocutolo alignmentfreeofflinecalibrationofcommercialopticalseethroughheadmounteddisplayswithsimplifiedprocedures |