Tangible digital twin with shared visualization for collaborative air traffic management operations
In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, a...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/174715 |
| _version_ | 1811689518402633728 |
|---|---|
| author | Chen, Ken Nadirsha, Thaivalappil N. M. Lilith, Nimrod Alam, Sameer Svensson, Åsa |
| author2 | School of Mechanical and Aerospace Engineering |
| author_facet | School of Mechanical and Aerospace Engineering Chen, Ken Nadirsha, Thaivalappil N. M. Lilith, Nimrod Alam, Sameer Svensson, Åsa |
| author_sort | Chen, Ken |
| collection | NTU |
| description | In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, and real-time simulation, there is currently a lack of research on visualization and interaction design of digital twin from human factors’ perspective. Moreover, recent advances in virtual and mixed reality (VR/MR) technology give potential for designing interactive digital twin in 3D space that can assist human operators in performing their duties. Therefore, this research aims to explore visualization and interaction design of an MR-based digital twin prototype that can assist air traffic controllers (ATCOs) in carrying out ground control tasks. Specifically, MR headsets (Microsoft HoloLens 2) are leveraged to project out-of-tower view of airport traffic onto a 3D printed airport model at a 1:1 scale. The spatially aligned tangible system enables ATCOs to perform typical ATM operations by directly touching the 3D printed airport model. In addition, collaborative features are implemented into the system, which enable multiple ATCOs to have a shared view of the airport traffic and collaboratively perform ATM operations when wearing the MR headsets. Finally, a user study was conducted with the recruitment of ten licensed ATCOs to investigate usability of the system and ATCOs’ workload, situational awareness and trust when using the system. The feedback from the ATCOs in the user study suggested that: (1) The visual and interactive design were acceptable; (2) The ATCOs indicated varied workload under different scenarios; (3) The ATCOs developed a high level of shared situational awareness with shared view of the system; and (4) The ATCOs were open and confident to use the system in its’ mature form. The findings show that the MR-based tangible airport digital tower system has high potential in collaborative ATM operations by improving user performance and user experience. |
| first_indexed | 2024-10-01T05:49:22Z |
| format | Journal Article |
| id | ntu-10356/174715 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:49:22Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1747152024-04-13T16:49:01Z Tangible digital twin with shared visualization for collaborative air traffic management operations Chen, Ken Nadirsha, Thaivalappil N. M. Lilith, Nimrod Alam, Sameer Svensson, Åsa School of Mechanical and Aerospace Engineering Saab-NTU Joint Lab Engineering Mixed reality Air traffic management In recent years, digital twin technology has become increasingly popular and been widely applied across different industries including air traffic management (ATM). While previous studies on digital twin in ATM focused on factors such as system architecture, synthetic data generation and transfer, and real-time simulation, there is currently a lack of research on visualization and interaction design of digital twin from human factors’ perspective. Moreover, recent advances in virtual and mixed reality (VR/MR) technology give potential for designing interactive digital twin in 3D space that can assist human operators in performing their duties. Therefore, this research aims to explore visualization and interaction design of an MR-based digital twin prototype that can assist air traffic controllers (ATCOs) in carrying out ground control tasks. Specifically, MR headsets (Microsoft HoloLens 2) are leveraged to project out-of-tower view of airport traffic onto a 3D printed airport model at a 1:1 scale. The spatially aligned tangible system enables ATCOs to perform typical ATM operations by directly touching the 3D printed airport model. In addition, collaborative features are implemented into the system, which enable multiple ATCOs to have a shared view of the airport traffic and collaboratively perform ATM operations when wearing the MR headsets. Finally, a user study was conducted with the recruitment of ten licensed ATCOs to investigate usability of the system and ATCOs’ workload, situational awareness and trust when using the system. The feedback from the ATCOs in the user study suggested that: (1) The visual and interactive design were acceptable; (2) The ATCOs indicated varied workload under different scenarios; (3) The ATCOs developed a high level of shared situational awareness with shared view of the system; and (4) The ATCOs were open and confident to use the system in its’ mature form. The findings show that the MR-based tangible airport digital tower system has high potential in collaborative ATM operations by improving user performance and user experience. Submitted/Accepted version 2024-04-08T05:50:21Z 2024-04-08T05:50:21Z 2024 Journal Article Chen, K., Nadirsha, T. N. M., Lilith, N., Alam, S. & Svensson, Å. (2024). Tangible digital twin with shared visualization for collaborative air traffic management operations. Transportation Research Part C: Emerging Technologies, 161, 104546-. https://dx.doi.org/10.1016/j.trc.2024.104546 0968-090X https://hdl.handle.net/10356/174715 10.1016/j.trc.2024.104546 2-s2.0-85187139937 161 104546 en Transportation Research Part C: Emerging Technologies © 2024 Elsevier Ltd. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1016/j.trc.2024.104546. application/pdf |
| spellingShingle | Engineering Mixed reality Air traffic management Chen, Ken Nadirsha, Thaivalappil N. M. Lilith, Nimrod Alam, Sameer Svensson, Åsa Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title | Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title_full | Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title_fullStr | Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title_full_unstemmed | Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title_short | Tangible digital twin with shared visualization for collaborative air traffic management operations |
| title_sort | tangible digital twin with shared visualization for collaborative air traffic management operations |
| topic | Engineering Mixed reality Air traffic management |
| url | https://hdl.handle.net/10356/174715 |
| work_keys_str_mv | AT chenken tangibledigitaltwinwithsharedvisualizationforcollaborativeairtrafficmanagementoperations AT nadirshathaivalappilnm tangibledigitaltwinwithsharedvisualizationforcollaborativeairtrafficmanagementoperations AT lilithnimrod tangibledigitaltwinwithsharedvisualizationforcollaborativeairtrafficmanagementoperations AT alamsameer tangibledigitaltwinwithsharedvisualizationforcollaborativeairtrafficmanagementoperations AT svenssonasa tangibledigitaltwinwithsharedvisualizationforcollaborativeairtrafficmanagementoperations |