Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality
Augmented and virtual reality systems (AR/VR) are rapidly becoming key components of the wireless landscape. For immersive AR/VR experience, these devices should be able to construct accurate depth perception of the surrounding environment. Current AR/VR devices rely heavily on using RGB-D depth cam...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9382300/ |
| _version_ | 1818866194048876544 |
|---|---|
| author | Abdelrahman Taha Qi Qu Sam Alex Ping Wang William L. Abbott Ahmed Alkhateeb |
| author_facet | Abdelrahman Taha Qi Qu Sam Alex Ping Wang William L. Abbott Ahmed Alkhateeb |
| author_sort | Abdelrahman Taha |
| collection | DOAJ |
| description | Augmented and virtual reality systems (AR/VR) are rapidly becoming key components of the wireless landscape. For immersive AR/VR experience, these devices should be able to construct accurate depth perception of the surrounding environment. Current AR/VR devices rely heavily on using RGB-D depth cameras to achieve this goal. The performance of these depth cameras, however, has clear limitations in several scenarios, such as the cases with shiny objects, dark surfaces, and abrupt color transition among other limitations. In this paper, we propose a novel solution for AR/VR depth map construction using mmWave MIMO communication transceivers. This is motivated by the deployment of advanced mmWave communication systems in future AR/VR devices for meeting the high data rate demands and by the interesting propagation characteristics of mmWave signals. Accounting for the constraints on these systems, we develop a comprehensive framework for constructing accurate and high-resolution depth maps using mmWave systems. In this framework, we developed new sensing beamforming codebook approaches that are specific for the depth map construction objective. Using these codebooks, and leveraging tools from successive interference cancellation, we develop a joint beam processing approach that can construct high-resolution depth maps using practical mmWave antenna arrays. Extensive simulation results highlight the potential of the proposed solution in building accurate depth maps. Further, these simulations show the promising gains of mmWave based depth perception compared to RGB-based approaches in several important use cases. |
| first_indexed | 2024-12-19T10:59:34Z |
| format | Article |
| id | doaj.art-16a7c78d99c043a6aadd959c30a5299b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T10:59:34Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-16a7c78d99c043a6aadd959c30a5299b2022-12-21T20:24:42ZengIEEEIEEE Access2169-35362021-01-019483414836310.1109/ACCESS.2021.30678399382300Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented RealityAbdelrahman Taha0https://orcid.org/0000-0002-6046-6641Qi Qu1Sam Alex2Ping Wang3William L. Abbott4https://orcid.org/0000-0002-9622-4003Ahmed Alkhateeb5https://orcid.org/0000-0001-5648-1569School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USAFacebook, Inc., Menlo Park, CA, USAFacebook, Inc., Menlo Park, CA, USAFacebook, Inc., Menlo Park, CA, USAFacebook, Inc., Menlo Park, CA, USASchool of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USAAugmented and virtual reality systems (AR/VR) are rapidly becoming key components of the wireless landscape. For immersive AR/VR experience, these devices should be able to construct accurate depth perception of the surrounding environment. Current AR/VR devices rely heavily on using RGB-D depth cameras to achieve this goal. The performance of these depth cameras, however, has clear limitations in several scenarios, such as the cases with shiny objects, dark surfaces, and abrupt color transition among other limitations. In this paper, we propose a novel solution for AR/VR depth map construction using mmWave MIMO communication transceivers. This is motivated by the deployment of advanced mmWave communication systems in future AR/VR devices for meeting the high data rate demands and by the interesting propagation characteristics of mmWave signals. Accounting for the constraints on these systems, we develop a comprehensive framework for constructing accurate and high-resolution depth maps using mmWave systems. In this framework, we developed new sensing beamforming codebook approaches that are specific for the depth map construction objective. Using these codebooks, and leveraging tools from successive interference cancellation, we develop a joint beam processing approach that can construct high-resolution depth maps using practical mmWave antenna arrays. Extensive simulation results highlight the potential of the proposed solution in building accurate depth maps. Further, these simulations show the promising gains of mmWave based depth perception compared to RGB-based approaches in several important use cases.https://ieeexplore.ieee.org/document/9382300/Beamforming codebooksdepth mapsmillimeter wavevirtual reality |
| spellingShingle | Abdelrahman Taha Qi Qu Sam Alex Ping Wang William L. Abbott Ahmed Alkhateeb Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality IEEE Access Beamforming codebooks depth maps millimeter wave virtual reality |
| title | Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality |
| title_full | Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality |
| title_fullStr | Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality |
| title_full_unstemmed | Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality |
| title_short | Millimeter Wave MIMO-Based Depth Maps for Wireless Virtual and Augmented Reality |
| title_sort | millimeter wave mimo based depth maps for wireless virtual and augmented reality |
| topic | Beamforming codebooks depth maps millimeter wave virtual reality |
| url | https://ieeexplore.ieee.org/document/9382300/ |
| work_keys_str_mv | AT abdelrahmantaha millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality AT qiqu millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality AT samalex millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality AT pingwang millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality AT williamlabbott millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality AT ahmedalkhateeb millimeterwavemimobaseddepthmapsforwirelessvirtualandaugmentedreality |