Through-Wall Human Mesh Recovery Using Radio Signals
This paper presents RF-Avatar, a neural network model that can estimate 3D meshes of the human body in the presence of occlusions, baggy clothes, and bad lighting conditions. We leverage that radio frequency (RF) signals in the WiFi range traverse clothes and occlusions and bounce off the human body...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/129671 |
| _version_ | 1826205580392398848 |
|---|---|
| author | Zhao, Mingmin Liu, Yingcheng Raghu, Aniruddh Zhao, Hang Li, Tianhong Torralba, Antonio Katabi, Dina |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Zhao, Mingmin Liu, Yingcheng Raghu, Aniruddh Zhao, Hang Li, Tianhong Torralba, Antonio Katabi, Dina |
| author_sort | Zhao, Mingmin |
| collection | MIT |
| description | This paper presents RF-Avatar, a neural network model that can estimate 3D meshes of the human body in the presence of occlusions, baggy clothes, and bad lighting conditions. We leverage that radio frequency (RF) signals in the WiFi range traverse clothes and occlusions and bounce off the human body. Our model parses such radio signals and recovers 3D body meshes. Our meshes are dynamic and smoothly track the movements of the corresponding people. Further, our model works both in single and multi-person scenarios. Inferring body meshes from radio signals is a highly under-constrained problem. Our model deals with this challenge using: 1) a combination of strong and weak supervision, 2) a multi-headed self-attention mechanism that attends differently to temporal information in the radio signal, and 3) an adversarially trained temporal discriminator that imposes a prior on the dynamics of human motion. Our results show that RF-Avatar accurately recovers dynamic 3D meshes in the presence of occlusions, baggy clothes, bad lighting conditions, and even through walls. |
| first_indexed | 2024-09-23T13:15:23Z |
| format | Article |
| id | mit-1721.1/129671 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:15:23Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1296712022-09-28T12:58:18Z Through-Wall Human Mesh Recovery Using Radio Signals Zhao, Mingmin Liu, Yingcheng Raghu, Aniruddh Zhao, Hang Li, Tianhong Torralba, Antonio Katabi, Dina Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science This paper presents RF-Avatar, a neural network model that can estimate 3D meshes of the human body in the presence of occlusions, baggy clothes, and bad lighting conditions. We leverage that radio frequency (RF) signals in the WiFi range traverse clothes and occlusions and bounce off the human body. Our model parses such radio signals and recovers 3D body meshes. Our meshes are dynamic and smoothly track the movements of the corresponding people. Further, our model works both in single and multi-person scenarios. Inferring body meshes from radio signals is a highly under-constrained problem. Our model deals with this challenge using: 1) a combination of strong and weak supervision, 2) a multi-headed self-attention mechanism that attends differently to temporal information in the radio signal, and 3) an adversarially trained temporal discriminator that imposes a prior on the dynamics of human motion. Our results show that RF-Avatar accurately recovers dynamic 3D meshes in the presence of occlusions, baggy clothes, bad lighting conditions, and even through walls. 2021-02-03T23:08:49Z 2021-02-03T23:08:49Z 2020-02 2019-10 2020-12-23T16:33:54Z Article http://purl.org/eprint/type/ConferencePaper 9781728148038 2380-7504 https://hdl.handle.net/1721.1/129671 Zhao, Mingmin et al. "Through-Wall Human Mesh Recovery Using Radio Signals." 2019 IEEE/CVF International Conference on Computer Vision (ICCV), October 2019, Seoul, Korea, Institute of Electrical and Electronics Engineers, February 2020 © 2019 IEEE en http://dx.doi.org/10.1109/iccv.2019.01021 2019 IEEE/CVF International Conference on Computer Vision (ICCV) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Zhao, Mingmin Liu, Yingcheng Raghu, Aniruddh Zhao, Hang Li, Tianhong Torralba, Antonio Katabi, Dina Through-Wall Human Mesh Recovery Using Radio Signals |
| title | Through-Wall Human Mesh Recovery Using Radio Signals |
| title_full | Through-Wall Human Mesh Recovery Using Radio Signals |
| title_fullStr | Through-Wall Human Mesh Recovery Using Radio Signals |
| title_full_unstemmed | Through-Wall Human Mesh Recovery Using Radio Signals |
| title_short | Through-Wall Human Mesh Recovery Using Radio Signals |
| title_sort | through wall human mesh recovery using radio signals |
| url | https://hdl.handle.net/1721.1/129671 |
| work_keys_str_mv | AT zhaomingmin throughwallhumanmeshrecoveryusingradiosignals AT liuyingcheng throughwallhumanmeshrecoveryusingradiosignals AT raghuaniruddh throughwallhumanmeshrecoveryusingradiosignals AT zhaohang throughwallhumanmeshrecoveryusingradiosignals AT litianhong throughwallhumanmeshrecoveryusingradiosignals AT torralbaantonio throughwallhumanmeshrecoveryusingradiosignals AT katabidina throughwallhumanmeshrecoveryusingradiosignals |