AI-based indoor human gesture recognition using FMCW radar
Human gesture recognition is an emerging necessity of in the industry. How- ever, traditional Computer Vision, which typically uses optical sensors, often fails in this task, due to unstable light conditions and possible blockages. Radar- based sensors, however, are able to detect an object in low-l...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Coursework |
| Language: | English |
| Published: |
Nanyang Technological University
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/161668 |
| _version_ | 1811695251007471616 |
|---|---|
| author | Liu, Boya |
| author2 | Wen Bihan |
| author_facet | Wen Bihan Liu, Boya |
| author_sort | Liu, Boya |
| collection | NTU |
| description | Human gesture recognition is an emerging necessity of in the industry. How- ever, traditional Computer Vision, which typically uses optical sensors, often fails in this task, due to unstable light conditions and possible blockages. Radar- based sensors, however, are able to detect an object in low-light conditions, as well as penetrate through blockages. Therefore, in this project, we focus on radar-based gesture recognition.
We used TI’s AW1642 FMCW radar to collect data from three volunteers, per- forming 7 common gestures in an indoor laboratory setting. A preprocessing pipeline was built to generate Micro-Doppler heatmaps, which describes the mo- tion of different part of human body. These are fed into our carefully-designed deep neural networks to perform gesture classification. We are able to achieve an average accuracy of 82.13% for the 7-gesture classification.
However, there remains an important need to generalize the solution, i.e., to achieve good performance on people not seen in the training set. In our ex- periments, we found the gaps in the same gestures of different people. We proposed to solve this using domain adversarial training. This allows us to ex- tract person/domain-independent features. When testing with unseen individuals, we are able to improve the accuracy by over 8% for 7-gesture classification.
Keywords: radar, FMCW radar, Deep-learning, Micro-Doppler, Gesture Classifi- cation, Domain Adaptation |
| first_indexed | 2024-10-01T07:20:30Z |
| format | Thesis-Master by Coursework |
| id | ntu-10356/161668 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:20:30Z |
| publishDate | 2022 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1616682022-09-14T00:04:11Z AI-based indoor human gesture recognition using FMCW radar Liu, Boya Wen Bihan School of Electrical and Electronic Engineering bihan.wen@ntu.edu.sg Engineering::Electrical and electronic engineering Human gesture recognition is an emerging necessity of in the industry. How- ever, traditional Computer Vision, which typically uses optical sensors, often fails in this task, due to unstable light conditions and possible blockages. Radar- based sensors, however, are able to detect an object in low-light conditions, as well as penetrate through blockages. Therefore, in this project, we focus on radar-based gesture recognition. We used TI’s AW1642 FMCW radar to collect data from three volunteers, per- forming 7 common gestures in an indoor laboratory setting. A preprocessing pipeline was built to generate Micro-Doppler heatmaps, which describes the mo- tion of different part of human body. These are fed into our carefully-designed deep neural networks to perform gesture classification. We are able to achieve an average accuracy of 82.13% for the 7-gesture classification. However, there remains an important need to generalize the solution, i.e., to achieve good performance on people not seen in the training set. In our ex- periments, we found the gaps in the same gestures of different people. We proposed to solve this using domain adversarial training. This allows us to ex- tract person/domain-independent features. When testing with unseen individuals, we are able to improve the accuracy by over 8% for 7-gesture classification. Keywords: radar, FMCW radar, Deep-learning, Micro-Doppler, Gesture Classifi- cation, Domain Adaptation Master of Science (Signal Processing) 2022-09-14T00:04:11Z 2022-09-14T00:04:11Z 2022 Thesis-Master by Coursework Liu, B. (2022). AI-based indoor human gesture recognition using FMCW radar. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/161668 https://hdl.handle.net/10356/161668 en application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering Liu, Boya AI-based indoor human gesture recognition using FMCW radar |
| title | AI-based indoor human gesture recognition using FMCW radar |
| title_full | AI-based indoor human gesture recognition using FMCW radar |
| title_fullStr | AI-based indoor human gesture recognition using FMCW radar |
| title_full_unstemmed | AI-based indoor human gesture recognition using FMCW radar |
| title_short | AI-based indoor human gesture recognition using FMCW radar |
| title_sort | ai based indoor human gesture recognition using fmcw radar |
| topic | Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/161668 |
| work_keys_str_mv | AT liuboya aibasedindoorhumangesturerecognitionusingfmcwradar |