Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction
We present a new approach to gesture recognition that tracks body and hands simultaneously and recognizes gestures continuously from an unseg-mented and unbounded input stream. Our system estimates 3D coordinates of upper body joints and classifies the appearance of hands into a set of canonical sha...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137458 |
| _version_ | 1826214488169250816 |
|---|---|
| author | Song, Yale Demirdjian, David Davis, Randall |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Song, Yale Demirdjian, David Davis, Randall |
| author_sort | Song, Yale |
| collection | MIT |
| description | We present a new approach to gesture recognition that tracks body and hands simultaneously and recognizes gestures continuously from an unseg-mented and unbounded input stream. Our system estimates 3D coordinates of upper body joints and classifies the appearance of hands into a set of canonical shapes. A novel multi-layered filtering technique with a temporal sliding window is developed to enable online sequence labeling and segmentation. Experimental results on the NATOPS dataset show the effectiveness of the approach. We also report on our recent work on multimodal gesture recognition and deep-hierarchical sequence representation learning that achieve the state-of-the-art performances on several real-world datasets. |
| first_indexed | 2024-09-23T16:06:14Z |
| format | Article |
| id | mit-1721.1/137458 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T16:06:14Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | mit-1721.1/1374582022-10-02T06:21:43Z Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction Song, Yale Demirdjian, David Davis, Randall Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory We present a new approach to gesture recognition that tracks body and hands simultaneously and recognizes gestures continuously from an unseg-mented and unbounded input stream. Our system estimates 3D coordinates of upper body joints and classifies the appearance of hands into a set of canonical shapes. A novel multi-layered filtering technique with a temporal sliding window is developed to enable online sequence labeling and segmentation. Experimental results on the NATOPS dataset show the effectiveness of the approach. We also report on our recent work on multimodal gesture recognition and deep-hierarchical sequence representation learning that achieve the state-of-the-art performances on several real-world datasets. 2021-11-05T13:51:15Z 2021-11-05T13:51:15Z 2010 2019-05-17T15:24:44Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/137458 Song, Yale, Demirdjian, David and Davis, Randall. 2010. "Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction." en Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf MIT web domain |
| spellingShingle | Song, Yale Demirdjian, David Davis, Randall Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title | Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title_full | Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title_fullStr | Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title_full_unstemmed | Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title_short | Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction |
| title_sort | continuous body and hand gesture recognition for natural human computer interaction |
| url | https://hdl.handle.net/1721.1/137458 |
| work_keys_str_mv | AT songyale continuousbodyandhandgesturerecognitionfornaturalhumancomputerinteraction AT demirdjiandavid continuousbodyandhandgesturerecognitionfornaturalhumancomputerinteraction AT davisrandall continuousbodyandhandgesturerecognitionfornaturalhumancomputerinteraction |