Real-time continuous gesture recognition for natural multimodal interaction
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2014.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2014
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/91036 |
| _version_ | 1826211390414651392 |
|---|---|
| author | Yin, Ying, Ph. D. Massachusetts Institute of Technology |
| author2 | Randall Davis. |
| author_facet | Randall Davis. Yin, Ying, Ph. D. Massachusetts Institute of Technology |
| author_sort | Yin, Ying, Ph. D. Massachusetts Institute of Technology |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2014. |
| first_indexed | 2024-09-23T15:05:09Z |
| format | Thesis |
| id | mit-1721.1/91036 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T15:05:09Z |
| publishDate | 2014 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/910362019-04-12T09:39:25Z Real-time continuous gesture recognition for natural multimodal interaction Yin, Ying, Ph. D. Massachusetts Institute of Technology Randall Davis. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Electrical Engineering and Computer Science. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2014. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. 81 Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 147-154). I have developed a real-time continuous gesture recognition system capable of dealing with two important problems that have previously been neglected: (a) smoothly handling two different kinds of gestures: those characterized by distinct paths and those characterized by distinct hand poses; and (b) determining how and when the system should respond to gestures. The novel approaches in this thesis include: a probabilistic recognition framework based on a flattened hierarchical hidden Markov model (HHMM) that unifies the recognition of path and pose gestures; and a method of using information from the hidden states in the HMM to identify different gesture phases (the pre-stroke, the nucleus and the post-stroke phases), allowing the system to respond appropriately to both gestures that require a discrete response and those needing a continuous response. The system is extensible: new gestures can be added by recording 3-6 repetitions of the gesture; the system will train an HMM model for the gesture and integrate it into the existing HMM, in a process that takes only a few minutes. Our evaluation shows that even using only a small number of training examples (e.g. 6), the system can achieve an average F1 score of 0.805 for two forms of gestures. To evaluate the performance of my system I collected a new dataset (YANG dataset) that includes both path and pose gestures, offering a combination currently lacking in the community and providing the challenge of recognizing different types of gestures mixed together. I also developed a novel hybrid evaluation metric that is more relevant to real- time interaction with different gesture flows. by Ying Yin. Ph. D. 2014-10-21T16:20:32Z 2014-10-21T16:20:32Z 2014 2014 Thesis http://hdl.handle.net/1721.1/91036 893096468 eng M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 154 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Yin, Ying, Ph. D. Massachusetts Institute of Technology Real-time continuous gesture recognition for natural multimodal interaction |
| title | Real-time continuous gesture recognition for natural multimodal interaction |
| title_full | Real-time continuous gesture recognition for natural multimodal interaction |
| title_fullStr | Real-time continuous gesture recognition for natural multimodal interaction |
| title_full_unstemmed | Real-time continuous gesture recognition for natural multimodal interaction |
| title_short | Real-time continuous gesture recognition for natural multimodal interaction |
| title_sort | real time continuous gesture recognition for natural multimodal interaction |
| topic | Electrical Engineering and Computer Science. |
| url | http://hdl.handle.net/1721.1/91036 |
| work_keys_str_mv | AT yinyingphdmassachusettsinstituteoftechnology realtimecontinuousgesturerecognitionfornaturalmultimodalinteraction |