Context-based visual feedback recognition
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2007.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2007
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/38686 |
| _version_ | 1826193967826337792 |
|---|---|
| author | Morency, Louis-Philippe, 1977- |
| author2 | Trevor Darrell. |
| author_facet | Trevor Darrell. Morency, Louis-Philippe, 1977- |
| author_sort | Morency, Louis-Philippe, 1977- |
| collection | MIT |
| description | Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2007. |
| first_indexed | 2024-09-23T09:47:48Z |
| format | Thesis |
| id | mit-1721.1/38686 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T09:47:48Z |
| publishDate | 2007 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/386862019-04-10T15:33:45Z Context-based visual feedback recognition Morency, Louis-Philippe, 1977- Trevor Darrell. Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. Electrical Engineering and Computer Science. Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2007. Includes bibliographical references (p. 183-195). During face-to-face conversation, people use visual feedback (e.g., head and eye gesture) to communicate relevant information and to synchronize rhythm between participants. When recognizing visual feedback, people often rely on more than their visual perception. For instance, knowledge about the current topic and from previous utterances help guide the recognition of nonverbal cues. The goal of this thesis is to augment computer interfaces with the ability to perceive visual feedback gestures and to enable the exploitation of contextual information from the current interaction state to improve visual feedback recognition. We introduce the concept of visual feedback anticipation where contextual knowledge from an interactive system (e.g. last spoken utterance from the robot or system events from the GUI interface) is analyzed online to anticipate visual feedback from a human participant and improve visual feedback recognition. Our multi-modal framework for context-based visual feedback recognition was successfully tested on conversational and non-embodied interfaces for head and eye gesture recognition. We also introduce Frame-based Hidden-state Conditional Random Field model, a new discriminative model for visual gesture recognition which can model the substructure of a gesture sequence, learn the dynamics between gesture labels, and can be directly applied to label unsegmented sequences. The FHCRF model outperforms previous approaches (i.e. HMM, SVM and CRF) for visual gesture recognition and can efficiently learn relevant contextual information necessary for visual feedback anticipation. A real-time visual feedback recognition library for interactive interfaces (called Watson) was developed to recognize head gaze, head gestures, and eye gaze using the images from a monocular or stereo camera and the context information from the interactive system. Watson was downloaded by more then 70 researchers around the world and was successfully used by MERL, USC, NTT, MIT Media Lab and many other research groups. by Louis-Philippe Morency. Ph.D. 2007-08-29T20:44:55Z 2007-08-29T20:44:55Z 2006 2007 Thesis http://hdl.handle.net/1721.1/38686 164437423 eng M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 195 p. application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Morency, Louis-Philippe, 1977- Context-based visual feedback recognition |
| title | Context-based visual feedback recognition |
| title_full | Context-based visual feedback recognition |
| title_fullStr | Context-based visual feedback recognition |
| title_full_unstemmed | Context-based visual feedback recognition |
| title_short | Context-based visual feedback recognition |
| title_sort | context based visual feedback recognition |
| topic | Electrical Engineering and Computer Science. |
| url | http://hdl.handle.net/1721.1/38686 |
| work_keys_str_mv | AT morencylouisphilippe1977 contextbasedvisualfeedbackrecognition |