Segmentation and Alignment of Speech and Sketching in a Design Environment
Sketches are commonly used in the early stages of design. Our previous system allows users to sketch mechanical systems that the computer interprets. However, some parts of the mechanical system might be too hard or too complicated to express in the sketch. Adding speech recognition to create a...
| Main Author: | |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/7103 |
| _version_ | 1826195356066512896 |
|---|---|
| author | Adler, Aaron D. |
| author_facet | Adler, Aaron D. |
| author_sort | Adler, Aaron D. |
| collection | MIT |
| description | Sketches are commonly used in the early stages of design. Our previous system allows users to sketch mechanical systems that the computer interprets. However, some parts of the mechanical system might be too hard or too complicated to express in the sketch. Adding speech recognition to create a multimodal system would move us toward our goal of creating a more natural user interface. This thesis examines the relationship between the verbal and sketch input, particularly how to segment and align the two inputs. Toward this end, subjects were recorded while they sketched and talked. These recordings were transcribed, and a set of rules to perform segmentation and alignment was created. These rules represent the knowledge that the computer needs to perform segmentation and alignment. The rules successfully interpreted the 24 data sets that they were given. |
| first_indexed | 2024-09-23T10:11:23Z |
| id | mit-1721.1/7103 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T10:11:23Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/71032019-04-12T08:33:56Z Segmentation and Alignment of Speech and Sketching in a Design Environment Adler, Aaron D. AI sketch design multimodal disambiguation segmentation alignment Sketches are commonly used in the early stages of design. Our previous system allows users to sketch mechanical systems that the computer interprets. However, some parts of the mechanical system might be too hard or too complicated to express in the sketch. Adding speech recognition to create a multimodal system would move us toward our goal of creating a more natural user interface. This thesis examines the relationship between the verbal and sketch input, particularly how to segment and align the two inputs. Toward this end, subjects were recorded while they sketched and talked. These recordings were transcribed, and a set of rules to perform segmentation and alignment was created. These rules represent the knowledge that the computer needs to perform segmentation and alignment. The rules successfully interpreted the 24 data sets that they were given. 2004-10-20T20:31:48Z 2004-10-20T20:31:48Z 2003-02-01 AITR-2003-004 http://hdl.handle.net/1721.1/7103 en_US AITR-2003-004 193 p. 34430522 bytes 46149955 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | AI sketch design multimodal disambiguation segmentation alignment Adler, Aaron D. Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title | Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title_full | Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title_fullStr | Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title_full_unstemmed | Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title_short | Segmentation and Alignment of Speech and Sketching in a Design Environment |
| title_sort | segmentation and alignment of speech and sketching in a design environment |
| topic | AI sketch design multimodal disambiguation segmentation alignment |
| url | http://hdl.handle.net/1721.1/7103 |
| work_keys_str_mv | AT adleraarond segmentationandalignmentofspeechandsketchinginadesignenvironment |