Creating the Perception-based LADDER sketch recognition language
Sketch recognition is automated understanding of hand-drawn diagrams. Current sketch recognition systems exist for only a handful of domains, which contain on the order of 10--20 shapes. Our goal was to create a generalized method for recognition that could work for many domains, increasing the numb...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2012
|
| Online Access: | http://hdl.handle.net/1721.1/71125 https://orcid.org/0000-0001-5232-7281 |
| _version_ | 1811088868228726784 |
|---|---|
| author | Hammond, Tracy A. Davis, Randall |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Hammond, Tracy A. Davis, Randall |
| author_sort | Hammond, Tracy A. |
| collection | MIT |
| description | Sketch recognition is automated understanding of hand-drawn diagrams. Current sketch recognition systems exist for only a handful of domains, which contain on the order of 10--20 shapes. Our goal was to create a generalized method for recognition that could work for many domains, increasing the number of shapes that could be recognized in real-time, while maintaining a high accuracy. In an effort to effectively recognize shapes while allowing drawing freedom (both drawing-style freedom and perceptually-valid variations), we created the shape description language modeled after the way people naturally describe shapes to 1) create an intuitive and easy to understand description, providing transparency to the underlying recognition process, and 2) to improve recognition by providing recognition flexibility (drawing freedom) that is aligned with how humans perceive shapes. This paper describes the results of a study performed to see how users naturally describe shapes. A sample of 35 subjects described or drew approximately 16 shapes each. Results show a common vocabulary related to Gestalt grouping and singularities. Results also show that perception, similarity, and context play an important role in how people describe shapes. This study resulted in a language (LADDER) that allows shape recognizers for any domain to be automatically generated from a single hand-drawn example of each shape. Sketch systems for over 30 different domains have been automatically generated based on this language. The largest domain contained 923 distinct shapes, and achieved a recognition accuracy of 83% (and a top-3 accuracy of 87%) on a corpus of over 11,000 sketches, which recognizes almost two orders of magnitude more shapes than any other existing system. |
| first_indexed | 2024-09-23T14:08:40Z |
| format | Article |
| id | mit-1721.1/71125 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:08:40Z |
| publishDate | 2012 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/711252022-09-28T18:50:36Z Creating the Perception-based LADDER sketch recognition language Hammond, Tracy A. Davis, Randall Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Davis, Randall Davis, Randall Sketch recognition is automated understanding of hand-drawn diagrams. Current sketch recognition systems exist for only a handful of domains, which contain on the order of 10--20 shapes. Our goal was to create a generalized method for recognition that could work for many domains, increasing the number of shapes that could be recognized in real-time, while maintaining a high accuracy. In an effort to effectively recognize shapes while allowing drawing freedom (both drawing-style freedom and perceptually-valid variations), we created the shape description language modeled after the way people naturally describe shapes to 1) create an intuitive and easy to understand description, providing transparency to the underlying recognition process, and 2) to improve recognition by providing recognition flexibility (drawing freedom) that is aligned with how humans perceive shapes. This paper describes the results of a study performed to see how users naturally describe shapes. A sample of 35 subjects described or drew approximately 16 shapes each. Results show a common vocabulary related to Gestalt grouping and singularities. Results also show that perception, similarity, and context play an important role in how people describe shapes. This study resulted in a language (LADDER) that allows shape recognizers for any domain to be automatically generated from a single hand-drawn example of each shape. Sketch systems for over 30 different domains have been automatically generated based on this language. The largest domain contained 923 distinct shapes, and achieved a recognition accuracy of 83% (and a top-3 accuracy of 87%) on a corpus of over 11,000 sketches, which recognizes almost two orders of magnitude more shapes than any other existing system. National Science Foundation (U.S.) (grant 0757557) National Science Foundation (U.S.) (grant 0943499) 2012-06-08T15:25:56Z 2012-06-08T15:25:56Z 2010-08 2010-02 Article http://purl.org/eprint/type/ConferencePaper 978-1-4503-0103-9 http://hdl.handle.net/1721.1/71125 Tracy Hammond and Randall Davis. 2010. Creating the perception-based LADDER sketch recognition language. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (DIS '10). ACM, New York, NY, USA, 141-150. DOI=10.1145/1858171.1858197 http://doi.acm.org/10.1145/1858171.1858197 https://orcid.org/0000-0001-5232-7281 en_US http://dx.doi.org/10.1145/1858171.1858197 Proceedings of the 8th ACM Conference on Designing Interactive Systems Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for Computing Machinery (ACM) Davis via Amy Stout |
| spellingShingle | Hammond, Tracy A. Davis, Randall Creating the Perception-based LADDER sketch recognition language |
| title | Creating the Perception-based LADDER sketch recognition language |
| title_full | Creating the Perception-based LADDER sketch recognition language |
| title_fullStr | Creating the Perception-based LADDER sketch recognition language |
| title_full_unstemmed | Creating the Perception-based LADDER sketch recognition language |
| title_short | Creating the Perception-based LADDER sketch recognition language |
| title_sort | creating the perception based ladder sketch recognition language |
| url | http://hdl.handle.net/1721.1/71125 https://orcid.org/0000-0001-5232-7281 |
| work_keys_str_mv | AT hammondtracya creatingtheperceptionbasedladdersketchrecognitionlanguage AT davisrandall creatingtheperceptionbasedladdersketchrecognitionlanguage |