Demonstration of Qurk: A Query Processor for Human Operators
Crowdsourcing technologies such as Amazon's Mechanical Turk ("MTurk") service have exploded in popularity in recent years. These services are increasingly used for complex human-reliant data processing tasks, such as labelling a collection of images, combining two sets of images to id...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2012
|
| Online Access: | http://hdl.handle.net/1721.1/72373 https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X |
| _version_ | 1826192065944354816 |
|---|---|
| author | Marcus, Adam Wu, Eugene Karger, David R. Madden, Samuel R. Miller, Robert C. |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Marcus, Adam Wu, Eugene Karger, David R. Madden, Samuel R. Miller, Robert C. |
| author_sort | Marcus, Adam |
| collection | MIT |
| description | Crowdsourcing technologies such as Amazon's Mechanical Turk ("MTurk") service have exploded in popularity in recent years. These services are increasingly used for complex human-reliant data processing tasks, such as labelling a collection of images, combining two sets of images to identify people that appear in both, or extracting sentiment from a corpus of text snippets. There are several challenges in designing a workflow that filters, aggregates, sorts and joins human-generated data sources. Currently, crowdsourcing-based workflows are hand-built, resulting in increasingly complex programs. Additionally, developers must hand-optimize tradeoffs among monetary cost, accuracy, and time to completion of results. These challenges are well-suited to a declarative query interface that allows developers to describe their worflow at a high level and automatically optimizes workflow and tuning parameters. In this demonstration, we will present Qurk, a novel query system that allows human-based processing for relational databases. The audience will interact with the system to build queries and monitor their progress. The audience will also see Qurk from an MTurk user's perspective, and complete several tasks to better understand how a query is processed. |
| first_indexed | 2024-09-23T09:05:45Z |
| format | Article |
| id | mit-1721.1/72373 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T09:05:45Z |
| publishDate | 2012 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/723732022-09-30T13:23:59Z Demonstration of Qurk: A Query Processor for Human Operators Marcus, Adam Wu, Eugene Karger, David R. Madden, Samuel R. Miller, Robert C. Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Karger, David R. Marcus, Adam Wu, Eugene Karger, David R. Madden, Samuel R. Miller, Robert C. Crowdsourcing technologies such as Amazon's Mechanical Turk ("MTurk") service have exploded in popularity in recent years. These services are increasingly used for complex human-reliant data processing tasks, such as labelling a collection of images, combining two sets of images to identify people that appear in both, or extracting sentiment from a corpus of text snippets. There are several challenges in designing a workflow that filters, aggregates, sorts and joins human-generated data sources. Currently, crowdsourcing-based workflows are hand-built, resulting in increasingly complex programs. Additionally, developers must hand-optimize tradeoffs among monetary cost, accuracy, and time to completion of results. These challenges are well-suited to a declarative query interface that allows developers to describe their worflow at a high level and automatically optimizes workflow and tuning parameters. In this demonstration, we will present Qurk, a novel query system that allows human-based processing for relational databases. The audience will interact with the system to build queries and monitor their progress. The audience will also see Qurk from an MTurk user's perspective, and complete several tasks to better understand how a query is processed. 2012-08-28T16:22:03Z 2012-08-28T16:22:03Z 2011-06 Article http://purl.org/eprint/type/ConferencePaper 978-1-4503-0661-4 http://hdl.handle.net/1721.1/72373 Adam Marcus, Eugene Wu, David R. Karger, Samuel Madden, and Robert C. Miller. 2011. Demonstration of Qurk: a query processor for humanoperators. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data (SIGMOD '11). ACM, New York, NY, USA, 1315-1318. https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X en_US http://dx.doi.org/10.1145/1989323.1989486 Proceedings of the 2011 ACM SIGMOD International Conference on Management of data (SIGMOD '11) Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for Computing Machinery (ACM) MIT web domain |
| spellingShingle | Marcus, Adam Wu, Eugene Karger, David R. Madden, Samuel R. Miller, Robert C. Demonstration of Qurk: A Query Processor for Human Operators |
| title | Demonstration of Qurk: A Query Processor for Human Operators |
| title_full | Demonstration of Qurk: A Query Processor for Human Operators |
| title_fullStr | Demonstration of Qurk: A Query Processor for Human Operators |
| title_full_unstemmed | Demonstration of Qurk: A Query Processor for Human Operators |
| title_short | Demonstration of Qurk: A Query Processor for Human Operators |
| title_sort | demonstration of qurk a query processor for human operators |
| url | http://hdl.handle.net/1721.1/72373 https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X |
| work_keys_str_mv | AT marcusadam demonstrationofqurkaqueryprocessorforhumanoperators AT wueugene demonstrationofqurkaqueryprocessorforhumanoperators AT kargerdavidr demonstrationofqurkaqueryprocessorforhumanoperators AT maddensamuelr demonstrationofqurkaqueryprocessorforhumanoperators AT millerrobertc demonstrationofqurkaqueryprocessorforhumanoperators |