Crowds in two seconds: Enabling realtime crowd-powered interfaces
Interactive systems must respond to user input within seconds. Therefore, to create realtime crowd-powered interfaces, we need to dramatically lower crowd latency. In this paper, we introduce the use of synchronous crowds for on-demand, realtime crowdsourcing. With synchronous crowds, systems can dy...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2012
|
| Online Access: | http://hdl.handle.net/1721.1/72377 https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X |
| _version_ | 1826195152964681728 |
|---|---|
| author | Bernstein, Michael S. Brandt, Joel Miller, Robert C. Karger, David R. |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Bernstein, Michael S. Brandt, Joel Miller, Robert C. Karger, David R. |
| author_sort | Bernstein, Michael S. |
| collection | MIT |
| description | Interactive systems must respond to user input within seconds. Therefore, to create realtime crowd-powered interfaces, we need to dramatically lower crowd latency. In this paper, we introduce the use of synchronous crowds for on-demand, realtime crowdsourcing. With synchronous crowds, systems can dynamically adapt tasks by leveraging the fact that workers are present at the same time. We develop techniques that recruit synchronous crowds in two seconds and use them to execute complex search tasks in ten seconds. The first technique, the retainer model, pays workers a small wage to wait and respond quickly when asked. We offer empirically derived guidelines for a retainer system that is low-cost and produces on-demand crowds in two seconds. Our second technique, rapid refinement, observes early signs of agreement in synchronous crowds and dynamically narrows the search space to focus on promising directions. This approach produces results that, on average, are of more reliable quality and arrive faster than the fastest crowd member working alone. To explore benefits and limitations of these techniques for interaction, we present three applications: Adrenaline, a crowd-powered camera where workers quickly filter a short video down to the best single moment for a photo; and Puppeteer and A|B, which examine creative generation tasks, communication with workers, and low-latency voting. |
| first_indexed | 2024-09-23T10:07:51Z |
| format | Article |
| id | mit-1721.1/72377 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T10:07:51Z |
| publishDate | 2012 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/723772022-09-30T19:04:52Z Crowds in two seconds: Enabling realtime crowd-powered interfaces Bernstein, Michael S. Brandt, Joel Miller, Robert C. Karger, David R. Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Karger, David R. Bernstein, Michael S. Miller, Robert C. Karger, David R. Interactive systems must respond to user input within seconds. Therefore, to create realtime crowd-powered interfaces, we need to dramatically lower crowd latency. In this paper, we introduce the use of synchronous crowds for on-demand, realtime crowdsourcing. With synchronous crowds, systems can dynamically adapt tasks by leveraging the fact that workers are present at the same time. We develop techniques that recruit synchronous crowds in two seconds and use them to execute complex search tasks in ten seconds. The first technique, the retainer model, pays workers a small wage to wait and respond quickly when asked. We offer empirically derived guidelines for a retainer system that is low-cost and produces on-demand crowds in two seconds. Our second technique, rapid refinement, observes early signs of agreement in synchronous crowds and dynamically narrows the search space to focus on promising directions. This approach produces results that, on average, are of more reliable quality and arrive faster than the fastest crowd member working alone. To explore benefits and limitations of these techniques for interaction, we present three applications: Adrenaline, a crowd-powered camera where workers quickly filter a short video down to the best single moment for a photo; and Puppeteer and A|B, which examine creative generation tasks, communication with workers, and low-latency voting. 2012-08-28T18:02:44Z 2012-08-28T18:02:44Z 2011-10 Article http://purl.org/eprint/type/ConferencePaper 978-1-4503-0716-1 http://hdl.handle.net/1721.1/72377 Michael S. Bernstein, Joel Brandt, Robert C. Miller, and David R. Karger. 2011. Crowds in two seconds: enabling realtime crowd-powered interfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 33-42. https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X en_US http://dx.doi.org/10.1145/2047196.2047201 Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11) Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for Computing Machinery (ACM) MIT web domain |
| spellingShingle | Bernstein, Michael S. Brandt, Joel Miller, Robert C. Karger, David R. Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title | Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title_full | Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title_fullStr | Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title_full_unstemmed | Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title_short | Crowds in two seconds: Enabling realtime crowd-powered interfaces |
| title_sort | crowds in two seconds enabling realtime crowd powered interfaces |
| url | http://hdl.handle.net/1721.1/72377 https://orcid.org/0000-0002-0024-5847 https://orcid.org/0000-0002-0442-691X |
| work_keys_str_mv | AT bernsteinmichaels crowdsintwosecondsenablingrealtimecrowdpoweredinterfaces AT brandtjoel crowdsintwosecondsenablingrealtimecrowdpoweredinterfaces AT millerrobertc crowdsintwosecondsenablingrealtimecrowdpoweredinterfaces AT kargerdavidr crowdsintwosecondsenablingrealtimecrowdpoweredinterfaces |