Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms
Abstract This perspective paper explores challenges associated with online crowdsourced data collection, particularly focusing on longitudinal tasks with time-sensitive outcomes like response latencies. Based on our research, we identify two significant sources of bias: technical shortcomings such a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-58300-7 |
| _version_ | 1797219891030261760 |
|---|---|
| author | Patrick A. McConnell Christian Finetto Kirstin-Friederike Heise |
| author_facet | Patrick A. McConnell Christian Finetto Kirstin-Friederike Heise |
| author_sort | Patrick A. McConnell |
| collection | DOAJ |
| description | Abstract This perspective paper explores challenges associated with online crowdsourced data collection, particularly focusing on longitudinal tasks with time-sensitive outcomes like response latencies. Based on our research, we identify two significant sources of bias: technical shortcomings such as low, variable frame rates, and human factors, contributing to high attrition rates. We explored potential solutions to these problems, such as enforcing hardware acceleration and defining study-specific frame rate thresholds, as well as pre-screening participants and monitoring hardware performance and task engagement over each experimental session. With this discussion, we intend to provide recommendations on how to improve the quality and reliability of data collected via online crowdsourced platforms and emphasize the need for researchers to be cognizant of potential pitfalls in online research. |
| first_indexed | 2024-04-24T12:40:50Z |
| format | Article |
| id | doaj.art-b46789c21ee5478989c593827c6b8d71 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-24T12:40:50Z |
| publishDate | 2024-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-b46789c21ee5478989c593827c6b8d712024-04-07T11:17:40ZengNature PortfolioScientific Reports2045-23222024-04-0114111010.1038/s41598-024-58300-7Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platformsPatrick A. McConnell0Christian Finetto1Kirstin-Friederike Heise2Integrative Neuromodulation and Recovery (iNR) Laboratory, Department of Health Sciences and Research, Medical University of South CarolinaIntegrative Neuromodulation and Recovery (iNR) Laboratory, Department of Health Sciences and Research, Medical University of South CarolinaIntegrative Neuromodulation and Recovery (iNR) Laboratory, Department of Health Sciences and Research, Medical University of South CarolinaAbstract This perspective paper explores challenges associated with online crowdsourced data collection, particularly focusing on longitudinal tasks with time-sensitive outcomes like response latencies. Based on our research, we identify two significant sources of bias: technical shortcomings such as low, variable frame rates, and human factors, contributing to high attrition rates. We explored potential solutions to these problems, such as enforcing hardware acceleration and defining study-specific frame rate thresholds, as well as pre-screening participants and monitoring hardware performance and task engagement over each experimental session. With this discussion, we intend to provide recommendations on how to improve the quality and reliability of data collected via online crowdsourced platforms and emphasize the need for researchers to be cognizant of potential pitfalls in online research.https://doi.org/10.1038/s41598-024-58300-7 |
| spellingShingle | Patrick A. McConnell Christian Finetto Kirstin-Friederike Heise Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms Scientific Reports |
| title | Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| title_full | Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| title_fullStr | Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| title_full_unstemmed | Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| title_short | Methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| title_sort | methodological considerations for behavioral studies relying on response time outcomes through online crowdsourcing platforms |
| url | https://doi.org/10.1038/s41598-024-58300-7 |
| work_keys_str_mv | AT patrickamcconnell methodologicalconsiderationsforbehavioralstudiesrelyingonresponsetimeoutcomesthroughonlinecrowdsourcingplatforms AT christianfinetto methodologicalconsiderationsforbehavioralstudiesrelyingonresponsetimeoutcomesthroughonlinecrowdsourcingplatforms AT kirstinfriederikeheise methodologicalconsiderationsforbehavioralstudiesrelyingonresponsetimeoutcomesthroughonlinecrowdsourcingplatforms |