Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study
In response time (RT) research, RT outliers are typically excluded from statistical analysis to improve the signal-to-noise ratio. Nevertheless, there exist several methods for outlier exclusion. This poses the question, how these methods differ with respect to recovering the uncontaminated RT distr...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2021.675558/full |
| _version_ | 1818664384700874752 |
|---|---|
| author | Alexander Berger Markus Kiefer |
| author_facet | Alexander Berger Markus Kiefer |
| author_sort | Alexander Berger |
| collection | DOAJ |
| description | In response time (RT) research, RT outliers are typically excluded from statistical analysis to improve the signal-to-noise ratio. Nevertheless, there exist several methods for outlier exclusion. This poses the question, how these methods differ with respect to recovering the uncontaminated RT distribution. In the present simulation study, two RT distributions with a given population difference were simulated in each iteration. RTs were replaced by outliers following two different approaches. The first approach generated outliers at the tails of the distribution, the second one inserted outliers overlapping with the genuine RT distribution. We applied ten different outlier exclusion methods and tested, how many pairs of distributions significantly differed. Outlier exclusion methods were compared in terms of bias. Bias was defined as the deviation of the proportion of significant differences after outlier exclusion from the proportion of significant differences in the uncontaminated samples (before introducing outliers). Our results showed large differences in bias between the exclusion methods. Some methods showed a high rate of Type-I errors and should therefore clearly not be used. Overall, our results showed that applying an exclusion method based on z-scores / standard deviations introduced only small biases, while the absence of outlier exclusion showed the largest absolute bias. |
| first_indexed | 2024-12-17T05:31:53Z |
| format | Article |
| id | doaj.art-f824f476c63f4a1abb1d179945403dd3 |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-17T05:31:53Z |
| publishDate | 2021-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-f824f476c63f4a1abb1d179945403dd32022-12-21T22:01:42ZengFrontiers Media S.A.Frontiers in Psychology1664-10782021-06-011210.3389/fpsyg.2021.675558675558Comparison of Different Response Time Outlier Exclusion Methods: A Simulation StudyAlexander BergerMarkus KieferIn response time (RT) research, RT outliers are typically excluded from statistical analysis to improve the signal-to-noise ratio. Nevertheless, there exist several methods for outlier exclusion. This poses the question, how these methods differ with respect to recovering the uncontaminated RT distribution. In the present simulation study, two RT distributions with a given population difference were simulated in each iteration. RTs were replaced by outliers following two different approaches. The first approach generated outliers at the tails of the distribution, the second one inserted outliers overlapping with the genuine RT distribution. We applied ten different outlier exclusion methods and tested, how many pairs of distributions significantly differed. Outlier exclusion methods were compared in terms of bias. Bias was defined as the deviation of the proportion of significant differences after outlier exclusion from the proportion of significant differences in the uncontaminated samples (before introducing outliers). Our results showed large differences in bias between the exclusion methods. Some methods showed a high rate of Type-I errors and should therefore clearly not be used. Overall, our results showed that applying an exclusion method based on z-scores / standard deviations introduced only small biases, while the absence of outlier exclusion showed the largest absolute bias.https://www.frontiersin.org/articles/10.3389/fpsyg.2021.675558/fullresponse timereaction timeoutlier exclusionsimulation studymental chronometry |
| spellingShingle | Alexander Berger Markus Kiefer Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study Frontiers in Psychology response time reaction time outlier exclusion simulation study mental chronometry |
| title | Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study |
| title_full | Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study |
| title_fullStr | Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study |
| title_full_unstemmed | Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study |
| title_short | Comparison of Different Response Time Outlier Exclusion Methods: A Simulation Study |
| title_sort | comparison of different response time outlier exclusion methods a simulation study |
| topic | response time reaction time outlier exclusion simulation study mental chronometry |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2021.675558/full |
| work_keys_str_mv | AT alexanderberger comparisonofdifferentresponsetimeoutlierexclusionmethodsasimulationstudy AT markuskiefer comparisonofdifferentresponsetimeoutlierexclusionmethodsasimulationstudy |