Pitfalls in post hoc analyses of population receptive field data
Data binning involves grouping observations into bins and calculating bin-wise summary statistics. It can cope with overplotting and noise, making it a versatile tool for comparing many observations. However, data binning goes awry if the same observations are used for binning (selection) and contra...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-11-01
|
| Series: | NeuroImage |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1053811922006723 |
| _version_ | 1811247875489792000 |
|---|---|
| author | Susanne Stoll Elisa Infanti Benjamin de Haas D. Samuel Schwarzkopf |
| author_facet | Susanne Stoll Elisa Infanti Benjamin de Haas D. Samuel Schwarzkopf |
| author_sort | Susanne Stoll |
| collection | DOAJ |
| description | Data binning involves grouping observations into bins and calculating bin-wise summary statistics. It can cope with overplotting and noise, making it a versatile tool for comparing many observations. However, data binning goes awry if the same observations are used for binning (selection) and contrasting (selective analysis). This creates circularity, biasing noise components and resulting in artifactual changes in the form of regression towards the mean. Importantly, these artifactual changes are a statistical necessity. Here, we use (null) simulations and empirical repeat data to expose this flaw in the scope of post hoc analyses of population receptive field data. In doing so, we reveal that the type of data analysis, data properties, and circular data cleaning are factors shaping the appearance of such artifactual changes. We furthermore highlight that circular data cleaning and circular sorting of change scores are selection practices that result in artifactual changes even without circular data binning. These pitfalls might have led to erroneous claims about changes in population receptive fields in previous work and can be mitigated by using independent data for selection purposes. Our evaluations highlight the urgency for us researchers to make the validation of analysis pipelines standard practice. |
| first_indexed | 2024-04-12T15:17:10Z |
| format | Article |
| id | doaj.art-22cffc1e52964ca3bb1dedd26fdd579c |
| institution | Directory Open Access Journal |
| issn | 1095-9572 |
| language | English |
| last_indexed | 2024-04-12T15:17:10Z |
| publishDate | 2022-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage |
| spelling | doaj.art-22cffc1e52964ca3bb1dedd26fdd579c2022-12-22T03:27:34ZengElsevierNeuroImage1095-95722022-11-01263119557Pitfalls in post hoc analyses of population receptive field dataSusanne Stoll0Elisa Infanti1Benjamin de Haas2D. Samuel Schwarzkopf3Corresponding author.; Experimental Psychology, University College London, 26 Bedford Way, London, WC1H 0AP, UKExperimental Psychology, University College London, 26 Bedford Way, London, WC1H 0AP, UKAbteilung Allgemeine Psychologie, Justus-Liebig-Universit.±t Gie..en, Otto-Behaghel-Str. 10F, 35394 Gie..en, GermanySchool of Optometry and Vision Science, The University of Auckland, Private Bag 92019, Auckland 1142, New ZealandData binning involves grouping observations into bins and calculating bin-wise summary statistics. It can cope with overplotting and noise, making it a versatile tool for comparing many observations. However, data binning goes awry if the same observations are used for binning (selection) and contrasting (selective analysis). This creates circularity, biasing noise components and resulting in artifactual changes in the form of regression towards the mean. Importantly, these artifactual changes are a statistical necessity. Here, we use (null) simulations and empirical repeat data to expose this flaw in the scope of post hoc analyses of population receptive field data. In doing so, we reveal that the type of data analysis, data properties, and circular data cleaning are factors shaping the appearance of such artifactual changes. We furthermore highlight that circular data cleaning and circular sorting of change scores are selection practices that result in artifactual changes even without circular data binning. These pitfalls might have led to erroneous claims about changes in population receptive fields in previous work and can be mitigated by using independent data for selection purposes. Our evaluations highlight the urgency for us researchers to make the validation of analysis pipelines standard practice.http://www.sciencedirect.com/science/article/pii/S1053811922006723Regression towards the meanCircularityDouble-dippingValidationFunctional magnetic resonance imaging |
| spellingShingle | Susanne Stoll Elisa Infanti Benjamin de Haas D. Samuel Schwarzkopf Pitfalls in post hoc analyses of population receptive field data NeuroImage Regression towards the mean Circularity Double-dipping Validation Functional magnetic resonance imaging |
| title | Pitfalls in post hoc analyses of population receptive field data |
| title_full | Pitfalls in post hoc analyses of population receptive field data |
| title_fullStr | Pitfalls in post hoc analyses of population receptive field data |
| title_full_unstemmed | Pitfalls in post hoc analyses of population receptive field data |
| title_short | Pitfalls in post hoc analyses of population receptive field data |
| title_sort | pitfalls in post hoc analyses of population receptive field data |
| topic | Regression towards the mean Circularity Double-dipping Validation Functional magnetic resonance imaging |
| url | http://www.sciencedirect.com/science/article/pii/S1053811922006723 |
| work_keys_str_mv | AT susannestoll pitfallsinposthocanalysesofpopulationreceptivefielddata AT elisainfanti pitfallsinposthocanalysesofpopulationreceptivefielddata AT benjamindehaas pitfallsinposthocanalysesofpopulationreceptivefielddata AT dsamuelschwarzkopf pitfallsinposthocanalysesofpopulationreceptivefielddata |