Quantifying participation biases on social media
Abstract Around seven-in-ten Americans use social media (SM) to connect and engage, making these platforms excellent sources of information to understand human behavior and other problems relevant to social sciences. While the presence of a behavior can be detected, it is unclear who or under what c...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-07-01
|
| Series: | EPJ Data Science |
| Subjects: | |
| Online Access: | https://doi.org/10.1140/epjds/s13688-023-00405-6 |
| _version_ | 1827890499704324096 |
|---|---|
| author | Neeti Pokhriyal Benjamin A. Valentino Soroush Vosoughi |
| author_facet | Neeti Pokhriyal Benjamin A. Valentino Soroush Vosoughi |
| author_sort | Neeti Pokhriyal |
| collection | DOAJ |
| description | Abstract Around seven-in-ten Americans use social media (SM) to connect and engage, making these platforms excellent sources of information to understand human behavior and other problems relevant to social sciences. While the presence of a behavior can be detected, it is unclear who or under what circumstances the behavior was generated. Despite the large sample sizes of SM datasets, they almost always come with significant biases, some of which have been studied before. Here, we hypothesize the presence of a largely unrecognized form of bias on SM platforms, called participation bias, that is distinct from selection bias. It is defined as the skew in the demographics of the participants who opt-in to discussions of the topic, compared to the demographics of the underlying SM platform. To infer the participant’s demographics, we propose a novel generative probabilistic framework that links surveys and SM data at the granularity of demographic subgroups (and not individuals). Our method is distinct from existing approaches that elicit such information at the individual level using their profile name, images, and other metadata, thus infringing upon their privacy. We design a statistical simulation to simulate multiple SM platforms and a diverse range of topics to validate the model’s estimates in different scenarios. We use Twitter data as a case study to demonstrate participation bias on the topic of gun violence delineated by political party affiliation and gender. Although Twitter’s user population leans Democratic and has an equal number of men and women according to Pew, our model’s estimates point to the presence of participation bias on the topic of gun control in the opposite direction, with slightly more Republicans than Democrats, and more men compared to women. Our study cautions that in the rush to use digital data for decision-making and understanding public opinions, we must account for the biases inherent in how SM data are produced, lest we may also arrive at biased inferences about the public. |
| first_indexed | 2024-03-12T21:10:35Z |
| format | Article |
| id | doaj.art-0425af6c29e149eea6a74c31a83bb7a9 |
| institution | Directory Open Access Journal |
| issn | 2193-1127 |
| language | English |
| last_indexed | 2024-03-12T21:10:35Z |
| publishDate | 2023-07-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EPJ Data Science |
| spelling | doaj.art-0425af6c29e149eea6a74c31a83bb7a92023-07-30T11:09:56ZengSpringerOpenEPJ Data Science2193-11272023-07-0112112010.1140/epjds/s13688-023-00405-6Quantifying participation biases on social mediaNeeti Pokhriyal0Benjamin A. Valentino1Soroush Vosoughi2Department of Computer Science, Dartmouth CollegeDepartment of Government, Dartmouth CollegeDepartment of Computer Science, Dartmouth CollegeAbstract Around seven-in-ten Americans use social media (SM) to connect and engage, making these platforms excellent sources of information to understand human behavior and other problems relevant to social sciences. While the presence of a behavior can be detected, it is unclear who or under what circumstances the behavior was generated. Despite the large sample sizes of SM datasets, they almost always come with significant biases, some of which have been studied before. Here, we hypothesize the presence of a largely unrecognized form of bias on SM platforms, called participation bias, that is distinct from selection bias. It is defined as the skew in the demographics of the participants who opt-in to discussions of the topic, compared to the demographics of the underlying SM platform. To infer the participant’s demographics, we propose a novel generative probabilistic framework that links surveys and SM data at the granularity of demographic subgroups (and not individuals). Our method is distinct from existing approaches that elicit such information at the individual level using their profile name, images, and other metadata, thus infringing upon their privacy. We design a statistical simulation to simulate multiple SM platforms and a diverse range of topics to validate the model’s estimates in different scenarios. We use Twitter data as a case study to demonstrate participation bias on the topic of gun violence delineated by political party affiliation and gender. Although Twitter’s user population leans Democratic and has an equal number of men and women according to Pew, our model’s estimates point to the presence of participation bias on the topic of gun control in the opposite direction, with slightly more Republicans than Democrats, and more men compared to women. Our study cautions that in the rush to use digital data for decision-making and understanding public opinions, we must account for the biases inherent in how SM data are produced, lest we may also arrive at biased inferences about the public.https://doi.org/10.1140/epjds/s13688-023-00405-6Social media dataProbabilistic modelingBias quantification |
| spellingShingle | Neeti Pokhriyal Benjamin A. Valentino Soroush Vosoughi Quantifying participation biases on social media EPJ Data Science Social media data Probabilistic modeling Bias quantification |
| title | Quantifying participation biases on social media |
| title_full | Quantifying participation biases on social media |
| title_fullStr | Quantifying participation biases on social media |
| title_full_unstemmed | Quantifying participation biases on social media |
| title_short | Quantifying participation biases on social media |
| title_sort | quantifying participation biases on social media |
| topic | Social media data Probabilistic modeling Bias quantification |
| url | https://doi.org/10.1140/epjds/s13688-023-00405-6 |
| work_keys_str_mv | AT neetipokhriyal quantifyingparticipationbiasesonsocialmedia AT benjaminavalentino quantifyingparticipationbiasesonsocialmedia AT soroushvosoughi quantifyingparticipationbiasesonsocialmedia |