Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement
All research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observ...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2023-02-01
|
| Series: | International Journal of Qualitative Methods |
| Online Access: | https://doi.org/10.1177/16094069231160973 |
| _version_ | 1811158061074612224 |
|---|---|
| author | Sherice N. Clarke S Sushil Katherine Dennis Ung-Sang Lee Andrea Gomoll Zaynab Gates |
| author_facet | Sherice N. Clarke S Sushil Katherine Dennis Ung-Sang Lee Andrea Gomoll Zaynab Gates |
| author_sort | Sherice N. Clarke |
| collection | DOAJ |
| description | All research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observe the same phenomena in data. Researchers report indices of agreement to serve as evidence of consistency and dependability of interpretations, and we do so to make claims about the trustworthiness of our research accounts. However, few studies report the methods of how multiple analysts developed alignment in their interpretation of data, a process that undergirds accounts of consistency, dependability, and trustworthiness. In this article, we review the issues and options around achieving intercoder agreement. Drawing on our experience from a longitudinal, team-based research project that required rapid cycles of qualitative data analysis, we reflect on the challenges we had achieving high intercoder agreement (which refer to as the perils). It was through these challenges that we developed a method that helps to foster shared ways of seeing data, and thus alignment in our interpretations of phenomena in data. In this article, we present this method as a tool for dyadic and team-based qualitative data analysis to facilitate reliable and consistent high-inference interpretations of data with multiple analysts. |
| first_indexed | 2024-04-10T05:16:49Z |
| format | Article |
| id | doaj.art-395d5200a80e44eca38dc82f64aed806 |
| institution | Directory Open Access Journal |
| issn | 1609-4069 |
| language | English |
| last_indexed | 2024-04-10T05:16:49Z |
| publishDate | 2023-02-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | International Journal of Qualitative Methods |
| spelling | doaj.art-395d5200a80e44eca38dc82f64aed8062023-03-08T17:03:25ZengSAGE PublishingInternational Journal of Qualitative Methods1609-40692023-02-012210.1177/16094069231160973Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder AgreementSherice N. ClarkeS SushilKatherine DennisUng-Sang LeeAndrea GomollZaynab GatesAll research, e.g., qualitative or quantitative, is concerned with the extent to which analyses can adequately describe the phenomena it seeks to describe. In qualitative research, we use internal validity checks like intercoder agreement to measure the extent to which independent researchers observe the same phenomena in data. Researchers report indices of agreement to serve as evidence of consistency and dependability of interpretations, and we do so to make claims about the trustworthiness of our research accounts. However, few studies report the methods of how multiple analysts developed alignment in their interpretation of data, a process that undergirds accounts of consistency, dependability, and trustworthiness. In this article, we review the issues and options around achieving intercoder agreement. Drawing on our experience from a longitudinal, team-based research project that required rapid cycles of qualitative data analysis, we reflect on the challenges we had achieving high intercoder agreement (which refer to as the perils). It was through these challenges that we developed a method that helps to foster shared ways of seeing data, and thus alignment in our interpretations of phenomena in data. In this article, we present this method as a tool for dyadic and team-based qualitative data analysis to facilitate reliable and consistent high-inference interpretations of data with multiple analysts.https://doi.org/10.1177/16094069231160973 |
| spellingShingle | Sherice N. Clarke S Sushil Katherine Dennis Ung-Sang Lee Andrea Gomoll Zaynab Gates Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement International Journal of Qualitative Methods |
| title | Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement |
| title_full | Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement |
| title_fullStr | Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement |
| title_full_unstemmed | Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement |
| title_short | Developing Shared Ways of Seeing Data: The Perils and Possibilities of Achieving Intercoder Agreement |
| title_sort | developing shared ways of seeing data the perils and possibilities of achieving intercoder agreement |
| url | https://doi.org/10.1177/16094069231160973 |
| work_keys_str_mv | AT shericenclarke developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement AT ssushil developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement AT katherinedennis developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement AT ungsanglee developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement AT andreagomoll developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement AT zaynabgates developingsharedwaysofseeingdatatheperilsandpossibilitiesofachievingintercoderagreement |