Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification
Background: Observer studies in pathology often utilize a limited number of representative slides per case, selected and reported in a nonstandardized manner. Reference diagnoses are commonly assumed to be generalizable to all slides of a case. We examined these issues in the context of pathologist...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2021-01-01
|
| Series: | Journal of Pathology Informatics |
| Subjects: | |
| Online Access: | http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2021;volume=12;issue=1;spage=15;epage=15;aulast=Gavrielides |
| _version_ | 1818208753867030528 |
|---|---|
| author | Marios A Gavrielides Brigitte M Ronnett Russell Vang Fahime Sheikhzadeh Jeffrey D Seidman |
| author_facet | Marios A Gavrielides Brigitte M Ronnett Russell Vang Fahime Sheikhzadeh Jeffrey D Seidman |
| author_sort | Marios A Gavrielides |
| collection | DOAJ |
| description | Background: Observer studies in pathology often utilize a limited number of representative slides per case, selected and reported in a nonstandardized manner. Reference diagnoses are commonly assumed to be generalizable to all slides of a case. We examined these issues in the context of pathologist concordance for histologic subtype classification of ovarian carcinomas (OCs). Materials and Methods: A cohort of 114 OCs consisting of 72 cases with a single representative slide (Group 1) and 42 cases with multiple representative slides (148 slides, 2-“6 sections per case, Group 2) was independently reviewed by three experts in gynecologic pathology (case-based review). In a follow-up study, each individual slide was independently reviewed in a randomized order by the same pathologists (section-based review). Results: Average interobserver concordance varied from 100% for Group 1 to 64.3% for Group 2 (86.8% across all cases). Across Group 2, 19 cases (45.2%) had at least one slide classified as a different subtype than the subtype assigned from case-based review, demonstrating the impact of intratumoral heterogeneity. Section-based concordance across individual sections from Group 2 was comparable to case-based concordance for those cases indicating diagnostic challenges at the individual section level. Findings demonstrate the increased diagnostic complexity of heterogeneous tumors that require multiple section sampling and its impact on pathologist performance. Conclusions: The proportion of cases with multiple representative slides in cohorts used in validation studies, such as those conducted to evaluate artificial intelligence/machine learning tools, can influence diagnostic performance, and if not accounted for, can cause disparities between research and real-world observations and between research studies. Case selection in validation studies should account for tumor heterogeneity to create balanced datasets in terms of diagnostic complexity. |
| first_indexed | 2024-12-12T04:49:50Z |
| format | Article |
| id | doaj.art-653a1fd2e9654afdadb7edd808da0ff0 |
| institution | Directory Open Access Journal |
| issn | 2153-3539 2153-3539 |
| language | English |
| last_indexed | 2024-12-12T04:49:50Z |
| publishDate | 2021-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of Pathology Informatics |
| spelling | doaj.art-653a1fd2e9654afdadb7edd808da0ff02022-12-22T00:37:31ZengElsevierJournal of Pathology Informatics2153-35392153-35392021-01-01121151510.4103/jpi.jpi_56_20Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classificationMarios A GavrielidesBrigitte M RonnettRussell VangFahime SheikhzadehJeffrey D SeidmanBackground: Observer studies in pathology often utilize a limited number of representative slides per case, selected and reported in a nonstandardized manner. Reference diagnoses are commonly assumed to be generalizable to all slides of a case. We examined these issues in the context of pathologist concordance for histologic subtype classification of ovarian carcinomas (OCs). Materials and Methods: A cohort of 114 OCs consisting of 72 cases with a single representative slide (Group 1) and 42 cases with multiple representative slides (148 slides, 2-“6 sections per case, Group 2) was independently reviewed by three experts in gynecologic pathology (case-based review). In a follow-up study, each individual slide was independently reviewed in a randomized order by the same pathologists (section-based review). Results: Average interobserver concordance varied from 100% for Group 1 to 64.3% for Group 2 (86.8% across all cases). Across Group 2, 19 cases (45.2%) had at least one slide classified as a different subtype than the subtype assigned from case-based review, demonstrating the impact of intratumoral heterogeneity. Section-based concordance across individual sections from Group 2 was comparable to case-based concordance for those cases indicating diagnostic challenges at the individual section level. Findings demonstrate the increased diagnostic complexity of heterogeneous tumors that require multiple section sampling and its impact on pathologist performance. Conclusions: The proportion of cases with multiple representative slides in cohorts used in validation studies, such as those conducted to evaluate artificial intelligence/machine learning tools, can influence diagnostic performance, and if not accounted for, can cause disparities between research and real-world observations and between research studies. Case selection in validation studies should account for tumor heterogeneity to create balanced datasets in terms of diagnostic complexity.http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2021;volume=12;issue=1;spage=15;epage=15;aulast=Gavrielidesobserver studyovarian cancerpathologist concordancerepresentative slidessubtype classification |
| spellingShingle | Marios A Gavrielides Brigitte M Ronnett Russell Vang Fahime Sheikhzadeh Jeffrey D Seidman Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification Journal of Pathology Informatics observer study ovarian cancer pathologist concordance representative slides subtype classification |
| title | Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification |
| title_full | Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification |
| title_fullStr | Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification |
| title_full_unstemmed | Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification |
| title_short | Selection of representative histologic slides in interobserver reproducibility studies: Insights from expert review for ovarian carcinoma subtype classification |
| title_sort | selection of representative histologic slides in interobserver reproducibility studies insights from expert review for ovarian carcinoma subtype classification |
| topic | observer study ovarian cancer pathologist concordance representative slides subtype classification |
| url | http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2021;volume=12;issue=1;spage=15;epage=15;aulast=Gavrielides |
| work_keys_str_mv | AT mariosagavrielides selectionofrepresentativehistologicslidesininterobserverreproducibilitystudiesinsightsfromexpertreviewforovariancarcinomasubtypeclassification AT brigittemronnett selectionofrepresentativehistologicslidesininterobserverreproducibilitystudiesinsightsfromexpertreviewforovariancarcinomasubtypeclassification AT russellvang selectionofrepresentativehistologicslidesininterobserverreproducibilitystudiesinsightsfromexpertreviewforovariancarcinomasubtypeclassification AT fahimesheikhzadeh selectionofrepresentativehistologicslidesininterobserverreproducibilitystudiesinsightsfromexpertreviewforovariancarcinomasubtypeclassification AT jeffreydseidman selectionofrepresentativehistologicslidesininterobserverreproducibilitystudiesinsightsfromexpertreviewforovariancarcinomasubtypeclassification |