AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer
Abstract The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recru...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-01-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-51723-2 |
| _version_ | 1827382311046348800 |
|---|---|
| author | Amanda Dy Ngoc-Nhu Jennifer Nguyen Julien Meyer Melanie Dawe Wei Shi Dimitri Androutsos Anthony Fyles Fei-Fei Liu Susan Done April Khademi |
| author_facet | Amanda Dy Ngoc-Nhu Jennifer Nguyen Julien Meyer Melanie Dawe Wei Shi Dimitri Androutsos Anthony Fyles Fei-Fei Liu Susan Done April Khademi |
| author_sort | Amanda Dy |
| collection | DOAJ |
| description | Abstract The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recruited to assess the Ki-67 PI of ten breast cancer tissue microarrays with and without AI. Accuracy, agreement, and turnaround time with and without AI were compared. Pathologists’ perspectives on AI were collected. Using AI led to a significant decrease in PI error (2.1% with AI vs. 5.9% without AI, p < 0.001), better inter-rater agreement (ICC: 0.70 vs. 0.92; Krippendorff’s α: 0.63 vs. 0.89; Fleiss’ Kappa: 0.40 vs. 0.86), and an 11.9% overall median reduction in turnaround time. Most pathologists (84%) found the AI reliable. For Ki-67 assessments, 76% of respondents believed AI enhances accuracy, 82% said it improves consistency, and 83% trust it will improve efficiency. This study highlights AI's potential to standardize Ki-67 scoring, especially between 5 and 30% PI—a range with low PI agreement. This could pave the way for a universally accepted PI score to guide treatment decisions, emphasizing the promising role of AI integration into pathologist workflows. |
| first_indexed | 2024-03-08T14:17:45Z |
| format | Article |
| id | doaj.art-f9b9e47961d24ef08c73f4a58d8c88d8 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-08T14:17:45Z |
| publishDate | 2024-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-f9b9e47961d24ef08c73f4a58d8c88d82024-01-14T12:21:03ZengNature PortfolioScientific Reports2045-23222024-01-0114111210.1038/s41598-024-51723-2AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancerAmanda Dy0Ngoc-Nhu Jennifer Nguyen1Julien Meyer2Melanie Dawe3Wei Shi4Dimitri Androutsos5Anthony Fyles6Fei-Fei Liu7Susan Done8April Khademi9Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan UniversityDepartment of Laboratory Medicine and Pathobiology, University of TorontoSchool of Health Services Management, Toronto Metropolitan UniversityPrincess Margaret Cancer Centre, University Health NetworkPrincess Margaret Cancer Centre, University Health NetworkElectrical, Computer, and Biomedical Engineering, Toronto Metropolitan UniversityPrincess Margaret Cancer Centre, University Health NetworkPrincess Margaret Cancer Centre, University Health NetworkPrincess Margaret Cancer Centre, University Health NetworkElectrical, Computer, and Biomedical Engineering, Toronto Metropolitan UniversityAbstract The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recruited to assess the Ki-67 PI of ten breast cancer tissue microarrays with and without AI. Accuracy, agreement, and turnaround time with and without AI were compared. Pathologists’ perspectives on AI were collected. Using AI led to a significant decrease in PI error (2.1% with AI vs. 5.9% without AI, p < 0.001), better inter-rater agreement (ICC: 0.70 vs. 0.92; Krippendorff’s α: 0.63 vs. 0.89; Fleiss’ Kappa: 0.40 vs. 0.86), and an 11.9% overall median reduction in turnaround time. Most pathologists (84%) found the AI reliable. For Ki-67 assessments, 76% of respondents believed AI enhances accuracy, 82% said it improves consistency, and 83% trust it will improve efficiency. This study highlights AI's potential to standardize Ki-67 scoring, especially between 5 and 30% PI—a range with low PI agreement. This could pave the way for a universally accepted PI score to guide treatment decisions, emphasizing the promising role of AI integration into pathologist workflows.https://doi.org/10.1038/s41598-024-51723-2 |
| spellingShingle | Amanda Dy Ngoc-Nhu Jennifer Nguyen Julien Meyer Melanie Dawe Wei Shi Dimitri Androutsos Anthony Fyles Fei-Fei Liu Susan Done April Khademi AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer Scientific Reports |
| title | AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer |
| title_full | AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer |
| title_fullStr | AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer |
| title_full_unstemmed | AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer |
| title_short | AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer |
| title_sort | ai improves accuracy agreement and efficiency of pathologists for ki67 assessments in breast cancer |
| url | https://doi.org/10.1038/s41598-024-51723-2 |
| work_keys_str_mv | AT amandady aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT ngocnhujennifernguyen aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT julienmeyer aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT melaniedawe aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT weishi aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT dimitriandroutsos aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT anthonyfyles aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT feifeiliu aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT susandone aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer AT aprilkhademi aiimprovesaccuracyagreementandefficiencyofpathologistsforki67assessmentsinbreastcancer |