A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses
We developed a machine learning model based on radiomics to predict the BI-RADS category of ultrasound-detected suspicious breast lesions and support medical decision-making towards short-interval follow-up versus tissue sampling. From a retrospective 2015–2019 series of ultrasound-guided core needl...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/12/1/187 |
| _version_ | 1797494647750131712 |
|---|---|
| author | Matteo Interlenghi Christian Salvatore Veronica Magni Gabriele Caldara Elia Schiavon Andrea Cozzi Simone Schiaffino Luca Alessandro Carbonaro Isabella Castiglioni Francesco Sardanelli |
| author_facet | Matteo Interlenghi Christian Salvatore Veronica Magni Gabriele Caldara Elia Schiavon Andrea Cozzi Simone Schiaffino Luca Alessandro Carbonaro Isabella Castiglioni Francesco Sardanelli |
| author_sort | Matteo Interlenghi |
| collection | DOAJ |
| description | We developed a machine learning model based on radiomics to predict the BI-RADS category of ultrasound-detected suspicious breast lesions and support medical decision-making towards short-interval follow-up versus tissue sampling. From a retrospective 2015–2019 series of ultrasound-guided core needle biopsies performed by four board-certified breast radiologists using six ultrasound systems from three vendors, we collected 821 images of 834 suspicious breast masses from 819 patients, 404 malignant and 430 benign according to histopathology. A balanced image set of biopsy-proven benign (n = 299) and malignant (n = 299) lesions was used for training and cross-validation of ensembles of machine learning algorithms supervised during learning by histopathological diagnosis as a reference standard. Based on a majority vote (over 80% of the votes to have a valid prediction of benign lesion), an ensemble of support vector machines showed an ability to reduce the biopsy rate of benign lesions by 15% to 18%, always keeping a sensitivity over 94%, when externally tested on 236 images from two image sets: (1) 123 lesions (51 malignant and 72 benign) obtained from two ultrasound systems used for training and from a different one, resulting in a positive predictive value (PPV) of 45.9% (95% confidence interval 36.3–55.7%) versus a radiologists’ PPV of 41.5% (<i>p</i> < 0.005), combined with a 98.0% sensitivity (89.6–99.9%); (2) 113 lesions (54 malignant and 59 benign) obtained from two ultrasound systems from vendors different from those used for training, resulting into a 50.5% PPV (40.4–60.6%) versus a radiologists’ PPV of 47.8% (<i>p</i> < 0.005), combined with a 94.4% sensitivity (84.6–98.8%). Errors in BI-RADS 3 category (i.e., assigned by the model as BI-RADS 4) were 0.8% and 2.7% in the <i>Testing set I</i> and <i>II</i>, respectively. The board-certified breast radiologist accepted the BI-RADS classes assigned by the model in 114 masses (92.7%) and modified the BI-RADS classes of 9 breast masses (7.3%). In six of nine cases, the model performed better than the radiologist did, since it assigned a BI-RADS 3 classification to histopathology-confirmed benign masses that were classified as BI-RADS 4 by the radiologist. |
| first_indexed | 2024-03-10T01:37:21Z |
| format | Article |
| id | doaj.art-4c4ba2c98c1744119c15902249e4fe18 |
| institution | Directory Open Access Journal |
| issn | 2075-4418 |
| language | English |
| last_indexed | 2024-03-10T01:37:21Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj.art-4c4ba2c98c1744119c15902249e4fe182023-11-23T13:29:47ZengMDPI AGDiagnostics2075-44182022-01-0112118710.3390/diagnostics12010187A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast MassesMatteo Interlenghi0Christian Salvatore1Veronica Magni2Gabriele Caldara3Elia Schiavon4Andrea Cozzi5Simone Schiaffino6Luca Alessandro Carbonaro7Isabella Castiglioni8Francesco Sardanelli9DeepTrace Technologies S.R.L., Via Conservatorio 17, 20122 Milano, ItalyDeepTrace Technologies S.R.L., Via Conservatorio 17, 20122 Milano, ItalyDepartment of Biomedical Sciences for Health, Università Degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milano, ItalyDepartment of Science, Technology and Society, Scuola Universitaria IUSS, Istituto Universitario di Studi Superiori, Piazza della Vittoria 15, 27100 Pavia, ItalyDeepTrace Technologies S.R.L., Via Conservatorio 17, 20122 Milano, ItalyDepartment of Biomedical Sciences for Health, Università Degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milano, ItalyUnit of Radiology, IRCCS Policlinico San Donato, Via Rodolfo Morandi 30, 20097 San Donato Milanese, ItalyDepartment of Radiology, ASST Grande Ospedale Metropolitano Niguarda, Piazza dell’Ospedale Maggiore 3, 20162 Milano, ItalyInstitute of Biomedical Imaging and Physiology, Consiglio Nazionale delle Ricerche, Via Fratelli Cervi 93, 20090 Segrate, ItalyDepartment of Biomedical Sciences for Health, Università Degli Studi di Milano, Via Luigi Mangiagalli 31, 20133 Milano, ItalyWe developed a machine learning model based on radiomics to predict the BI-RADS category of ultrasound-detected suspicious breast lesions and support medical decision-making towards short-interval follow-up versus tissue sampling. From a retrospective 2015–2019 series of ultrasound-guided core needle biopsies performed by four board-certified breast radiologists using six ultrasound systems from three vendors, we collected 821 images of 834 suspicious breast masses from 819 patients, 404 malignant and 430 benign according to histopathology. A balanced image set of biopsy-proven benign (n = 299) and malignant (n = 299) lesions was used for training and cross-validation of ensembles of machine learning algorithms supervised during learning by histopathological diagnosis as a reference standard. Based on a majority vote (over 80% of the votes to have a valid prediction of benign lesion), an ensemble of support vector machines showed an ability to reduce the biopsy rate of benign lesions by 15% to 18%, always keeping a sensitivity over 94%, when externally tested on 236 images from two image sets: (1) 123 lesions (51 malignant and 72 benign) obtained from two ultrasound systems used for training and from a different one, resulting in a positive predictive value (PPV) of 45.9% (95% confidence interval 36.3–55.7%) versus a radiologists’ PPV of 41.5% (<i>p</i> < 0.005), combined with a 98.0% sensitivity (89.6–99.9%); (2) 113 lesions (54 malignant and 59 benign) obtained from two ultrasound systems from vendors different from those used for training, resulting into a 50.5% PPV (40.4–60.6%) versus a radiologists’ PPV of 47.8% (<i>p</i> < 0.005), combined with a 94.4% sensitivity (84.6–98.8%). Errors in BI-RADS 3 category (i.e., assigned by the model as BI-RADS 4) were 0.8% and 2.7% in the <i>Testing set I</i> and <i>II</i>, respectively. The board-certified breast radiologist accepted the BI-RADS classes assigned by the model in 114 masses (92.7%) and modified the BI-RADS classes of 9 breast masses (7.3%). In six of nine cases, the model performed better than the radiologist did, since it assigned a BI-RADS 3 classification to histopathology-confirmed benign masses that were classified as BI-RADS 4 by the radiologist.https://www.mdpi.com/2075-4418/12/1/187breast cancerultrasound (US)core needle biopsymachine learningradiomicssensitivity |
| spellingShingle | Matteo Interlenghi Christian Salvatore Veronica Magni Gabriele Caldara Elia Schiavon Andrea Cozzi Simone Schiaffino Luca Alessandro Carbonaro Isabella Castiglioni Francesco Sardanelli A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses Diagnostics breast cancer ultrasound (US) core needle biopsy machine learning radiomics sensitivity |
| title | A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses |
| title_full | A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses |
| title_fullStr | A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses |
| title_full_unstemmed | A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses |
| title_short | A Machine Learning Ensemble Based on Radiomics to Predict BI-RADS Category and Reduce the Biopsy Rate of Ultrasound-Detected Suspicious Breast Masses |
| title_sort | machine learning ensemble based on radiomics to predict bi rads category and reduce the biopsy rate of ultrasound detected suspicious breast masses |
| topic | breast cancer ultrasound (US) core needle biopsy machine learning radiomics sensitivity |
| url | https://www.mdpi.com/2075-4418/12/1/187 |
| work_keys_str_mv | AT matteointerlenghi amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT christiansalvatore amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT veronicamagni amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT gabrielecaldara amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT eliaschiavon amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT andreacozzi amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT simoneschiaffino amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT lucaalessandrocarbonaro amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT isabellacastiglioni amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT francescosardanelli amachinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT matteointerlenghi machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT christiansalvatore machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT veronicamagni machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT gabrielecaldara machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT eliaschiavon machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT andreacozzi machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT simoneschiaffino machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT lucaalessandrocarbonaro machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT isabellacastiglioni machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses AT francescosardanelli machinelearningensemblebasedonradiomicstopredictbiradscategoryandreducethebiopsyrateofultrasounddetectedsuspiciousbreastmasses |