Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy
Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given tr...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Multidisciplinary Digital Publishing Institute
2023
|
| Online Access: | https://hdl.handle.net/1721.1/151112 |
| _version_ | 1811080143825797120 |
|---|---|
| author | Verma, Monu Abdelrahman, Leila Collado-Mesa, Fernando Abdel-Mottaleb, Mohamed |
| author2 | Massachusetts Institute of Technology. Media Laboratory |
| author_facet | Massachusetts Institute of Technology. Media Laboratory Verma, Monu Abdelrahman, Leila Collado-Mesa, Fernando Abdel-Mottaleb, Mohamed |
| author_sort | Verma, Monu |
| collection | MIT |
| description | Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given treatment regimen could increase the likelihood of achieving pCR and prevent toxicities caused by treatments that are not effective. Th early prediction of response to NST can increase the likelihood of survival and help with decisions regarding breast-conserving surgery. An automated NST prediction framework that is able to precisely predict which patient undergoing NST will achieve a pathological complete response (pCR) at an early stage of treatment is needed. Here, we propose an end-to-end efficient multimodal spatiotemporal deep learning framework (deep-NST) framework to predict the outcome of NST prior or at an early stage of treatment. The deep-NST model incorporates imaging data captured at different timestamps of NST regimens, a tumor’s molecular data, and a patient’s demographic data. The efficacy of the proposed work is validated on the publicly available ISPY-1 dataset, in terms of accuracy, area under the curve (AUC), and computational complexity. In addition, seven ablation experiments were carried out to evaluate the impact of each design module in the proposed work. The experimental results show that the proposed framework performs significantly better than other recent methods. |
| first_indexed | 2024-09-23T11:26:30Z |
| format | Article |
| id | mit-1721.1/151112 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T11:26:30Z |
| publishDate | 2023 |
| publisher | Multidisciplinary Digital Publishing Institute |
| record_format | dspace |
| spelling | mit-1721.1/1511122024-01-10T19:50:43Z Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy Verma, Monu Abdelrahman, Leila Collado-Mesa, Fernando Abdel-Mottaleb, Mohamed Massachusetts Institute of Technology. Media Laboratory Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given treatment regimen could increase the likelihood of achieving pCR and prevent toxicities caused by treatments that are not effective. Th early prediction of response to NST can increase the likelihood of survival and help with decisions regarding breast-conserving surgery. An automated NST prediction framework that is able to precisely predict which patient undergoing NST will achieve a pathological complete response (pCR) at an early stage of treatment is needed. Here, we propose an end-to-end efficient multimodal spatiotemporal deep learning framework (deep-NST) framework to predict the outcome of NST prior or at an early stage of treatment. The deep-NST model incorporates imaging data captured at different timestamps of NST regimens, a tumor’s molecular data, and a patient’s demographic data. The efficacy of the proposed work is validated on the publicly available ISPY-1 dataset, in terms of accuracy, area under the curve (AUC), and computational complexity. In addition, seven ablation experiments were carried out to evaluate the impact of each design module in the proposed work. The experimental results show that the proposed framework performs significantly better than other recent methods. 2023-07-13T17:33:17Z 2023-07-13T17:33:17Z 2023-07-03 2023-07-13T14:06:55Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/151112 Diagnostics 13 (13): 2251 (2023) PUBLISHER_CC http://dx.doi.org/10.3390/diagnostics13132251 Creative Commons Attribution http://creativecommons.org/licenses/by/4.0/ application/pdf Multidisciplinary Digital Publishing Institute Multidisciplinary Digital Publishing Institute |
| spellingShingle | Verma, Monu Abdelrahman, Leila Collado-Mesa, Fernando Abdel-Mottaleb, Mohamed Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title | Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title_full | Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title_fullStr | Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title_full_unstemmed | Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title_short | Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy |
| title_sort | multimodal spatiotemporal deep learning framework to predict response of breast cancer to neoadjuvant systemic therapy |
| url | https://hdl.handle.net/1721.1/151112 |
| work_keys_str_mv | AT vermamonu multimodalspatiotemporaldeeplearningframeworktopredictresponseofbreastcancertoneoadjuvantsystemictherapy AT abdelrahmanleila multimodalspatiotemporaldeeplearningframeworktopredictresponseofbreastcancertoneoadjuvantsystemictherapy AT colladomesafernando multimodalspatiotemporaldeeplearningframeworktopredictresponseofbreastcancertoneoadjuvantsystemictherapy AT abdelmottalebmohamed multimodalspatiotemporaldeeplearningframeworktopredictresponseofbreastcancertoneoadjuvantsystemictherapy |