Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images
Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augm...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-03-01
|
| Series: | Frontiers in Medicine |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmed.2023.1058919/full |
| _version_ | 1811159012112072704 |
|---|---|
| author | Alexander Partin Thomas Brettin Yitan Zhu James M. Dolezal Sara Kochanny Alexander T. Pearson Maulik Shukla Yvonne A. Evrard James H. Doroshow Rick L. Stevens Rick L. Stevens |
| author_facet | Alexander Partin Thomas Brettin Yitan Zhu James M. Dolezal Sara Kochanny Alexander T. Pearson Maulik Shukla Yvonne A. Evrard James H. Doroshow Rick L. Stevens Rick L. Stevens |
| author_sort | Alexander Partin |
| collection | DOAJ |
| description | Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). We explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. We propose two data augmentation methods which allow us training multimodal and unimodal NNs without changing architectures with a single larger dataset: 1) combine single-drug and drug-pair treatments by homogenizing drug representations, and 2) augment drug-pairs which doubles the sample size of all drug-pair samples. Unimodal NNs which use GE are compared to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the MCC metric, MM-Net outperforms all the baselines. Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs. |
| first_indexed | 2024-04-10T05:33:57Z |
| format | Article |
| id | doaj.art-e0d0f63467c2490f8c2c2db626e08d8c |
| institution | Directory Open Access Journal |
| issn | 2296-858X |
| language | English |
| last_indexed | 2024-04-10T05:33:57Z |
| publishDate | 2023-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Medicine |
| spelling | doaj.art-e0d0f63467c2490f8c2c2db626e08d8c2023-03-07T04:44:10ZengFrontiers Media S.A.Frontiers in Medicine2296-858X2023-03-011010.3389/fmed.2023.10589191058919Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology imagesAlexander Partin0Thomas Brettin1Yitan Zhu2James M. Dolezal3Sara Kochanny4Alexander T. Pearson5Maulik Shukla6Yvonne A. Evrard7James H. Doroshow8Rick L. Stevens9Rick L. Stevens10Division of Data Science and Learning, Argonne National Laboratory, Lemont, IL, United StatesDivision of Data Science and Learning, Argonne National Laboratory, Lemont, IL, United StatesDivision of Data Science and Learning, Argonne National Laboratory, Lemont, IL, United StatesSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, United StatesSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, United StatesSection of Hematology/Oncology, Department of Medicine, University of Chicago Medical Center, Chicago, IL, United StatesDivision of Data Science and Learning, Argonne National Laboratory, Lemont, IL, United StatesFrederick National Laboratory for Cancer Research, Leidos Biomedical Research, Inc., Frederick, MD, United StatesDivision of Cancer Therapeutics and Diagnosis, National Cancer Institute, Bethesda, MD, United StatesDivision of Data Science and Learning, Argonne National Laboratory, Lemont, IL, United StatesDepartment of Computer Science, The University of Chicago, Chicago, IL, United StatesPatient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). We explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. We propose two data augmentation methods which allow us training multimodal and unimodal NNs without changing architectures with a single larger dataset: 1) combine single-drug and drug-pair treatments by homogenizing drug representations, and 2) augment drug-pairs which doubles the sample size of all drug-pair samples. Unimodal NNs which use GE are compared to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the MCC metric, MM-Net outperforms all the baselines. Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.https://www.frontiersin.org/articles/10.3389/fmed.2023.1058919/fulldrug response predictionhistology whole-slide imagesgene expressionmultimodal deep learningpreclinical drug studiesdata augmentation |
| spellingShingle | Alexander Partin Thomas Brettin Yitan Zhu James M. Dolezal Sara Kochanny Alexander T. Pearson Maulik Shukla Yvonne A. Evrard James H. Doroshow Rick L. Stevens Rick L. Stevens Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images Frontiers in Medicine drug response prediction histology whole-slide images gene expression multimodal deep learning preclinical drug studies data augmentation |
| title | Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images |
| title_full | Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images |
| title_fullStr | Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images |
| title_full_unstemmed | Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images |
| title_short | Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images |
| title_sort | data augmentation and multimodal learning for predicting drug response in patient derived xenografts from gene expressions and histology images |
| topic | drug response prediction histology whole-slide images gene expression multimodal deep learning preclinical drug studies data augmentation |
| url | https://www.frontiersin.org/articles/10.3389/fmed.2023.1058919/full |
| work_keys_str_mv | AT alexanderpartin dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT thomasbrettin dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT yitanzhu dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT jamesmdolezal dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT sarakochanny dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT alexandertpearson dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT maulikshukla dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT yvonneaevrard dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT jameshdoroshow dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT ricklstevens dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages AT ricklstevens dataaugmentationandmultimodallearningforpredictingdrugresponseinpatientderivedxenograftsfromgeneexpressionsandhistologyimages |