Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy
Background: Deep learning tasks, which require large numbers of images, are widely applied in digital pathology. This poses challenges especially for supervised tasks since manual image annotation is an expensive and laborious process. This situation deteriorates even more in the case of a large var...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-01-01
|
| Series: | Journal of Pathology Informatics |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2153353923000093 |
| _version_ | 1811166127784460288 |
|---|---|
| author | Maryam Berijanian Nadine S. Schaadt Boqiang Huang Johannes Lotz Friedrich Feuerhake Dorit Merhof |
| author_facet | Maryam Berijanian Nadine S. Schaadt Boqiang Huang Johannes Lotz Friedrich Feuerhake Dorit Merhof |
| author_sort | Maryam Berijanian |
| collection | DOAJ |
| description | Background: Deep learning tasks, which require large numbers of images, are widely applied in digital pathology. This poses challenges especially for supervised tasks since manual image annotation is an expensive and laborious process. This situation deteriorates even more in the case of a large variability of images. Coping with this problem requires methods such as image augmentation and synthetic image generation. In this regard, unsupervised stain translation via GANs has gained much attention recently, but a separate network must be trained for each pair of source and target domains. This work enables unsupervised many-to-many translation of histopathological stains with a single network while seeking to maintain the shape and structure of the tissues. Methods: StarGAN-v2 is adapted for unsupervised many-to-many stain translation of histopathology images of breast tissues. An edge detector is incorporated to motivate the network to maintain the shape and structure of the tissues and to have an edge-preserving translation. Additionally, a subjective test is conducted on medical and technical experts in the field of digital pathology to evaluate the quality of generated images and to verify that they are indistinguishable from real images. As a proof of concept, breast cancer classifiers are trained with and without the generated images to quantify the effect of image augmentation using the synthetized images on classification accuracy. Results: The results show that adding an edge detector helps to improve the quality of translated images and to preserve the general structure of tissues. Quality control and subjective tests on our medical and technical experts show that the real and artificial images cannot be distinguished, thereby confirming that the synthetic images are technically plausible. Moreover, this research shows that, by augmenting the training dataset with the outputs of the proposed stain translation method, the accuracy of breast cancer classifier with ResNet-50 and VGG-16 improves by 8.0% and 9.3%, respectively. Conclusions: This research indicates that a translation from an arbitrary source stain to other stains can be performed effectively within the proposed framework. The generated images are realistic and could be employed to train deep neural networks to improve their performance and cope with the problem of insufficient numbers of annotated images. |
| first_indexed | 2024-04-10T15:48:37Z |
| format | Article |
| id | doaj.art-775f5b2df9cf479f97935b8b3cb55878 |
| institution | Directory Open Access Journal |
| issn | 2153-3539 |
| language | English |
| last_indexed | 2024-04-10T15:48:37Z |
| publishDate | 2023-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of Pathology Informatics |
| spelling | doaj.art-775f5b2df9cf479f97935b8b3cb558782023-02-12T04:14:37ZengElsevierJournal of Pathology Informatics2153-35392023-01-0114100195Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracyMaryam Berijanian0Nadine S. Schaadt1Boqiang Huang2Johannes Lotz3Friedrich Feuerhake4Dorit Merhof5Department of Computational Mathematics, Science and Engineering (CMSE), Michigan State University, East Lansing, USA; Institute of Imaging and Computer Vision, RWTH Aachen University, Aachen, GermanyInstitute for Pathology, Hannover Medical School, Hannover, GermanyInstitute of Image Analysis and Computer Vision, Faculty of Informatics and Data Science, University of Regensburg, Regensburg, GermanyFraunhofer Institute for Digital Medicine MEVIS, Lübeck, GermanyInstitute for Pathology, Hannover Medical School, Hannover, Germany; Institute for Neuropathology, University Clinic Freiburg, Freiburg, GermanyInstitute of Image Analysis and Computer Vision, Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany; Corresponding author at: University of Regensburg, 93040 Regensburg, Germany.Background: Deep learning tasks, which require large numbers of images, are widely applied in digital pathology. This poses challenges especially for supervised tasks since manual image annotation is an expensive and laborious process. This situation deteriorates even more in the case of a large variability of images. Coping with this problem requires methods such as image augmentation and synthetic image generation. In this regard, unsupervised stain translation via GANs has gained much attention recently, but a separate network must be trained for each pair of source and target domains. This work enables unsupervised many-to-many translation of histopathological stains with a single network while seeking to maintain the shape and structure of the tissues. Methods: StarGAN-v2 is adapted for unsupervised many-to-many stain translation of histopathology images of breast tissues. An edge detector is incorporated to motivate the network to maintain the shape and structure of the tissues and to have an edge-preserving translation. Additionally, a subjective test is conducted on medical and technical experts in the field of digital pathology to evaluate the quality of generated images and to verify that they are indistinguishable from real images. As a proof of concept, breast cancer classifiers are trained with and without the generated images to quantify the effect of image augmentation using the synthetized images on classification accuracy. Results: The results show that adding an edge detector helps to improve the quality of translated images and to preserve the general structure of tissues. Quality control and subjective tests on our medical and technical experts show that the real and artificial images cannot be distinguished, thereby confirming that the synthetic images are technically plausible. Moreover, this research shows that, by augmenting the training dataset with the outputs of the proposed stain translation method, the accuracy of breast cancer classifier with ResNet-50 and VGG-16 improves by 8.0% and 9.3%, respectively. Conclusions: This research indicates that a translation from an arbitrary source stain to other stains can be performed effectively within the proposed framework. The generated images are realistic and could be employed to train deep neural networks to improve their performance and cope with the problem of insufficient numbers of annotated images.http://www.sciencedirect.com/science/article/pii/S2153353923000093Deep learningStain translationUnsupervisedDigital pathologyBreast cancer classifierAdversarial networks |
| spellingShingle | Maryam Berijanian Nadine S. Schaadt Boqiang Huang Johannes Lotz Friedrich Feuerhake Dorit Merhof Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy Journal of Pathology Informatics Deep learning Stain translation Unsupervised Digital pathology Breast cancer classifier Adversarial networks |
| title | Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy |
| title_full | Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy |
| title_fullStr | Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy |
| title_full_unstemmed | Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy |
| title_short | Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy |
| title_sort | unsupervised many to many stain translation for histological image augmentation to improve classification accuracy |
| topic | Deep learning Stain translation Unsupervised Digital pathology Breast cancer classifier Adversarial networks |
| url | http://www.sciencedirect.com/science/article/pii/S2153353923000093 |
| work_keys_str_mv | AT maryamberijanian unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy AT nadinesschaadt unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy AT boqianghuang unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy AT johanneslotz unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy AT friedrichfeuerhake unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy AT doritmerhof unsupervisedmanytomanystaintranslationforhistologicalimageaugmentationtoimproveclassificationaccuracy |