Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation
Annotating microscopy images for nuclei segmentation by medical experts is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/8/3/71 |
| _version_ | 1797470417331421184 |
|---|---|
| author | Ye Liu Sophia J. Wagner Tingying Peng |
| author_facet | Ye Liu Sophia J. Wagner Tingying Peng |
| author_sort | Ye Liu |
| collection | DOAJ |
| description | Annotating microscopy images for nuclei segmentation by medical experts is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with different imaging modalities, such as bright-field and fluorescence microscopy. Using disentangled representations for content and style, we can preserve the structure of the original image while altering its style during augmentation. We evaluate our data augmentation on the 2018 Data Science Bowl dataset consisting of various cell assays, lighting conditions, and imaging modalities. With our style augmentation, the segmentation accuracy of the two top-ranked Mask R-CNN-based nuclei segmentation algorithms in the competition increases significantly. Thus, our augmentation technique renders the downstream task more robust to the test data heterogeneity and helps counteract class imbalance without resampling of minority classes. |
| first_indexed | 2024-03-09T19:36:06Z |
| format | Article |
| id | doaj.art-f7f3305dee834997b7295a2bdc89121f |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-09T19:36:06Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-f7f3305dee834997b7295a2bdc89121f2023-11-24T01:55:06ZengMDPI AGJournal of Imaging2313-433X2022-03-01837110.3390/jimaging8030071Multi-Modality Microscopy Image Style Augmentation for Nuclei SegmentationYe Liu0Sophia J. Wagner1Tingying Peng2Department of Mathematics, Technical University Munich, 85748 Garching, GermanyDepartment of Mathematics, Technical University Munich, 85748 Garching, GermanyHelmholtz AI, Helmholtz Munich, 85764 Neuherberg, GermanyAnnotating microscopy images for nuclei segmentation by medical experts is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with different imaging modalities, such as bright-field and fluorescence microscopy. Using disentangled representations for content and style, we can preserve the structure of the original image while altering its style during augmentation. We evaluate our data augmentation on the 2018 Data Science Bowl dataset consisting of various cell assays, lighting conditions, and imaging modalities. With our style augmentation, the segmentation accuracy of the two top-ranked Mask R-CNN-based nuclei segmentation algorithms in the competition increases significantly. Thus, our augmentation technique renders the downstream task more robust to the test data heterogeneity and helps counteract class imbalance without resampling of minority classes.https://www.mdpi.com/2313-433X/8/3/71style transferdata augmentationnuclei segmentation |
| spellingShingle | Ye Liu Sophia J. Wagner Tingying Peng Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation Journal of Imaging style transfer data augmentation nuclei segmentation |
| title | Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation |
| title_full | Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation |
| title_fullStr | Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation |
| title_full_unstemmed | Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation |
| title_short | Multi-Modality Microscopy Image Style Augmentation for Nuclei Segmentation |
| title_sort | multi modality microscopy image style augmentation for nuclei segmentation |
| topic | style transfer data augmentation nuclei segmentation |
| url | https://www.mdpi.com/2313-433X/8/3/71 |
| work_keys_str_mv | AT yeliu multimodalitymicroscopyimagestyleaugmentationfornucleisegmentation AT sophiajwagner multimodalitymicroscopyimagestyleaugmentationfornucleisegmentation AT tingyingpeng multimodalitymicroscopyimagestyleaugmentationfornucleisegmentation |