Biopsy-free in vivo virtual histology of skin using deep learning
Abstract An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Publishing Group
2021-11-01
|
| Series: | Light: Science & Applications |
| Online Access: | https://doi.org/10.1038/s41377-021-00674-8 |
| _version_ | 1798017465479856128 |
|---|---|
| author | Jingxi Li Jason Garfinkel Xiaoran Zhang Di Wu Yijie Zhang Kevin de Haan Hongda Wang Tairan Liu Bijie Bai Yair Rivenson Gennady Rubinstein Philip O. Scumpia Aydogan Ozcan |
| author_facet | Jingxi Li Jason Garfinkel Xiaoran Zhang Di Wu Yijie Zhang Kevin de Haan Hongda Wang Tairan Liu Bijie Bai Yair Rivenson Gennady Rubinstein Philip O. Scumpia Aydogan Ozcan |
| author_sort | Jingxi Li |
| collection | DOAJ |
| description | Abstract An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies. |
| first_indexed | 2024-04-11T16:07:40Z |
| format | Article |
| id | doaj.art-a6bbc41d9a374c1398572da75fb519bf |
| institution | Directory Open Access Journal |
| issn | 2047-7538 |
| language | English |
| last_indexed | 2024-04-11T16:07:40Z |
| publishDate | 2021-11-01 |
| publisher | Nature Publishing Group |
| record_format | Article |
| series | Light: Science & Applications |
| spelling | doaj.art-a6bbc41d9a374c1398572da75fb519bf2022-12-22T04:14:47ZengNature Publishing GroupLight: Science & Applications2047-75382021-11-0110112210.1038/s41377-021-00674-8Biopsy-free in vivo virtual histology of skin using deep learningJingxi Li0Jason Garfinkel1Xiaoran Zhang2Di Wu3Yijie Zhang4Kevin de Haan5Hongda Wang6Tairan Liu7Bijie Bai8Yair Rivenson9Gennady Rubinstein10Philip O. Scumpia11Aydogan Ozcan12Electrical and Computer Engineering Department, University of CaliforniaDermatology and Laser CentreElectrical and Computer Engineering Department, University of CaliforniaComputer Science Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaDermatology and Laser CentreDivision of Dermatology, University of CaliforniaElectrical and Computer Engineering Department, University of CaliforniaAbstract An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.https://doi.org/10.1038/s41377-021-00674-8 |
| spellingShingle | Jingxi Li Jason Garfinkel Xiaoran Zhang Di Wu Yijie Zhang Kevin de Haan Hongda Wang Tairan Liu Bijie Bai Yair Rivenson Gennady Rubinstein Philip O. Scumpia Aydogan Ozcan Biopsy-free in vivo virtual histology of skin using deep learning Light: Science & Applications |
| title | Biopsy-free in vivo virtual histology of skin using deep learning |
| title_full | Biopsy-free in vivo virtual histology of skin using deep learning |
| title_fullStr | Biopsy-free in vivo virtual histology of skin using deep learning |
| title_full_unstemmed | Biopsy-free in vivo virtual histology of skin using deep learning |
| title_short | Biopsy-free in vivo virtual histology of skin using deep learning |
| title_sort | biopsy free in vivo virtual histology of skin using deep learning |
| url | https://doi.org/10.1038/s41377-021-00674-8 |
| work_keys_str_mv | AT jingxili biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT jasongarfinkel biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT xiaoranzhang biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT diwu biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT yijiezhang biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT kevindehaan biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT hongdawang biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT tairanliu biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT bijiebai biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT yairrivenson biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT gennadyrubinstein biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT philiposcumpia biopsyfreeinvivovirtualhistologyofskinusingdeeplearning AT aydoganozcan biopsyfreeinvivovirtualhistologyofskinusingdeeplearning |