Federated contrastive learning for decentralized unlabeled medical images
A label-efficient paradigm in computer vision is based on self-supervised contrastive pre-training on unlabeled data followed by fine-tuning with a small number of labels. Making practical use of a federated computing environment in the clinical domain and learning on medical images poses specific c...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Springer
2021
|
| _version_ | 1826260780071256064 |
|---|---|
| author | Dong, N Voiculescu, ID |
| author_facet | Dong, N Voiculescu, ID |
| author_sort | Dong, N |
| collection | OXFORD |
| description | A label-efficient paradigm in computer vision is based on self-supervised contrastive pre-training on unlabeled data followed by fine-tuning with a small number of labels. Making practical use of a federated computing environment in the clinical domain and learning on medical images poses specific challenges. In this work, we propose FedMoCo, a robust federated contrastive learning (FCL) framework, which makes efficient use of decentralized unlabeled medical data. FedMoCo has two novel modules: metadata transfer, an inter-node statistical data augmentation module, and self-adaptive aggregation, an aggregation module based on representational similarity analysis. To the best of our knowledge, this is the first FCL work on medical images. Our experiments show that FedMoCo can consistently outperform FedAvg, a seminal federated learning framework, in extracting meaningful representations for downstream tasks. We further show that FedMoCo can substantially reduce the amount of labeled data required in a downstream task, such as COVID-19 detection, to achieve a reasonable performance. |
| first_indexed | 2024-03-06T19:11:09Z |
| format | Conference item |
| id | oxford-uuid:16cdf7ad-e966-4a90-a4d0-57dd52fe79e9 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T19:11:09Z |
| publishDate | 2021 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:16cdf7ad-e966-4a90-a4d0-57dd52fe79e92022-03-26T10:33:36ZFederated contrastive learning for decentralized unlabeled medical imagesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:16cdf7ad-e966-4a90-a4d0-57dd52fe79e9EnglishSymplectic ElementsSpringer2021Dong, NVoiculescu, IDA label-efficient paradigm in computer vision is based on self-supervised contrastive pre-training on unlabeled data followed by fine-tuning with a small number of labels. Making practical use of a federated computing environment in the clinical domain and learning on medical images poses specific challenges. In this work, we propose FedMoCo, a robust federated contrastive learning (FCL) framework, which makes efficient use of decentralized unlabeled medical data. FedMoCo has two novel modules: metadata transfer, an inter-node statistical data augmentation module, and self-adaptive aggregation, an aggregation module based on representational similarity analysis. To the best of our knowledge, this is the first FCL work on medical images. Our experiments show that FedMoCo can consistently outperform FedAvg, a seminal federated learning framework, in extracting meaningful representations for downstream tasks. We further show that FedMoCo can substantially reduce the amount of labeled data required in a downstream task, such as COVID-19 detection, to achieve a reasonable performance. |
| spellingShingle | Dong, N Voiculescu, ID Federated contrastive learning for decentralized unlabeled medical images |
| title | Federated contrastive learning for decentralized unlabeled medical images |
| title_full | Federated contrastive learning for decentralized unlabeled medical images |
| title_fullStr | Federated contrastive learning for decentralized unlabeled medical images |
| title_full_unstemmed | Federated contrastive learning for decentralized unlabeled medical images |
| title_short | Federated contrastive learning for decentralized unlabeled medical images |
| title_sort | federated contrastive learning for decentralized unlabeled medical images |
| work_keys_str_mv | AT dongn federatedcontrastivelearningfordecentralizedunlabeledmedicalimages AT voiculescuid federatedcontrastivelearningfordecentralizedunlabeledmedicalimages |