Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification
Deep transfer learning aims at dealing with challenges in new tasks with insufficient samples. However, when it comes to few-shot learning scenarios, due to the low diversity of several known training samples, they are prone to be dominated by specificity, thus leading to one-sidedness local feature...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-05-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2021.654519/full |
| _version_ | 1819242344021491712 |
|---|---|
| author | Zhikui Chen Zhikui Chen Xu Zhang Wei Huang Jing Gao Jing Gao Suhua Zhang |
| author_facet | Zhikui Chen Zhikui Chen Xu Zhang Wei Huang Jing Gao Jing Gao Suhua Zhang |
| author_sort | Zhikui Chen |
| collection | DOAJ |
| description | Deep transfer learning aims at dealing with challenges in new tasks with insufficient samples. However, when it comes to few-shot learning scenarios, due to the low diversity of several known training samples, they are prone to be dominated by specificity, thus leading to one-sidedness local features instead of the reliable global feature of the actual categories they belong to. To alleviate the difficulty, we propose a cross-modal few-shot contextual transfer method that leverages the contextual information as a supplement and learns context awareness transfer in few-shot image classification scenes, which fully utilizes the information in heterogeneous data. The similarity measure in the image classification task is reformulated via fusing textual semantic modal information and visual semantic modal information extracted from images. This performs as a supplement and helps to inhibit the sample specificity. Besides, to better extract local visual features and reorganize the recognition pattern, the deep transfer scheme is also used for reusing a powerful extractor from the pre-trained model. Simulation experiments show that the introduction of cross-modal and intra-modal contextual information can effectively suppress the deviation of defining category features with few samples and improve the accuracy of few-shot image classification tasks. |
| first_indexed | 2024-12-23T14:38:18Z |
| format | Article |
| id | doaj.art-60eb3ae7451c47dbacafd56d210e2f18 |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-12-23T14:38:18Z |
| publishDate | 2021-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-60eb3ae7451c47dbacafd56d210e2f182022-12-21T17:43:18ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182021-05-011510.3389/fnbot.2021.654519654519Cross Modal Few-Shot Contextual Transfer for Heterogenous Image ClassificationZhikui Chen0Zhikui Chen1Xu Zhang2Wei Huang3Jing Gao4Jing Gao5Suhua Zhang6The School of Software Technology, Dalian University of Technology, Dalian, ChinaThe Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian, ChinaThe School of Software Technology, Dalian University of Technology, Dalian, ChinaDepartment of Critical Care Medicine, First Affiliated Hospital of Dalian Medical University, Dalian, ChinaThe School of Software Technology, Dalian University of Technology, Dalian, ChinaThe Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian, ChinaThe School of Software Technology, Dalian University of Technology, Dalian, ChinaDeep transfer learning aims at dealing with challenges in new tasks with insufficient samples. However, when it comes to few-shot learning scenarios, due to the low diversity of several known training samples, they are prone to be dominated by specificity, thus leading to one-sidedness local features instead of the reliable global feature of the actual categories they belong to. To alleviate the difficulty, we propose a cross-modal few-shot contextual transfer method that leverages the contextual information as a supplement and learns context awareness transfer in few-shot image classification scenes, which fully utilizes the information in heterogeneous data. The similarity measure in the image classification task is reformulated via fusing textual semantic modal information and visual semantic modal information extracted from images. This performs as a supplement and helps to inhibit the sample specificity. Besides, to better extract local visual features and reorganize the recognition pattern, the deep transfer scheme is also used for reusing a powerful extractor from the pre-trained model. Simulation experiments show that the introduction of cross-modal and intra-modal contextual information can effectively suppress the deviation of defining category features with few samples and improve the accuracy of few-shot image classification tasks.https://www.frontiersin.org/articles/10.3389/fnbot.2021.654519/fullfew-shot learningdeep transfer learningcontext awarenesscross modal informationimage classification |
| spellingShingle | Zhikui Chen Zhikui Chen Xu Zhang Wei Huang Jing Gao Jing Gao Suhua Zhang Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification Frontiers in Neurorobotics few-shot learning deep transfer learning context awareness cross modal information image classification |
| title | Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification |
| title_full | Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification |
| title_fullStr | Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification |
| title_full_unstemmed | Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification |
| title_short | Cross Modal Few-Shot Contextual Transfer for Heterogenous Image Classification |
| title_sort | cross modal few shot contextual transfer for heterogenous image classification |
| topic | few-shot learning deep transfer learning context awareness cross modal information image classification |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2021.654519/full |
| work_keys_str_mv | AT zhikuichen crossmodalfewshotcontextualtransferforheterogenousimageclassification AT zhikuichen crossmodalfewshotcontextualtransferforheterogenousimageclassification AT xuzhang crossmodalfewshotcontextualtransferforheterogenousimageclassification AT weihuang crossmodalfewshotcontextualtransferforheterogenousimageclassification AT jinggao crossmodalfewshotcontextualtransferforheterogenousimageclassification AT jinggao crossmodalfewshotcontextualtransferforheterogenousimageclassification AT suhuazhang crossmodalfewshotcontextualtransferforheterogenousimageclassification |