Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition
As eating-out became an indispensable part of our daily lives, demand for the food recognition of unfamiliar restaurant increased significantly due to health-care. Although there are many researches on generic food recognition, there are relatively fewer studies on restaurant food image recognition....
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8952686/ |
| _version_ | 1818853249688535040 |
|---|---|
| author | Gege Song Zhulin Tao Xianglin Huang Gang Cao Wei Liu Lifang Yang |
| author_facet | Gege Song Zhulin Tao Xianglin Huang Gang Cao Wei Liu Lifang Yang |
| author_sort | Gege Song |
| collection | DOAJ |
| description | As eating-out became an indispensable part of our daily lives, demand for the food recognition of unfamiliar restaurant increased significantly due to health-care. Although there are many researches on generic food recognition, there are relatively fewer studies on restaurant food image recognition. Meanwhile, it becomes extremely challenging for restaurant food image recognition due to insufficient food image. Prototypical network is common utilized to address the such a task in recent years. Although the methods based on prototypical network achieve impressive results in capturing similarities feature of the same food category, it fails to highlight important information on feature and instance level. Toward this end, we propose an effective food image recognition scheme by incorporating hybrid attention mechanism into prototypical network in this paper. Specifically, the image feature is first captured by convolutional neural network (CNN). Then the image attention weights yielded by instance-based attention mechanism are used to modulate the image feature of CNN for constructing class prototypes. And feature-based attention mechanism is employed to grasp important information of image for enriching image representation. Extensive experimental results on the large food image dataset verify that the performance of our proposed classification scheme outperforms the state-of-the-art ones. |
| first_indexed | 2024-12-19T07:33:49Z |
| format | Article |
| id | doaj.art-bf603968e60440878323bb54a490af50 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T07:33:49Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-bf603968e60440878323bb54a490af502022-12-21T20:30:37ZengIEEEIEEE Access2169-35362020-01-018148931490010.1109/ACCESS.2020.29648368952686Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot RecognitionGege Song0https://orcid.org/0000-0002-6739-2219Zhulin Tao1https://orcid.org/0000-0001-9011-8464Xianglin Huang2https://orcid.org/0000-0003-0324-4687Gang Cao3https://orcid.org/0000-0002-4549-0125Wei Liu4https://orcid.org/0000-0002-0038-6519Lifang Yang5https://orcid.org/0000-0003-1275-9244School of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaSchool of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaSchool of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaSchool of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaSchool of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaSchool of Computer Science and Cybersecurity, Communication University of China, Beijing, ChinaAs eating-out became an indispensable part of our daily lives, demand for the food recognition of unfamiliar restaurant increased significantly due to health-care. Although there are many researches on generic food recognition, there are relatively fewer studies on restaurant food image recognition. Meanwhile, it becomes extremely challenging for restaurant food image recognition due to insufficient food image. Prototypical network is common utilized to address the such a task in recent years. Although the methods based on prototypical network achieve impressive results in capturing similarities feature of the same food category, it fails to highlight important information on feature and instance level. Toward this end, we propose an effective food image recognition scheme by incorporating hybrid attention mechanism into prototypical network in this paper. Specifically, the image feature is first captured by convolutional neural network (CNN). Then the image attention weights yielded by instance-based attention mechanism are used to modulate the image feature of CNN for constructing class prototypes. And feature-based attention mechanism is employed to grasp important information of image for enriching image representation. Extensive experimental results on the large food image dataset verify that the performance of our proposed classification scheme outperforms the state-of-the-art ones.https://ieeexplore.ieee.org/document/8952686/Restaurant food image recognitionunfamiliar restaurantprototypical networkhybrid attention mechanism |
| spellingShingle | Gege Song Zhulin Tao Xianglin Huang Gang Cao Wei Liu Lifang Yang Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition IEEE Access Restaurant food image recognition unfamiliar restaurant prototypical network hybrid attention mechanism |
| title | Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition |
| title_full | Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition |
| title_fullStr | Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition |
| title_full_unstemmed | Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition |
| title_short | Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition |
| title_sort | hybrid attention based prototypical network for unfamiliar restaurant food image few shot recognition |
| topic | Restaurant food image recognition unfamiliar restaurant prototypical network hybrid attention mechanism |
| url | https://ieeexplore.ieee.org/document/8952686/ |
| work_keys_str_mv | AT gegesong hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition AT zhulintao hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition AT xianglinhuang hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition AT gangcao hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition AT weiliu hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition AT lifangyang hybridattentionbasedprototypicalnetworkforunfamiliarrestaurantfoodimagefewshotrecognition |