A GNN Architecture With Local and Global-Attention Feature for Image Classification
Convolutional neural network (CNN) is quite popular in computer vision, especially in image classification with excellent performance. However, limited by the convolution kernels, CNN-based classifiers are hard to extract global feature from the original image, while exact object locations in the en...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10148955/ |
| _version_ | 1797660816258891776 |
|---|---|
| author | Zhengshun Fei Junhao Guo Haibo Gong Lubin Ye Eric Attahi Bingqiang Huang |
| author_facet | Zhengshun Fei Junhao Guo Haibo Gong Lubin Ye Eric Attahi Bingqiang Huang |
| author_sort | Zhengshun Fei |
| collection | DOAJ |
| description | Convolutional neural network (CNN) is quite popular in computer vision, especially in image classification with excellent performance. However, limited by the convolution kernels, CNN-based classifiers are hard to extract global feature from the original image, while exact object locations in the environment are included in the global feature. One popular way to improve global feature extraction performance is to use graph neural network (GNN) which can aggregate global information through the connection relationship of different nodes. In this work, a novel end-to-end graph neural network architecture is proposed, in which local and global-attention feature are used simultaneously to achieve more accurate predictions. In this architecture, a CNN block is designed to learn local feature and graph convolutional neural network (GCN) is used to learn global feature. Global-attention feature for final prediction is down-sampled from global feature by the proposed global multi-head self-attention pooling (GMSAPool) based on self-attention mechanism, which reconstructs the input graph by introducing virtual node and automatically assigns different weights to each node to obtain a more representative global-attention feature. In addition, the proposed architecture can be trained without converting images to graphs in advance, and the computational burden can also be reduced. This approach is demonstrated on three open datasets (Agricultural Disease, Caltech256 and CIFAR-100) to validate the effectiveness. The determined experimental results showed that: 1) The proposed model achieve 84.46%, 77.80%, and 83.33% on the Macro-F1 in three datasets respectively, improving over the best baselines; 2) Global-attention feature that is more conducive to the final prediction is extracted by GMSAPool from numerous nodes, in which Macro-P,Macro-R and Macro-F1 are respectively improved 3.655%,1.12%,2.715% on average. |
| first_indexed | 2024-03-11T18:36:18Z |
| format | Article |
| id | doaj.art-dbf3ba4f16fd44168d237e5ed41da539 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T18:36:18Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-dbf3ba4f16fd44168d237e5ed41da5392023-10-12T23:01:15ZengIEEEIEEE Access2169-35362023-01-011111022111023310.1109/ACCESS.2023.328524610148955A GNN Architecture With Local and Global-Attention Feature for Image ClassificationZhengshun Fei0https://orcid.org/0000-0001-8111-690XJunhao Guo1https://orcid.org/0000-0001-7907-0260Haibo Gong2Lubin Ye3Eric Attahi4Bingqiang Huang5https://orcid.org/0000-0003-0551-7263School of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaSchool of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaSchool of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaHangzhou Vocational and Technical College, Information Engineering Institute, Hangzhou, ChinaSchool of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaSchool of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou, ChinaConvolutional neural network (CNN) is quite popular in computer vision, especially in image classification with excellent performance. However, limited by the convolution kernels, CNN-based classifiers are hard to extract global feature from the original image, while exact object locations in the environment are included in the global feature. One popular way to improve global feature extraction performance is to use graph neural network (GNN) which can aggregate global information through the connection relationship of different nodes. In this work, a novel end-to-end graph neural network architecture is proposed, in which local and global-attention feature are used simultaneously to achieve more accurate predictions. In this architecture, a CNN block is designed to learn local feature and graph convolutional neural network (GCN) is used to learn global feature. Global-attention feature for final prediction is down-sampled from global feature by the proposed global multi-head self-attention pooling (GMSAPool) based on self-attention mechanism, which reconstructs the input graph by introducing virtual node and automatically assigns different weights to each node to obtain a more representative global-attention feature. In addition, the proposed architecture can be trained without converting images to graphs in advance, and the computational burden can also be reduced. This approach is demonstrated on three open datasets (Agricultural Disease, Caltech256 and CIFAR-100) to validate the effectiveness. The determined experimental results showed that: 1) The proposed model achieve 84.46%, 77.80%, and 83.33% on the Macro-F1 in three datasets respectively, improving over the best baselines; 2) Global-attention feature that is more conducive to the final prediction is extracted by GMSAPool from numerous nodes, in which Macro-P,Macro-R and Macro-F1 are respectively improved 3.655%,1.12%,2.715% on average.https://ieeexplore.ieee.org/document/10148955/Global-attention featureself-attentiongraph global poolinggraph convolutional neural networkconvolutional neural networkimage processing |
| spellingShingle | Zhengshun Fei Junhao Guo Haibo Gong Lubin Ye Eric Attahi Bingqiang Huang A GNN Architecture With Local and Global-Attention Feature for Image Classification IEEE Access Global-attention feature self-attention graph global pooling graph convolutional neural network convolutional neural network image processing |
| title | A GNN Architecture With Local and Global-Attention Feature for Image Classification |
| title_full | A GNN Architecture With Local and Global-Attention Feature for Image Classification |
| title_fullStr | A GNN Architecture With Local and Global-Attention Feature for Image Classification |
| title_full_unstemmed | A GNN Architecture With Local and Global-Attention Feature for Image Classification |
| title_short | A GNN Architecture With Local and Global-Attention Feature for Image Classification |
| title_sort | gnn architecture with local and global attention feature for image classification |
| topic | Global-attention feature self-attention graph global pooling graph convolutional neural network convolutional neural network image processing |
| url | https://ieeexplore.ieee.org/document/10148955/ |
| work_keys_str_mv | AT zhengshunfei agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT junhaoguo agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT haibogong agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT lubinye agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT ericattahi agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT bingqianghuang agnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT zhengshunfei gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT junhaoguo gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT haibogong gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT lubinye gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT ericattahi gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification AT bingqianghuang gnnarchitecturewithlocalandglobalattentionfeatureforimageclassification |