Spatial Channel Attention for Deep Convolutional Neural Networks
Recently, the attention mechanism combining spatial and channel information has been widely used in various deep convolutional neural networks (CNNs), proving its great potential in improving model performance. However, this usually uses 2D global pooling operations to compress spatial information o...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-05-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/10/10/1750 |
| _version_ | 1797498152824078336 |
|---|---|
| author | Tonglai Liu Ronghai Luo Longqin Xu Dachun Feng Liang Cao Shuangyin Liu Jianjun Guo |
| author_facet | Tonglai Liu Ronghai Luo Longqin Xu Dachun Feng Liang Cao Shuangyin Liu Jianjun Guo |
| author_sort | Tonglai Liu |
| collection | DOAJ |
| description | Recently, the attention mechanism combining spatial and channel information has been widely used in various deep convolutional neural networks (CNNs), proving its great potential in improving model performance. However, this usually uses 2D global pooling operations to compress spatial information or scaling methods to reduce the computational overhead in channel attention. These methods will result in severe information loss. Therefore, we propose a Spatial channel attention mechanism that captures cross-dimensional interaction, which does not involve dimensionality reduction and brings significant performance improvement with negligible computational overhead. The proposed attention mechanism can be seamlessly integrated into any convolutional neural network since it is a lightweight general module. Our method achieves a performance improvement of 2.08% on ResNet and 1.02% on MobileNetV2 in top-one error rate on the ImageNet dataset. |
| first_indexed | 2024-03-10T03:29:23Z |
| format | Article |
| id | doaj.art-e11afd6c9907493d8a54e8530712249c |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T03:29:23Z |
| publishDate | 2022-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-e11afd6c9907493d8a54e8530712249c2023-11-23T12:01:53ZengMDPI AGMathematics2227-73902022-05-011010175010.3390/math10101750Spatial Channel Attention for Deep Convolutional Neural NetworksTonglai Liu0Ronghai Luo1Longqin Xu2Dachun Feng3Liang Cao4Shuangyin Liu5Jianjun Guo6College of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaSchool of Information and Communication, Guilin University of Electronic Technology, Guilin 541004, ChinaCollege of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaRecently, the attention mechanism combining spatial and channel information has been widely used in various deep convolutional neural networks (CNNs), proving its great potential in improving model performance. However, this usually uses 2D global pooling operations to compress spatial information or scaling methods to reduce the computational overhead in channel attention. These methods will result in severe information loss. Therefore, we propose a Spatial channel attention mechanism that captures cross-dimensional interaction, which does not involve dimensionality reduction and brings significant performance improvement with negligible computational overhead. The proposed attention mechanism can be seamlessly integrated into any convolutional neural network since it is a lightweight general module. Our method achieves a performance improvement of 2.08% on ResNet and 1.02% on MobileNetV2 in top-one error rate on the ImageNet dataset.https://www.mdpi.com/2227-7390/10/10/1750attention mechanismimage classificationdeep learningcross-dimensional interaction |
| spellingShingle | Tonglai Liu Ronghai Luo Longqin Xu Dachun Feng Liang Cao Shuangyin Liu Jianjun Guo Spatial Channel Attention for Deep Convolutional Neural Networks Mathematics attention mechanism image classification deep learning cross-dimensional interaction |
| title | Spatial Channel Attention for Deep Convolutional Neural Networks |
| title_full | Spatial Channel Attention for Deep Convolutional Neural Networks |
| title_fullStr | Spatial Channel Attention for Deep Convolutional Neural Networks |
| title_full_unstemmed | Spatial Channel Attention for Deep Convolutional Neural Networks |
| title_short | Spatial Channel Attention for Deep Convolutional Neural Networks |
| title_sort | spatial channel attention for deep convolutional neural networks |
| topic | attention mechanism image classification deep learning cross-dimensional interaction |
| url | https://www.mdpi.com/2227-7390/10/10/1750 |
| work_keys_str_mv | AT tonglailiu spatialchannelattentionfordeepconvolutionalneuralnetworks AT ronghailuo spatialchannelattentionfordeepconvolutionalneuralnetworks AT longqinxu spatialchannelattentionfordeepconvolutionalneuralnetworks AT dachunfeng spatialchannelattentionfordeepconvolutionalneuralnetworks AT liangcao spatialchannelattentionfordeepconvolutionalneuralnetworks AT shuangyinliu spatialchannelattentionfordeepconvolutionalneuralnetworks AT jianjunguo spatialchannelattentionfordeepconvolutionalneuralnetworks |