Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images
Visible image quality is very susceptible to changes in illumination, and there are limitations in ship classification using images acquired by a single sensor. This study proposes a ship classification method based on an attention mechanism and multi-scale convolutional neural network (MSCNN) for v...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/9/12/2022 |
| _version_ | 1797546257187602432 |
|---|---|
| author | Yongmei Ren Jie Yang Zhiqiang Guo Qingnian Zhang Hui Cao |
| author_facet | Yongmei Ren Jie Yang Zhiqiang Guo Qingnian Zhang Hui Cao |
| author_sort | Yongmei Ren |
| collection | DOAJ |
| description | Visible image quality is very susceptible to changes in illumination, and there are limitations in ship classification using images acquired by a single sensor. This study proposes a ship classification method based on an attention mechanism and multi-scale convolutional neural network (MSCNN) for visible and infrared images. First, the features of visible and infrared images are extracted by a two-stream symmetric multi-scale convolutional neural network module, and then concatenated to make full use of the complementary features present in multi-modal images. After that, the attention mechanism is applied to the concatenated fusion features to emphasize local details areas in the feature map, aiming to further improve feature representation capability of the model. Lastly, attention weights and the original concatenated fusion features are added element by element and fed into fully connected layers and Softmax output layer for final classification output. Effectiveness of the proposed method is verified on a visible and infrared spectra (VAIS) dataset, which shows 93.81% accuracy in classification results. Compared with other state-of-the-art methods, the proposed method could extract features more effectively and has better overall classification performance. |
| first_indexed | 2024-03-10T14:27:28Z |
| format | Article |
| id | doaj.art-b98329eb07dd4e99aadd3057c8c2be75 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T14:27:28Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-b98329eb07dd4e99aadd3057c8c2be752023-11-20T22:55:34ZengMDPI AGElectronics2079-92922020-11-01912202210.3390/electronics9122022Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared ImagesYongmei Ren0Jie Yang1Zhiqiang Guo2Qingnian Zhang3Hui Cao4Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, School of Information Engineering, Wuhan University of Technology, Wuhan 430070, ChinaHubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, School of Information Engineering, Wuhan University of Technology, Wuhan 430070, ChinaHubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, School of Information Engineering, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Transportation, Wuhan University of Technology, Wuhan 430070, ChinaHubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, School of Information Engineering, Wuhan University of Technology, Wuhan 430070, ChinaVisible image quality is very susceptible to changes in illumination, and there are limitations in ship classification using images acquired by a single sensor. This study proposes a ship classification method based on an attention mechanism and multi-scale convolutional neural network (MSCNN) for visible and infrared images. First, the features of visible and infrared images are extracted by a two-stream symmetric multi-scale convolutional neural network module, and then concatenated to make full use of the complementary features present in multi-modal images. After that, the attention mechanism is applied to the concatenated fusion features to emphasize local details areas in the feature map, aiming to further improve feature representation capability of the model. Lastly, attention weights and the original concatenated fusion features are added element by element and fed into fully connected layers and Softmax output layer for final classification output. Effectiveness of the proposed method is verified on a visible and infrared spectra (VAIS) dataset, which shows 93.81% accuracy in classification results. Compared with other state-of-the-art methods, the proposed method could extract features more effectively and has better overall classification performance.https://www.mdpi.com/2079-9292/9/12/2022ship classificationfeature fusionattention mechanismconvolutional neural networkinfrared imagevisible image |
| spellingShingle | Yongmei Ren Jie Yang Zhiqiang Guo Qingnian Zhang Hui Cao Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images Electronics ship classification feature fusion attention mechanism convolutional neural network infrared image visible image |
| title | Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images |
| title_full | Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images |
| title_fullStr | Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images |
| title_full_unstemmed | Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images |
| title_short | Ship Classification Based on Attention Mechanism and Multi-Scale Convolutional Neural Network for Visible and Infrared Images |
| title_sort | ship classification based on attention mechanism and multi scale convolutional neural network for visible and infrared images |
| topic | ship classification feature fusion attention mechanism convolutional neural network infrared image visible image |
| url | https://www.mdpi.com/2079-9292/9/12/2022 |
| work_keys_str_mv | AT yongmeiren shipclassificationbasedonattentionmechanismandmultiscaleconvolutionalneuralnetworkforvisibleandinfraredimages AT jieyang shipclassificationbasedonattentionmechanismandmultiscaleconvolutionalneuralnetworkforvisibleandinfraredimages AT zhiqiangguo shipclassificationbasedonattentionmechanismandmultiscaleconvolutionalneuralnetworkforvisibleandinfraredimages AT qingnianzhang shipclassificationbasedonattentionmechanismandmultiscaleconvolutionalneuralnetworkforvisibleandinfraredimages AT huicao shipclassificationbasedonattentionmechanismandmultiscaleconvolutionalneuralnetworkforvisibleandinfraredimages |