Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism
The use of 360° omnidirectional images has occurred widely in areas where comprehensive visual information is required due to their large visual field coverage. However, many extant convolutional neural networks based on 360° omnidirectional images have not performed well in computer vision tasks. T...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/23/12398 |
| _version_ | 1797463581272309760 |
|---|---|
| author | Xin Zhang Degang Yang Tingting Song Yichen Ye Jie Zhou Yingze Song |
| author_facet | Xin Zhang Degang Yang Tingting Song Yichen Ye Jie Zhou Yingze Song |
| author_sort | Xin Zhang |
| collection | DOAJ |
| description | The use of 360° omnidirectional images has occurred widely in areas where comprehensive visual information is required due to their large visual field coverage. However, many extant convolutional neural networks based on 360° omnidirectional images have not performed well in computer vision tasks. This occurs because 360° omnidirectional images are processed into plane images by equirectangular projection, which generates discontinuities at the edges and can result in serious distortion. At present, most methods to alleviate these problems are based on multi-projection and resampling, which can result in huge computational overhead. Therefore, a novel edge continuity distortion-aware block (ECDAB) for 360° omnidirectional images is proposed here, which prevents the discontinuity of edges and distortion by recombining and segmenting features. To further improve the performance of the network, a novel convolutional row-column attention block (CRCAB) is also proposed. CRCAB captures row-to-row and column-to-column dependencies to aggregate global information, enabling stronger representation of the extracted features. Moreover, to reduce the memory overhead of CRCAB, we propose an improved convolutional row-column attention block (ICRCAB), which can adjust the number of vectors in the row-column direction. Finally, to verify the effectiveness of the proposed networks, we conducted experiments on both traditional images and 360° omnidirectional image datasets. The experimental results demonstrated that better performance than for the baseline model was obtained by the network using ECDAB or CRCAB. |
| first_indexed | 2024-03-09T17:52:46Z |
| format | Article |
| id | doaj.art-e1c2ec8ef1b346b18a581ffdacb0cbcd |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T17:52:46Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-e1c2ec8ef1b346b18a581ffdacb0cbcd2023-11-24T10:35:51ZengMDPI AGApplied Sciences2076-34172022-12-0112231239810.3390/app122312398Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention MechanismXin Zhang0Degang Yang1Tingting Song2Yichen Ye3Jie Zhou4Yingze Song5College of Computer and Information Science, Chongqing Normal University, Chongqing 401331, ChinaCollege of Computer and Information Science, Chongqing Normal University, Chongqing 401331, ChinaCollege of Computer and Information Science, Chongqing Normal University, Chongqing 401331, ChinaCollege of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaCollege of Computer and Information Science, Chongqing Normal University, Chongqing 401331, ChinaCollege of Computer and Information Science, Chongqing Normal University, Chongqing 401331, ChinaThe use of 360° omnidirectional images has occurred widely in areas where comprehensive visual information is required due to their large visual field coverage. However, many extant convolutional neural networks based on 360° omnidirectional images have not performed well in computer vision tasks. This occurs because 360° omnidirectional images are processed into plane images by equirectangular projection, which generates discontinuities at the edges and can result in serious distortion. At present, most methods to alleviate these problems are based on multi-projection and resampling, which can result in huge computational overhead. Therefore, a novel edge continuity distortion-aware block (ECDAB) for 360° omnidirectional images is proposed here, which prevents the discontinuity of edges and distortion by recombining and segmenting features. To further improve the performance of the network, a novel convolutional row-column attention block (CRCAB) is also proposed. CRCAB captures row-to-row and column-to-column dependencies to aggregate global information, enabling stronger representation of the extracted features. Moreover, to reduce the memory overhead of CRCAB, we propose an improved convolutional row-column attention block (ICRCAB), which can adjust the number of vectors in the row-column direction. Finally, to verify the effectiveness of the proposed networks, we conducted experiments on both traditional images and 360° omnidirectional image datasets. The experimental results demonstrated that better performance than for the baseline model was obtained by the network using ECDAB or CRCAB.https://www.mdpi.com/2076-3417/12/23/12398computer visionobject detection360° omnidirectional imagesrow-column attention mechanism |
| spellingShingle | Xin Zhang Degang Yang Tingting Song Yichen Ye Jie Zhou Yingze Song Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism Applied Sciences computer vision object detection 360° omnidirectional images row-column attention mechanism |
| title | Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism |
| title_full | Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism |
| title_fullStr | Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism |
| title_full_unstemmed | Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism |
| title_short | Classification and Object Detection of 360° Omnidirectional Images Based on Continuity-Distortion Processing and Attention Mechanism |
| title_sort | classification and object detection of 360° omnidirectional images based on continuity distortion processing and attention mechanism |
| topic | computer vision object detection 360° omnidirectional images row-column attention mechanism |
| url | https://www.mdpi.com/2076-3417/12/23/12398 |
| work_keys_str_mv | AT xinzhang classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism AT degangyang classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism AT tingtingsong classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism AT yichenye classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism AT jiezhou classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism AT yingzesong classificationandobjectdetectionof360omnidirectionalimagesbasedoncontinuitydistortionprocessingandattentionmechanism |