Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images
Automated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task remains challenging due to the high variation of defects in shapes...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/15/5137 |
| _version_ | 1797525113481986048 |
|---|---|
| author | Elham Eslami Hae-Bum Yun |
| author_facet | Elham Eslami Hae-Bum Yun |
| author_sort | Elham Eslami |
| collection | DOAJ |
| description | Automated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task remains challenging due to the high variation of defects in shapes and sizes, demanding a better incorporation of contextual information into deep networks. In this paper, we show that an attention-based multi-scale convolutional neural network (A+MCNN) improves the automated classification of common distress and non-distress objects in pavement images by (i) encoding contextual information through multi-scale input tiles and (ii) employing a mid-fusion approach with an attention module for heterogeneous image contexts from different input scales. A+MCNN is trained and tested with four distress classes (crack, crack seal, patch, pothole), five non-distress classes (joint, marker, manhole cover, curbing, shoulder), and two pavement classes (asphalt, concrete). A+MCNN is compared with four deep classifiers that are widely used in transportation applications and a generic CNN classifier (as the control model). The results show that A+MCNN consistently outperforms the baselines by 1∼26% on average in terms of the F-score. A comprehensive discussion is also presented regarding how these classifiers perform differently on different road objects, which has been rarely addressed in the existing literature. |
| first_indexed | 2024-03-10T09:08:06Z |
| format | Article |
| id | doaj.art-5340c0d20c2045948707b9a8613c9d5b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T09:08:06Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-5340c0d20c2045948707b9a8613c9d5b2023-11-22T06:11:07ZengMDPI AGSensors1424-82202021-07-012115513710.3390/s21155137Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road ImagesElham Eslami0Hae-Bum Yun1Civil, Environmental, and Construction, Engineering Department, University of Central Florida, Orlando, FL 32816, USACivil, Environmental, and Construction, Engineering Department, University of Central Florida, Orlando, FL 32816, USAAutomated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task remains challenging due to the high variation of defects in shapes and sizes, demanding a better incorporation of contextual information into deep networks. In this paper, we show that an attention-based multi-scale convolutional neural network (A+MCNN) improves the automated classification of common distress and non-distress objects in pavement images by (i) encoding contextual information through multi-scale input tiles and (ii) employing a mid-fusion approach with an attention module for heterogeneous image contexts from different input scales. A+MCNN is trained and tested with four distress classes (crack, crack seal, patch, pothole), five non-distress classes (joint, marker, manhole cover, curbing, shoulder), and two pavement classes (asphalt, concrete). A+MCNN is compared with four deep classifiers that are widely used in transportation applications and a generic CNN classifier (as the control model). The results show that A+MCNN consistently outperforms the baselines by 1∼26% on average in terms of the F-score. A comprehensive discussion is also presented regarding how these classifiers perform differently on different road objects, which has been rarely addressed in the existing literature.https://www.mdpi.com/1424-8220/21/15/5137smart infrastructure assessmentroad safetyautomated pavement condition assessmentconvolutional neural networkdeep learning |
| spellingShingle | Elham Eslami Hae-Bum Yun Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images Sensors smart infrastructure assessment road safety automated pavement condition assessment convolutional neural network deep learning |
| title | Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images |
| title_full | Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images |
| title_fullStr | Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images |
| title_full_unstemmed | Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images |
| title_short | Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images |
| title_sort | attention based multi scale convolutional neural network a mcnn for multi class classification in road images |
| topic | smart infrastructure assessment road safety automated pavement condition assessment convolutional neural network deep learning |
| url | https://www.mdpi.com/1424-8220/21/15/5137 |
| work_keys_str_mv | AT elhameslami attentionbasedmultiscaleconvolutionalneuralnetworkamcnnformulticlassclassificationinroadimages AT haebumyun attentionbasedmultiscaleconvolutionalneuralnetworkamcnnformulticlassclassificationinroadimages |