Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect
Deep learning techniques underpinned by extensive data sources encompassing complex pavement features have proven effective in early pavement damage detection. With pavement features exhibiting temperature variation, inexpensive infra-red imaging technology in combination with deep learning techniqu...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/23/9365 |
| _version_ | 1797462098841698304 |
|---|---|
| author | Sindhu Chandra Khaled AlMansoor Cheng Chen Yunfan Shi Hyungjoon Seo |
| author_facet | Sindhu Chandra Khaled AlMansoor Cheng Chen Yunfan Shi Hyungjoon Seo |
| author_sort | Sindhu Chandra |
| collection | DOAJ |
| description | Deep learning techniques underpinned by extensive data sources encompassing complex pavement features have proven effective in early pavement damage detection. With pavement features exhibiting temperature variation, inexpensive infra-red imaging technology in combination with deep learning techniques can detect pavement damages effectively. Previous experiments based on pavement data captured during summer sunny conditions when subjected to SA-ResNet deep learning architecture technique demonstrated 96.47% prediction accuracy. This paper has extended the same deep learning approach to a different dataset comprised of images captured during winter sunny conditions to compare the prediction accuracy, sensitivity and recall score with summer conditions. The results suggest that irrespective of the prevalent weather season, the proposed deep learning algorithm categorises pavement features around 92% accurately (95.18% in summer and 91.67% in winter conditions), suggesting the beneficial replacement of one image type with other. The data captured in sunny conditions during summer and winter show prediction accuracies of DC = 96.47% > MSX = 95.24% > IR-T = 93.83% and DC = 94.14% > MSX = 90.69% > IR-T = 90.173%, respectively. DC images demonstrated a sensitivity of 96.47% and 94.20% for summer and winter conditions, respectively, to demonstrate that reliable categorisation is possible with deep learning techniques irrespective of the weather season. However, summer conditions showing better overall prediction accuracy than winter conditions suggests that inexpensive IR-T imaging cameras with medium resolution levels can still be an economical solution, unlike expensive alternate options, but their usage has to be limited to summer sunny conditions. |
| first_indexed | 2024-03-09T17:31:41Z |
| format | Article |
| id | doaj.art-288483c9762845a4ab984b6559836526 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T17:31:41Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-288483c9762845a4ab984b65598365262023-11-24T12:13:00ZengMDPI AGSensors1424-82202022-12-012223936510.3390/s22239365Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal EffectSindhu Chandra0Khaled AlMansoor1Cheng Chen2Yunfan Shi3Hyungjoon Seo4Faculty of Engineering and Design, University of Bath, Architecture and Civil Engineering, Bath BA2 7AY, UKDepartment of Civil Engineering and Industrial Design, University of Liverpool, Liverpool L69 3BX, UKDepartment of Civil Engineering, Xi’an Jiaotong Liverpool University, Suzhou 215000, ChinaDepartment of Civil Engineering and Industrial Design, University of Liverpool, Liverpool L69 3BX, UKDepartment of Civil Engineering and Industrial Design, University of Liverpool, Liverpool L69 3BX, UKDeep learning techniques underpinned by extensive data sources encompassing complex pavement features have proven effective in early pavement damage detection. With pavement features exhibiting temperature variation, inexpensive infra-red imaging technology in combination with deep learning techniques can detect pavement damages effectively. Previous experiments based on pavement data captured during summer sunny conditions when subjected to SA-ResNet deep learning architecture technique demonstrated 96.47% prediction accuracy. This paper has extended the same deep learning approach to a different dataset comprised of images captured during winter sunny conditions to compare the prediction accuracy, sensitivity and recall score with summer conditions. The results suggest that irrespective of the prevalent weather season, the proposed deep learning algorithm categorises pavement features around 92% accurately (95.18% in summer and 91.67% in winter conditions), suggesting the beneficial replacement of one image type with other. The data captured in sunny conditions during summer and winter show prediction accuracies of DC = 96.47% > MSX = 95.24% > IR-T = 93.83% and DC = 94.14% > MSX = 90.69% > IR-T = 90.173%, respectively. DC images demonstrated a sensitivity of 96.47% and 94.20% for summer and winter conditions, respectively, to demonstrate that reliable categorisation is possible with deep learning techniques irrespective of the weather season. However, summer conditions showing better overall prediction accuracy than winter conditions suggests that inexpensive IR-T imaging cameras with medium resolution levels can still be an economical solution, unlike expensive alternate options, but their usage has to be limited to summer sunny conditions.https://www.mdpi.com/1424-8220/22/23/9365summer pavement defect detectionwinter pavement defect detectionmachine learningthermal analysismultichannel image fusion |
| spellingShingle | Sindhu Chandra Khaled AlMansoor Cheng Chen Yunfan Shi Hyungjoon Seo Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect Sensors summer pavement defect detection winter pavement defect detection machine learning thermal analysis multichannel image fusion |
| title | Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect |
| title_full | Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect |
| title_fullStr | Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect |
| title_full_unstemmed | Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect |
| title_short | Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect |
| title_sort | deep learning based infrared thermal image analysis of complex pavement defect conditions considering seasonal effect |
| topic | summer pavement defect detection winter pavement defect detection machine learning thermal analysis multichannel image fusion |
| url | https://www.mdpi.com/1424-8220/22/23/9365 |
| work_keys_str_mv | AT sindhuchandra deeplearningbasedinfraredthermalimageanalysisofcomplexpavementdefectconditionsconsideringseasonaleffect AT khaledalmansoor deeplearningbasedinfraredthermalimageanalysisofcomplexpavementdefectconditionsconsideringseasonaleffect AT chengchen deeplearningbasedinfraredthermalimageanalysisofcomplexpavementdefectconditionsconsideringseasonaleffect AT yunfanshi deeplearningbasedinfraredthermalimageanalysisofcomplexpavementdefectconditionsconsideringseasonaleffect AT hyungjoonseo deeplearningbasedinfraredthermalimageanalysisofcomplexpavementdefectconditionsconsideringseasonaleffect |