Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras
Abstract Monitoring road conditions, e.g., water build-up due to intense rainfall, plays a fundamental role in ensuring road safety while increasing resilience to the effects of climate change. Distributed cameras provide an easy and affordable alternative to instrumented weather stations, enabling...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-12-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-26180-4 |
| _version_ | 1797973745004969984 |
|---|---|
| author | Fabio Garcea Giacomo Blanco Alberto Croci Fabrizio Lamberti Riccardo Mamone Ruben Ricupero Lia Morra Paola Allamano |
| author_facet | Fabio Garcea Giacomo Blanco Alberto Croci Fabrizio Lamberti Riccardo Mamone Ruben Ricupero Lia Morra Paola Allamano |
| author_sort | Fabio Garcea |
| collection | DOAJ |
| description | Abstract Monitoring road conditions, e.g., water build-up due to intense rainfall, plays a fundamental role in ensuring road safety while increasing resilience to the effects of climate change. Distributed cameras provide an easy and affordable alternative to instrumented weather stations, enabling diffused and capillary road monitoring. Here, we propose a deep learning-based solution to automatically detect wet road events in continuous video streams acquired by road-side surveillance cameras. Our contribution is two-fold: first, we employ a convolutional Long Short-Term Memory model (convLSTM) to detect subtle changes in the road appearance, introducing a novel temporally consistent data augmentation to increase robustness to outdoor illumination conditions. Second, we present a contrastive self-supervised framework that is uniquely tailored to surveillance camera networks. The proposed technique was validated on a large-scale dataset comprising roughly 2000 full day sequences (roughly 400K video frames, of which 300K unlabelled), acquired from several road-side cameras over a span of two years. Experimental results show the effectiveness of self-supervised and semi-supervised learning, increasing the frame classification performance (measured by the Area under the ROC curve) from 0.86 to 0.92. From the standpoint of event detection, we show that incorporating temporal features through a convLSTM model both improves the detection rate of wet road events (+ 10%) and reduces false positive alarms ( $$-$$ - 45%). The proposed techniques could benefit also other tasks related to weather analysis from road-side and vehicle-mounted cameras. |
| first_indexed | 2024-04-11T04:09:07Z |
| format | Article |
| id | doaj.art-ea16ba64c74048608eb7426a028768fb |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-11T04:09:07Z |
| publishDate | 2022-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-ea16ba64c74048608eb7426a028768fb2023-01-01T12:16:26ZengNature PortfolioScientific Reports2045-23222022-12-0112111810.1038/s41598-022-26180-4Self-supervised and semi-supervised learning for road condition estimation from distributed road-side camerasFabio Garcea0Giacomo Blanco1Alberto Croci2Fabrizio Lamberti3Riccardo Mamone4Ruben Ricupero5Lia Morra6Paola Allamano7Dipartimento di Automatica e Informatica, Politecnico di TorinoDipartimento di Automatica e Informatica, Politecnico di TorinoWaterview srlDipartimento di Automatica e Informatica, Politecnico di TorinoDipartimento di Automatica e Informatica, Politecnico di TorinoWaterview srlDipartimento di Automatica e Informatica, Politecnico di TorinoWaterview srlAbstract Monitoring road conditions, e.g., water build-up due to intense rainfall, plays a fundamental role in ensuring road safety while increasing resilience to the effects of climate change. Distributed cameras provide an easy and affordable alternative to instrumented weather stations, enabling diffused and capillary road monitoring. Here, we propose a deep learning-based solution to automatically detect wet road events in continuous video streams acquired by road-side surveillance cameras. Our contribution is two-fold: first, we employ a convolutional Long Short-Term Memory model (convLSTM) to detect subtle changes in the road appearance, introducing a novel temporally consistent data augmentation to increase robustness to outdoor illumination conditions. Second, we present a contrastive self-supervised framework that is uniquely tailored to surveillance camera networks. The proposed technique was validated on a large-scale dataset comprising roughly 2000 full day sequences (roughly 400K video frames, of which 300K unlabelled), acquired from several road-side cameras over a span of two years. Experimental results show the effectiveness of self-supervised and semi-supervised learning, increasing the frame classification performance (measured by the Area under the ROC curve) from 0.86 to 0.92. From the standpoint of event detection, we show that incorporating temporal features through a convLSTM model both improves the detection rate of wet road events (+ 10%) and reduces false positive alarms ( $$-$$ - 45%). The proposed techniques could benefit also other tasks related to weather analysis from road-side and vehicle-mounted cameras.https://doi.org/10.1038/s41598-022-26180-4 |
| spellingShingle | Fabio Garcea Giacomo Blanco Alberto Croci Fabrizio Lamberti Riccardo Mamone Ruben Ricupero Lia Morra Paola Allamano Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras Scientific Reports |
| title | Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras |
| title_full | Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras |
| title_fullStr | Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras |
| title_full_unstemmed | Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras |
| title_short | Self-supervised and semi-supervised learning for road condition estimation from distributed road-side cameras |
| title_sort | self supervised and semi supervised learning for road condition estimation from distributed road side cameras |
| url | https://doi.org/10.1038/s41598-022-26180-4 |
| work_keys_str_mv | AT fabiogarcea selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT giacomoblanco selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT albertocroci selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT fabriziolamberti selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT riccardomamone selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT rubenricupero selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT liamorra selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras AT paolaallamano selfsupervisedandsemisupervisedlearningforroadconditionestimationfromdistributedroadsidecameras |