ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network
Abstract Lane detection is one of the key techniques to realize advanced driving assistance and automatic driving. However, lane detection networks based on deep learning have significant shortcomings. The detection results are often unsatisfactory when there are shadows, degraded lane markings, and...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2022-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-022-00909-0 |
| _version_ | 1797674956059836416 |
|---|---|
| author | Rongyun Zhang Yufeng Du Peicheng Shi Lifeng Zhao Yaming Liu Haoran Li |
| author_facet | Rongyun Zhang Yufeng Du Peicheng Shi Lifeng Zhao Yaming Liu Haoran Li |
| author_sort | Rongyun Zhang |
| collection | DOAJ |
| description | Abstract Lane detection is one of the key techniques to realize advanced driving assistance and automatic driving. However, lane detection networks based on deep learning have significant shortcomings. The detection results are often unsatisfactory when there are shadows, degraded lane markings, and vehicle occlusion lanes. Therefore, a continuous multi-frame image sequence lane detection network is proposed. Specifically, the continuous six-frame image sequence is input into the network, in which the scene information of each frame image is extracted by an encoder composed of Swin Transformer blocks and input into the PredRNN. Continuous multi-frame of the driving scene is modeled as time-series by ST-LSTM blocks, and then, the shape changes and motion trajectory in the spatiotemporal sequence are effectively modeled. Finally, through the decoder composed of Swin Transformer blocks, the features are obtained and reconstructed to complete the detection task. Extensive experiments on two large-scale datasets demonstrate that the proposed method outperforms the competing methods in lane detection, especially in handling difficult situations. Experiments are carried out based on the TuSimple dataset. The results show: for easy scenes, the validation accuracy is 97.46%, the test accuracy is 97.37%, and the precision is 0.865. For complex scenes, the validation accuracy is 97.38%, the test accuracy is 97.29%, and the precision is 0.859. The running time is 4.4 ms. Experiments are carried out based on the CULane dataset. The results show that, for easy scenes, the validation accuracy is 97.03%, the test accuracy is 96.84%, and the precision is 0.837. For complex scenes, the validation accuracy is 96.18%, the test accuracy is 95.92%, and the precision is 0.829. The running time is 6.5 ms. |
| first_indexed | 2024-03-11T22:07:40Z |
| format | Article |
| id | doaj.art-2d94c6a5b779466e849e76c01336eb0e |
| institution | Directory Open Access Journal |
| issn | 2199-4536 2198-6053 |
| language | English |
| last_indexed | 2024-03-11T22:07:40Z |
| publishDate | 2022-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj.art-2d94c6a5b779466e849e76c01336eb0e2023-09-24T11:35:22ZengSpringerComplex & Intelligent Systems2199-45362198-60532022-11-01954837485510.1007/s40747-022-00909-0ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid networkRongyun Zhang0Yufeng Du1Peicheng Shi2Lifeng Zhao3Yaming Liu4Haoran Li5The School of Mechanical and Automotive Engineering, Anhui Polytechnic UniversityThe School of Mechanical and Automotive Engineering, Anhui Polytechnic UniversityAutomotive New Technology Anhui Engineering and Technology Research Center, Anhui Polytechnic UniversityThe School of Automotive and Traffic Engineering, Hefei University of TechnologyThe School of Mechanical and Automotive Engineering, Anhui Polytechnic UniversityThe School of Mechanical and Automotive Engineering, Anhui Polytechnic UniversityAbstract Lane detection is one of the key techniques to realize advanced driving assistance and automatic driving. However, lane detection networks based on deep learning have significant shortcomings. The detection results are often unsatisfactory when there are shadows, degraded lane markings, and vehicle occlusion lanes. Therefore, a continuous multi-frame image sequence lane detection network is proposed. Specifically, the continuous six-frame image sequence is input into the network, in which the scene information of each frame image is extracted by an encoder composed of Swin Transformer blocks and input into the PredRNN. Continuous multi-frame of the driving scene is modeled as time-series by ST-LSTM blocks, and then, the shape changes and motion trajectory in the spatiotemporal sequence are effectively modeled. Finally, through the decoder composed of Swin Transformer blocks, the features are obtained and reconstructed to complete the detection task. Extensive experiments on two large-scale datasets demonstrate that the proposed method outperforms the competing methods in lane detection, especially in handling difficult situations. Experiments are carried out based on the TuSimple dataset. The results show: for easy scenes, the validation accuracy is 97.46%, the test accuracy is 97.37%, and the precision is 0.865. For complex scenes, the validation accuracy is 97.38%, the test accuracy is 97.29%, and the precision is 0.859. The running time is 4.4 ms. Experiments are carried out based on the CULane dataset. The results show that, for easy scenes, the validation accuracy is 97.03%, the test accuracy is 96.84%, and the precision is 0.837. For complex scenes, the validation accuracy is 96.18%, the test accuracy is 95.92%, and the precision is 0.829. The running time is 6.5 ms.https://doi.org/10.1007/s40747-022-00909-0Lane detectionDeep learningSwin transformerContinuous multi-frame |
| spellingShingle | Rongyun Zhang Yufeng Du Peicheng Shi Lifeng Zhao Yaming Liu Haoran Li ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network Complex & Intelligent Systems Lane detection Deep learning Swin transformer Continuous multi-frame |
| title | ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network |
| title_full | ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network |
| title_fullStr | ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network |
| title_full_unstemmed | ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network |
| title_short | ST-MAE: robust lane detection in continuous multi-frame driving scenes based on a deep hybrid network |
| title_sort | st mae robust lane detection in continuous multi frame driving scenes based on a deep hybrid network |
| topic | Lane detection Deep learning Swin transformer Continuous multi-frame |
| url | https://doi.org/10.1007/s40747-022-00909-0 |
| work_keys_str_mv | AT rongyunzhang stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork AT yufengdu stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork AT peichengshi stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork AT lifengzhao stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork AT yamingliu stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork AT haoranli stmaerobustlanedetectionincontinuousmultiframedrivingscenesbasedonadeephybridnetwork |