A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement
An accurate object pose is essential to assess its state and predict its movements. In recent years, scholars have often predicted object poses by matching an image with a virtual 3D model or by regressing the six-degree-of-freedom pose of the target directly from the pixel data via deep learning me...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/20/8027 |
| _version_ | 1797469902930444288 |
|---|---|
| author | Wenjun Huang Wenbo Li Luqi Tang Xiaoming Zhu Bin Zou |
| author_facet | Wenjun Huang Wenbo Li Luqi Tang Xiaoming Zhu Bin Zou |
| author_sort | Wenjun Huang |
| collection | DOAJ |
| description | An accurate object pose is essential to assess its state and predict its movements. In recent years, scholars have often predicted object poses by matching an image with a virtual 3D model or by regressing the six-degree-of-freedom pose of the target directly from the pixel data via deep learning methods. However, these approaches may ignore a fact that was proposed in the early days of computer vision research, i.e., that object parts are strongly represented in the object pose. In this study, we propose a novel and lightweight deep learning framework, YAEN (yaw angle estimation network), for accurate object yaw angle prediction from a monocular camera based on the arrangement of parts. YAEN uses an encoding–decoding structure for vehicle yaw angle prediction. The vehicle part arrangement information is extracted by the part-encoding network, and the yaw angle is extracted from vehicle part arrangement information by the yaw angle decoding network. Because vehicle part information is refined by the encoder, the decoding network structure is lightweight; the YAEN model has low hardware requirements and can reach a detection speed of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>97</mn><mi>F</mi><mi>P</mi><mi>S</mi></mrow></semantics></math></inline-formula> on a <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>2070</mn><mi mathvariant="normal">s</mi><mo> </mo></mrow></semantics></math></inline-formula>graphics cards. To improve the performance of our model, we used asymmetric convolution and SSE (sum of squared errors) loss functions of adding the sign. To verify the effectiveness of this model, we constructed an accurate yaw angle dataset under real-world conditions with two vehicles equipped with high-precision positioning devices. Experimental results prove that our method can achieve satisfactory prediction performance in scenarios in which vehicles do not obscure each other, with an average prediction error of less than 3.1° and an accuracy of 96.45% for prediction errors of less than 10° in real driving scenarios. |
| first_indexed | 2024-03-09T19:29:32Z |
| format | Article |
| id | doaj.art-31341c78827f49d0b711b7745e2e49b1 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T19:29:32Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-31341c78827f49d0b711b7745e2e49b12023-11-24T02:30:34ZengMDPI AGSensors1424-82202022-10-012220802710.3390/s22208027A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part ArrangementWenjun Huang0Wenbo Li1Luqi Tang2Xiaoming Zhu3Bin Zou4Foshan Xianhu Laboratory of the Advanced Energy Science and Technology Guangdong Laboratory, Xianhu Hydrogen Valley, Foshan 528200, ChinaFoshan Xianhu Laboratory of the Advanced Energy Science and Technology Guangdong Laboratory, Xianhu Hydrogen Valley, Foshan 528200, ChinaFoshan Xianhu Laboratory of the Advanced Energy Science and Technology Guangdong Laboratory, Xianhu Hydrogen Valley, Foshan 528200, ChinaFoshan Xianhu Laboratory of the Advanced Energy Science and Technology Guangdong Laboratory, Xianhu Hydrogen Valley, Foshan 528200, ChinaFoshan Xianhu Laboratory of the Advanced Energy Science and Technology Guangdong Laboratory, Xianhu Hydrogen Valley, Foshan 528200, ChinaAn accurate object pose is essential to assess its state and predict its movements. In recent years, scholars have often predicted object poses by matching an image with a virtual 3D model or by regressing the six-degree-of-freedom pose of the target directly from the pixel data via deep learning methods. However, these approaches may ignore a fact that was proposed in the early days of computer vision research, i.e., that object parts are strongly represented in the object pose. In this study, we propose a novel and lightweight deep learning framework, YAEN (yaw angle estimation network), for accurate object yaw angle prediction from a monocular camera based on the arrangement of parts. YAEN uses an encoding–decoding structure for vehicle yaw angle prediction. The vehicle part arrangement information is extracted by the part-encoding network, and the yaw angle is extracted from vehicle part arrangement information by the yaw angle decoding network. Because vehicle part information is refined by the encoder, the decoding network structure is lightweight; the YAEN model has low hardware requirements and can reach a detection speed of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>97</mn><mi>F</mi><mi>P</mi><mi>S</mi></mrow></semantics></math></inline-formula> on a <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>2070</mn><mi mathvariant="normal">s</mi><mo> </mo></mrow></semantics></math></inline-formula>graphics cards. To improve the performance of our model, we used asymmetric convolution and SSE (sum of squared errors) loss functions of adding the sign. To verify the effectiveness of this model, we constructed an accurate yaw angle dataset under real-world conditions with two vehicles equipped with high-precision positioning devices. Experimental results prove that our method can achieve satisfactory prediction performance in scenarios in which vehicles do not obscure each other, with an average prediction error of less than 3.1° and an accuracy of 96.45% for prediction errors of less than 10° in real driving scenarios.https://www.mdpi.com/1424-8220/22/20/8027pose estimationyaw angle estimationconvolutional neural networkpart arrangementmonocular camera |
| spellingShingle | Wenjun Huang Wenbo Li Luqi Tang Xiaoming Zhu Bin Zou A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement Sensors pose estimation yaw angle estimation convolutional neural network part arrangement monocular camera |
| title | A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement |
| title_full | A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement |
| title_fullStr | A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement |
| title_full_unstemmed | A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement |
| title_short | A Deep Learning Framework for Accurate Vehicle Yaw Angle Estimation from a Monocular Camera Based on Part Arrangement |
| title_sort | deep learning framework for accurate vehicle yaw angle estimation from a monocular camera based on part arrangement |
| topic | pose estimation yaw angle estimation convolutional neural network part arrangement monocular camera |
| url | https://www.mdpi.com/1424-8220/22/20/8027 |
| work_keys_str_mv | AT wenjunhuang adeeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT wenboli adeeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT luqitang adeeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT xiaomingzhu adeeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT binzou adeeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT wenjunhuang deeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT wenboli deeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT luqitang deeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT xiaomingzhu deeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement AT binzou deeplearningframeworkforaccuratevehicleyawangleestimationfromamonocularcamerabasedonpartarrangement |