Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision
The existing roadheader has problems, such as unable real-time positioning, inaccurate positioning, and positioning failure caused by camera view occlusion in visual positioning. In order to solve the above problems, a positioning scheme of the cantilever roadheader based on near-infrared binocular...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Industry and Mine Automation
2022-08-01
|
| Series: | Gong-kuang zidonghua |
| Subjects: | |
| Online Access: | http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.17896 |
| _version_ | 1827988588302696448 |
|---|---|
| author | WANG Xue ZHOU Hongxu ZHANG Lei WANG Huaying |
| author_facet | WANG Xue ZHOU Hongxu ZHANG Lei WANG Huaying |
| author_sort | WANG Xue |
| collection | DOAJ |
| description | The existing roadheader has problems, such as unable real-time positioning, inaccurate positioning, and positioning failure caused by camera view occlusion in visual positioning. In order to solve the above problems, a positioning scheme of the cantilever roadheader based on near-infrared binocular stereo vision is proposed. A near-infrared LED target is arranged on the fuselage and arm of cantilever roadheader. Taking LED as the near-infrared target, the characteristic information of the roadheader is constructed. The three-dimensional spatial positioning of the roadheader fuselage and the cutting part is realized through image processing and pose calculation. The binocular stereo vision camera is arranged at the top of a roadway. The distance between the roadheader and the binocular stereo vision camera gradually increases as the roadheader continues to advance. It leads to the failure of binocular image acquisition, which leads to the failure of the visual solution of the pose of the cutting part. In order to solve this problem, a magnetic field assisted positioning method of the cutting part based on one-dimensional convolution neural network (1D-CNN) is introduced. Three-axis digital magnetometers are arranged on two sides of the fuselage of the roadheader. The permanent magnet is arranged on the machine arm. The strength component of a magnetic field and pose data obtained by binocular stereo vision camera are used as training data to construct the 1D-CNN model, and the pose of a cutting part of the roadheader is output under the condition that vision measurement fails. The cantilever roadheader positioning based on near-infraared binocular stereo vision scheme is tested from the aspects of depth information and its fuselage of the roadheader and the cutting position are verified. The results showed that the measurement error of the fuselage is within ±11 mm, and the relative error is within 0.4%. The measurement error of the cutting part is within ±50 mm, and the relative error is within 1%.The relative pose error between the roadheader fuselage and the cutting part is within ±2.5°, the root-mean-square error of the pitch angle is 0.930 1°, and the root-mean-square error of the yaw angle is 0.922 0°. The errors are within the allowable range of roadway operation. The results show that the cantilever roadheader positioning scheme based on near-infrared binocular stereo vision is effective and reliable. The effectiveness of the magnetic field assisted positioning method based on 1D-CNN is verified. In order to simulate the complex magnetic field environment in coal mine underground, the interference magnetic source is randomly added near the roadheader. The results show that the predicted values of the pitch angle, yaw angle and rolling angle of the cutting part of the roadheader by this method are basically consistent with the measured real values. The determination coefficients of the predicted pitch angle, yaw angle and rolling angle are 0.992 4, 0.995 9 and 0.917 4 respectively. It shows that the magnetic field assisted positioning method of the cutting part of the roadheader based on 1D-CNN can better meet the positioning requirements of the roadheader in the case of visual positioning failure. |
| first_indexed | 2024-04-10T00:05:53Z |
| format | Article |
| id | doaj.art-b1c9012b3998496283b706a95b2158b8 |
| institution | Directory Open Access Journal |
| issn | 1671-251X |
| language | zho |
| last_indexed | 2024-04-10T00:05:53Z |
| publishDate | 2022-08-01 |
| publisher | Editorial Department of Industry and Mine Automation |
| record_format | Article |
| series | Gong-kuang zidonghua |
| spelling | doaj.art-b1c9012b3998496283b706a95b2158b82023-03-17T01:01:44ZzhoEditorial Department of Industry and Mine AutomationGong-kuang zidonghua1671-251X2022-08-014874351, 5710.13272/j.issn.1671-251x.17896Research on the cantilever roadheader positioning based on near-infrared binocular stereo visionWANG XueZHOU Hongxu0ZHANG LeiWANG HuayingSchool of Mathematics and Physics Science and Engineering, Hebei University of Engineering, Handan 056038, ChinaThe existing roadheader has problems, such as unable real-time positioning, inaccurate positioning, and positioning failure caused by camera view occlusion in visual positioning. In order to solve the above problems, a positioning scheme of the cantilever roadheader based on near-infrared binocular stereo vision is proposed. A near-infrared LED target is arranged on the fuselage and arm of cantilever roadheader. Taking LED as the near-infrared target, the characteristic information of the roadheader is constructed. The three-dimensional spatial positioning of the roadheader fuselage and the cutting part is realized through image processing and pose calculation. The binocular stereo vision camera is arranged at the top of a roadway. The distance between the roadheader and the binocular stereo vision camera gradually increases as the roadheader continues to advance. It leads to the failure of binocular image acquisition, which leads to the failure of the visual solution of the pose of the cutting part. In order to solve this problem, a magnetic field assisted positioning method of the cutting part based on one-dimensional convolution neural network (1D-CNN) is introduced. Three-axis digital magnetometers are arranged on two sides of the fuselage of the roadheader. The permanent magnet is arranged on the machine arm. The strength component of a magnetic field and pose data obtained by binocular stereo vision camera are used as training data to construct the 1D-CNN model, and the pose of a cutting part of the roadheader is output under the condition that vision measurement fails. The cantilever roadheader positioning based on near-infraared binocular stereo vision scheme is tested from the aspects of depth information and its fuselage of the roadheader and the cutting position are verified. The results showed that the measurement error of the fuselage is within ±11 mm, and the relative error is within 0.4%. The measurement error of the cutting part is within ±50 mm, and the relative error is within 1%.The relative pose error between the roadheader fuselage and the cutting part is within ±2.5°, the root-mean-square error of the pitch angle is 0.930 1°, and the root-mean-square error of the yaw angle is 0.922 0°. The errors are within the allowable range of roadway operation. The results show that the cantilever roadheader positioning scheme based on near-infrared binocular stereo vision is effective and reliable. The effectiveness of the magnetic field assisted positioning method based on 1D-CNN is verified. In order to simulate the complex magnetic field environment in coal mine underground, the interference magnetic source is randomly added near the roadheader. The results show that the predicted values of the pitch angle, yaw angle and rolling angle of the cutting part of the roadheader by this method are basically consistent with the measured real values. The determination coefficients of the predicted pitch angle, yaw angle and rolling angle are 0.992 4, 0.995 9 and 0.917 4 respectively. It shows that the magnetic field assisted positioning method of the cutting part of the roadheader based on 1D-CNN can better meet the positioning requirements of the roadheader in the case of visual positioning failure.http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.17896cantilever roadheadercutting position posereal-time positioning of cutting partnear-infrared binocular visionpose calculationone-dimensional convolutional neural networkmagnetic field assisted positioning |
| spellingShingle | WANG Xue ZHOU Hongxu ZHANG Lei WANG Huaying Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision Gong-kuang zidonghua cantilever roadheader cutting position pose real-time positioning of cutting part near-infrared binocular vision pose calculation one-dimensional convolutional neural network magnetic field assisted positioning |
| title | Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision |
| title_full | Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision |
| title_fullStr | Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision |
| title_full_unstemmed | Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision |
| title_short | Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision |
| title_sort | research on the cantilever roadheader positioning based on near infrared binocular stereo vision |
| topic | cantilever roadheader cutting position pose real-time positioning of cutting part near-infrared binocular vision pose calculation one-dimensional convolutional neural network magnetic field assisted positioning |
| url | http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.17896 |
| work_keys_str_mv | AT wangxue researchonthecantileverroadheaderpositioningbasedonnearinfraredbinocularstereovision AT zhouhongxu researchonthecantileverroadheaderpositioningbasedonnearinfraredbinocularstereovision AT zhanglei researchonthecantileverroadheaderpositioningbasedonnearinfraredbinocularstereovision AT wanghuaying researchonthecantileverroadheaderpositioningbasedonnearinfraredbinocularstereovision |