Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images
Six-dimensional pose estimation for non-Lambertian objects, such as metal parts, is essential in intelligent manufacturing. Current methods pay much less attention to the influence of the surface reflection problem in 6D pose estimation. In this paper, we propose a cross-attention-based reflection-a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/10/12/1107 |
| _version_ | 1797456656830824448 |
|---|---|
| author | Chenrui Wu Long Chen Shiqing Wu |
| author_facet | Chenrui Wu Long Chen Shiqing Wu |
| author_sort | Chenrui Wu |
| collection | DOAJ |
| description | Six-dimensional pose estimation for non-Lambertian objects, such as metal parts, is essential in intelligent manufacturing. Current methods pay much less attention to the influence of the surface reflection problem in 6D pose estimation. In this paper, we propose a cross-attention-based reflection-aware 6D pose estimation network (CAR6D) for solving the surface reflection problem in 6D pose estimation. We use a pseudo-Siamese network structure to extract features from both an RGB image and a 3D model. The cross-attention layers are designed as a bi-directional filter for each of the inputs (the RGB image and 3D model) to focus on calculating the correspondences of the objects. The network is trained to segment the reflection area from the object area. Training images with ground-truth labels of the reflection area are generated with a physical-based rendering method. The experimental results on a 6D dataset of metal parts demonstrate the superiority of CAR6D in comparison with other state-of-the-art models. |
| first_indexed | 2024-03-09T16:10:58Z |
| format | Article |
| id | doaj.art-65c6b00ce23b4de78def2f599a385a8c |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-09T16:10:58Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-65c6b00ce23b4de78def2f599a385a8c2023-11-24T16:15:44ZengMDPI AGMachines2075-17022022-11-011012110710.3390/machines10121107Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB ImagesChenrui Wu0Long Chen1Shiqing Wu2College of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaCollege of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaCollege of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSix-dimensional pose estimation for non-Lambertian objects, such as metal parts, is essential in intelligent manufacturing. Current methods pay much less attention to the influence of the surface reflection problem in 6D pose estimation. In this paper, we propose a cross-attention-based reflection-aware 6D pose estimation network (CAR6D) for solving the surface reflection problem in 6D pose estimation. We use a pseudo-Siamese network structure to extract features from both an RGB image and a 3D model. The cross-attention layers are designed as a bi-directional filter for each of the inputs (the RGB image and 3D model) to focus on calculating the correspondences of the objects. The network is trained to segment the reflection area from the object area. Training images with ground-truth labels of the reflection area are generated with a physical-based rendering method. The experimental results on a 6D dataset of metal parts demonstrate the superiority of CAR6D in comparison with other state-of-the-art models.https://www.mdpi.com/2075-1702/10/12/11076D pose estimationnon-Lambertian objectsphysical-based renderingdense matchingPnP |
| spellingShingle | Chenrui Wu Long Chen Shiqing Wu Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images Machines 6D pose estimation non-Lambertian objects physical-based rendering dense matching PnP |
| title | Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images |
| title_full | Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images |
| title_fullStr | Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images |
| title_full_unstemmed | Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images |
| title_short | Cross-Attention-Based Reflection-Aware 6D Pose Estimation Network for Non-Lambertian Objects from RGB Images |
| title_sort | cross attention based reflection aware 6d pose estimation network for non lambertian objects from rgb images |
| topic | 6D pose estimation non-Lambertian objects physical-based rendering dense matching PnP |
| url | https://www.mdpi.com/2075-1702/10/12/1107 |
| work_keys_str_mv | AT chenruiwu crossattentionbasedreflectionaware6dposeestimationnetworkfornonlambertianobjectsfromrgbimages AT longchen crossattentionbasedreflectionaware6dposeestimationnetworkfornonlambertianobjectsfromrgbimages AT shiqingwu crossattentionbasedreflectionaware6dposeestimationnetworkfornonlambertianobjectsfromrgbimages |