Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation
Pose estimation is crucial for automating assembly tasks, yet achieving sufficient accuracy for assembly automation remains challenging and part-specific. This paper presents a novel, streamlined approach to pose estimation that facilitates automation of assembly tasks. Our proposed method employs d...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/13/6107 |
| _version_ | 1797436001835024384 |
|---|---|
| author | Debo Shi Alireza Rahimpour Amin Ghafourian Mohammad Mahdi Naddaf Shargh Devesh Upadhyay Ty A. Lasky Iman Soltani |
| author_facet | Debo Shi Alireza Rahimpour Amin Ghafourian Mohammad Mahdi Naddaf Shargh Devesh Upadhyay Ty A. Lasky Iman Soltani |
| author_sort | Debo Shi |
| collection | DOAJ |
| description | Pose estimation is crucial for automating assembly tasks, yet achieving sufficient accuracy for assembly automation remains challenging and part-specific. This paper presents a novel, streamlined approach to pose estimation that facilitates automation of assembly tasks. Our proposed method employs deep learning on a limited number of annotated images to identify a set of keypoints on the parts of interest. To compensate for network shortcomings and enhance accuracy we incorporated a Bayesian updating stage that leverages our detailed knowledge of the assembly part design. This Bayesian updating step refines the network output, significantly improving pose estimation accuracy. For this purpose, we utilized a subset of network-generated keypoint positions with higher quality as measurements, while for the remaining keypoints, the network outputs only serve as priors. The geometry data aid in constructing likelihood functions, which in turn result in enhanced posterior distributions of keypoint pixel positions. We then employed the maximum a posteriori (MAP) estimates of keypoint locations to obtain a final pose, allowing for an update to the nominal assembly trajectory. We evaluated our method on a 14-point snap-fit dash trim assembly for a Ford Mustang dashboard, demonstrating promising results. Our approach does not require tailoring to new applications, nor does it rely on extensive machine learning expertise or large amounts of training data. This makes our method a scalable and adaptable solution for the production floors. |
| first_indexed | 2024-03-09T10:56:23Z |
| format | Article |
| id | doaj.art-830442d115544f3e905c367a61acb620 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T10:56:23Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-830442d115544f3e905c367a61acb6202023-12-01T01:37:24ZengMDPI AGSensors1424-82202023-07-012313610710.3390/s23136107Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly AutomationDebo Shi0Alireza Rahimpour1Amin Ghafourian2Mohammad Mahdi Naddaf Shargh3Devesh Upadhyay4Ty A. Lasky5Iman Soltani6Department of Electrical and Computer Engineering, University of California Davis, Davis, CA 95616, USAGreenfield Labs, Ford Motor Company, Palo Alto, CA 94304, USADepartment of Mechanical and Aerospace Engineering, University of California Davis, Davis, CA 95616, USAGreenfield Labs, Ford Motor Company, Palo Alto, CA 94304, USAGreenfield Labs, Ford Motor Company, Palo Alto, CA 94304, USADepartment of Mechanical and Aerospace Engineering, University of California Davis, Davis, CA 95616, USADepartment of Mechanical and Aerospace Engineering, University of California Davis, Davis, CA 95616, USAPose estimation is crucial for automating assembly tasks, yet achieving sufficient accuracy for assembly automation remains challenging and part-specific. This paper presents a novel, streamlined approach to pose estimation that facilitates automation of assembly tasks. Our proposed method employs deep learning on a limited number of annotated images to identify a set of keypoints on the parts of interest. To compensate for network shortcomings and enhance accuracy we incorporated a Bayesian updating stage that leverages our detailed knowledge of the assembly part design. This Bayesian updating step refines the network output, significantly improving pose estimation accuracy. For this purpose, we utilized a subset of network-generated keypoint positions with higher quality as measurements, while for the remaining keypoints, the network outputs only serve as priors. The geometry data aid in constructing likelihood functions, which in turn result in enhanced posterior distributions of keypoint pixel positions. We then employed the maximum a posteriori (MAP) estimates of keypoint locations to obtain a final pose, allowing for an update to the nominal assembly trajectory. We evaluated our method on a 14-point snap-fit dash trim assembly for a Ford Mustang dashboard, demonstrating promising results. Our approach does not require tailoring to new applications, nor does it rely on extensive machine learning expertise or large amounts of training data. This makes our method a scalable and adaptable solution for the production floors.https://www.mdpi.com/1424-8220/23/13/6107keypoint detectionpose estimationassembly automationmanufacturing automationdeep learningAI |
| spellingShingle | Debo Shi Alireza Rahimpour Amin Ghafourian Mohammad Mahdi Naddaf Shargh Devesh Upadhyay Ty A. Lasky Iman Soltani Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation Sensors keypoint detection pose estimation assembly automation manufacturing automation deep learning AI |
| title | Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation |
| title_full | Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation |
| title_fullStr | Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation |
| title_full_unstemmed | Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation |
| title_short | Deep Bayesian-Assisted Keypoint Detection for Pose Estimation in Assembly Automation |
| title_sort | deep bayesian assisted keypoint detection for pose estimation in assembly automation |
| topic | keypoint detection pose estimation assembly automation manufacturing automation deep learning AI |
| url | https://www.mdpi.com/1424-8220/23/13/6107 |
| work_keys_str_mv | AT deboshi deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT alirezarahimpour deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT aminghafourian deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT mohammadmahdinaddafshargh deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT deveshupadhyay deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT tyalasky deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation AT imansoltani deepbayesianassistedkeypointdetectionforposeestimationinassemblyautomation |