OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over
In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods f...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/18/7807 |
| _version_ | 1797577050024837120 |
|---|---|
| author | Benedict Stephan Mona Köhler Steffen Müller Yan Zhang Horst-Michael Gross Gunther Notni |
| author_facet | Benedict Stephan Mona Köhler Steffen Müller Yan Zhang Horst-Michael Gross Gunther Notni |
| author_sort | Benedict Stephan |
| collection | DOAJ |
| description | In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application. |
| first_indexed | 2024-03-10T22:02:25Z |
| format | Article |
| id | doaj.art-78db197f016d454eb417591251d13255 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T22:02:25Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-78db197f016d454eb417591251d132552023-11-19T12:54:28ZengMDPI AGSensors1424-82202023-09-012318780710.3390/s23187807OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-OverBenedict Stephan0Mona Köhler1Steffen Müller2Yan Zhang3Horst-Michael Gross4Gunther Notni5Neuroinformatics and Cognitive Robotics Lab, Technische Universität Ilmenau, 98693 Ilmenau, GermanyNeuroinformatics and Cognitive Robotics Lab, Technische Universität Ilmenau, 98693 Ilmenau, GermanyNeuroinformatics and Cognitive Robotics Lab, Technische Universität Ilmenau, 98693 Ilmenau, GermanyGroup for Quality Assurance and Industrial Image Processing, Technische Universität Ilmenau, 98693 Ilmenau, GermanyNeuroinformatics and Cognitive Robotics Lab, Technische Universität Ilmenau, 98693 Ilmenau, GermanyGroup for Quality Assurance and Industrial Image Processing, Technische Universität Ilmenau, 98693 Ilmenau, GermanyIn the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application.https://www.mdpi.com/1424-8220/23/18/7807datasetthermal imagesemantic segmentationhand-over6D pose estimationautomated labeling |
| spellingShingle | Benedict Stephan Mona Köhler Steffen Müller Yan Zhang Horst-Michael Gross Gunther Notni OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over Sensors dataset thermal image semantic segmentation hand-over 6D pose estimation automated labeling |
| title | OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over |
| title_full | OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over |
| title_fullStr | OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over |
| title_full_unstemmed | OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over |
| title_short | OHO: A Multi-Modal, Multi-Purpose Dataset for Human-Robot Object Hand-Over |
| title_sort | oho a multi modal multi purpose dataset for human robot object hand over |
| topic | dataset thermal image semantic segmentation hand-over 6D pose estimation automated labeling |
| url | https://www.mdpi.com/1424-8220/23/18/7807 |
| work_keys_str_mv | AT benedictstephan ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover AT monakohler ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover AT steffenmuller ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover AT yanzhang ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover AT horstmichaelgross ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover AT gunthernotni ohoamultimodalmultipurposedatasetforhumanrobotobjecthandover |