A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots
Object detection and classification have countless applications in human–robot interacting systems. It is a necessary skill for autonomous robots that perform tasks in household scenarios. Despite the great advances in deep learning and computer vision, social robots performing non-trivial tasks usu...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2017-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/17/2/353 |
| _version_ | 1798005657835667456 |
|---|---|
| author | Marco A. Gutiérrez Luis J. Manso Harit Pandya Pedro Núñez |
| author_facet | Marco A. Gutiérrez Luis J. Manso Harit Pandya Pedro Núñez |
| author_sort | Marco A. Gutiérrez |
| collection | DOAJ |
| description | Object detection and classification have countless applications in human–robot interacting systems. It is a necessary skill for autonomous robots that perform tasks in household scenarios. Despite the great advances in deep learning and computer vision, social robots performing non-trivial tasks usually spend most of their time finding and modeling objects. Working in real scenarios means dealing with constant environment changes and relatively low-quality sensor data due to the distance at which objects are often found. Ambient intelligence systems equipped with different sensors can also benefit from the ability to find objects, enabling them to inform humans about their location. For these applications to succeed, systems need to detect the objects that may potentially contain other objects, working with relatively low-resolution sensor data. A passive learning architecture for sensors has been designed in order to take advantage of multimodal information, obtained using an RGB-D camera and trained semantic language models. The main contribution of the architecture lies in the improvement of the performance of the sensor under conditions of low resolution and high light variations using a combination of image labeling and word semantics. The tests performed on each of the stages of the architecture compare this solution with current research labeling techniques for the application of an autonomous social robot working in an apartment. The results obtained demonstrate that the proposed sensor architecture outperforms state-of-the-art approaches. |
| first_indexed | 2024-04-11T12:42:53Z |
| format | Article |
| id | doaj.art-2b119483e70c47c8b80482f18044010f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T12:42:53Z |
| publishDate | 2017-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-2b119483e70c47c8b80482f18044010f2022-12-22T04:23:26ZengMDPI AGSensors1424-82202017-02-0117235310.3390/s17020353s17020353A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social RobotsMarco A. Gutiérrez0Luis J. Manso1Harit Pandya2Pedro Núñez3Robotics and Artificial Vision Laboratory, University of Extremadura, 10003 Cáceres, SpainRobotics and Artificial Vision Laboratory, University of Extremadura, 10003 Cáceres, SpainRobotics Research Center, IIIT Hyderabad, 500032 Hyderabad, IndiaRobotics and Artificial Vision Laboratory, University of Extremadura, 10003 Cáceres, SpainObject detection and classification have countless applications in human–robot interacting systems. It is a necessary skill for autonomous robots that perform tasks in household scenarios. Despite the great advances in deep learning and computer vision, social robots performing non-trivial tasks usually spend most of their time finding and modeling objects. Working in real scenarios means dealing with constant environment changes and relatively low-quality sensor data due to the distance at which objects are often found. Ambient intelligence systems equipped with different sensors can also benefit from the ability to find objects, enabling them to inform humans about their location. For these applications to succeed, systems need to detect the objects that may potentially contain other objects, working with relatively low-resolution sensor data. A passive learning architecture for sensors has been designed in order to take advantage of multimodal information, obtained using an RGB-D camera and trained semantic language models. The main contribution of the architecture lies in the improvement of the performance of the sensor under conditions of low resolution and high light variations using a combination of image labeling and word semantics. The tests performed on each of the stages of the architecture compare this solution with current research labeling techniques for the application of an autonomous social robot working in an apartment. The results obtained demonstrate that the proposed sensor architecture outperforms state-of-the-art approaches.http://www.mdpi.com/1424-8220/17/2/353robot sensorsambient intelligence sensorsdeep learningobject detectionobject recognitionword semantics |
| spellingShingle | Marco A. Gutiérrez Luis J. Manso Harit Pandya Pedro Núñez A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots Sensors robot sensors ambient intelligence sensors deep learning object detection object recognition word semantics |
| title | A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots |
| title_full | A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots |
| title_fullStr | A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots |
| title_full_unstemmed | A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots |
| title_short | A Passive Learning Sensor Architecture for Multimodal Image Labeling: An Application for Social Robots |
| title_sort | passive learning sensor architecture for multimodal image labeling an application for social robots |
| topic | robot sensors ambient intelligence sensors deep learning object detection object recognition word semantics |
| url | http://www.mdpi.com/1424-8220/17/2/353 |
| work_keys_str_mv | AT marcoagutierrez apassivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT luisjmanso apassivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT haritpandya apassivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT pedronunez apassivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT marcoagutierrez passivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT luisjmanso passivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT haritpandya passivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots AT pedronunez passivelearningsensorarchitectureformultimodalimagelabelinganapplicationforsocialrobots |