Scene images and text information‐based object location of robot grasping
Abstract With the rapid development of artificial intelligence (AI), the application of this technology in the medical field is becoming increasingly extensive, along with a gradual increase in the amount of intelligent equipment in hospitals. Service robots can save human resources and replace nurs...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-06-01
|
| Series: | IET Cyber-systems and Robotics |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/csy2.12049 |
| _version_ | 1811344587303682048 |
|---|---|
| author | Zhichao Liu Kaixuan Ding Qingyang Xu Yong Song Xianfeng Yuan Yibin Li |
| author_facet | Zhichao Liu Kaixuan Ding Qingyang Xu Yong Song Xianfeng Yuan Yibin Li |
| author_sort | Zhichao Liu |
| collection | DOAJ |
| description | Abstract With the rapid development of artificial intelligence (AI), the application of this technology in the medical field is becoming increasingly extensive, along with a gradual increase in the amount of intelligent equipment in hospitals. Service robots can save human resources and replace nursing staff to achieve some work. In view of the phenomenon of mobile service robots' grabbing and distribution of patients' drugs in hospitals, a real‐time object detection and positioning system based on image and text information is proposed, which realizes the precise positioning and tracking of the grabbing objects and completes the grasping of a specific object (medicine bottle). The lightweight object detection model NanoDet is used to learn the features of the grasping objects and the object category, and bounding boxes are regressed. Then, the images in the bounding boxes are enhanced to overcome unfavourable factors, such as a small object region. The text detection and recognition model PP‐OCR is used to detect and recognise the enhanced images and extract the text information. The object information provided by the two models is fused, and the text recognition result is matched with the object detection box to achieve the precise positioning of the grasping object. The kernel correlation filter (KCF) tracking algorithm is introduced to achieve real‐time tracking of specific objects to precisely control the robot's grasping. Both deep learning models adopt lightweight networks to facilitate direct deployment. The experiments show that the proposed robot grasping detection system has high reliability, accuracy and real‐time performance. |
| first_indexed | 2024-04-13T19:49:30Z |
| format | Article |
| id | doaj.art-c47e4adde04a4e7494f6565873ca412c |
| institution | Directory Open Access Journal |
| issn | 2631-6315 |
| language | English |
| last_indexed | 2024-04-13T19:49:30Z |
| publishDate | 2022-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Cyber-systems and Robotics |
| spelling | doaj.art-c47e4adde04a4e7494f6565873ca412c2022-12-22T02:32:36ZengWileyIET Cyber-systems and Robotics2631-63152022-06-014211613010.1049/csy2.12049Scene images and text information‐based object location of robot graspingZhichao Liu0Kaixuan Ding1Qingyang Xu2Yong Song3Xianfeng Yuan4Yibin Li5School of Mechanical, Electrical & Information Engineering, Shandong University Weihai ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University Weihai ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University Weihai ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University Weihai ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University Weihai ChinaSchool of Control Science and Engineering Shandong University Jinan ChinaAbstract With the rapid development of artificial intelligence (AI), the application of this technology in the medical field is becoming increasingly extensive, along with a gradual increase in the amount of intelligent equipment in hospitals. Service robots can save human resources and replace nursing staff to achieve some work. In view of the phenomenon of mobile service robots' grabbing and distribution of patients' drugs in hospitals, a real‐time object detection and positioning system based on image and text information is proposed, which realizes the precise positioning and tracking of the grabbing objects and completes the grasping of a specific object (medicine bottle). The lightweight object detection model NanoDet is used to learn the features of the grasping objects and the object category, and bounding boxes are regressed. Then, the images in the bounding boxes are enhanced to overcome unfavourable factors, such as a small object region. The text detection and recognition model PP‐OCR is used to detect and recognise the enhanced images and extract the text information. The object information provided by the two models is fused, and the text recognition result is matched with the object detection box to achieve the precise positioning of the grasping object. The kernel correlation filter (KCF) tracking algorithm is introduced to achieve real‐time tracking of specific objects to precisely control the robot's grasping. Both deep learning models adopt lightweight networks to facilitate direct deployment. The experiments show that the proposed robot grasping detection system has high reliability, accuracy and real‐time performance.https://doi.org/10.1049/csy2.12049dexterous manipulatorsneural network‐basedrobot controlrobot visionservice robots |
| spellingShingle | Zhichao Liu Kaixuan Ding Qingyang Xu Yong Song Xianfeng Yuan Yibin Li Scene images and text information‐based object location of robot grasping IET Cyber-systems and Robotics dexterous manipulators neural network‐based robot control robot vision service robots |
| title | Scene images and text information‐based object location of robot grasping |
| title_full | Scene images and text information‐based object location of robot grasping |
| title_fullStr | Scene images and text information‐based object location of robot grasping |
| title_full_unstemmed | Scene images and text information‐based object location of robot grasping |
| title_short | Scene images and text information‐based object location of robot grasping |
| title_sort | scene images and text information based object location of robot grasping |
| topic | dexterous manipulators neural network‐based robot control robot vision service robots |
| url | https://doi.org/10.1049/csy2.12049 |
| work_keys_str_mv | AT zhichaoliu sceneimagesandtextinformationbasedobjectlocationofrobotgrasping AT kaixuanding sceneimagesandtextinformationbasedobjectlocationofrobotgrasping AT qingyangxu sceneimagesandtextinformationbasedobjectlocationofrobotgrasping AT yongsong sceneimagesandtextinformationbasedobjectlocationofrobotgrasping AT xianfengyuan sceneimagesandtextinformationbasedobjectlocationofrobotgrasping AT yibinli sceneimagesandtextinformationbasedobjectlocationofrobotgrasping |