Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots
Home service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurri...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/23/9482 |
| _version_ | 1797399589485019136 |
|---|---|
| author | Yangqing Ye Xiaolon Ma Xuanyi Zhou Guanjun Bao Weiwei Wan Shibo Cai |
| author_facet | Yangqing Ye Xiaolon Ma Xuanyi Zhou Guanjun Bao Weiwei Wan Shibo Cai |
| author_sort | Yangqing Ye |
| collection | DOAJ |
| description | Home service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurriness, presenting a significant challenge for object detection. In particular, daily life scenes contain small objects like fruits and tableware, which are often occluded, further complicating object recognition and positioning. A dynamic and real-time object detection algorithm is proposed for home service robots. This is composed of an image deblurring algorithm and an object detection algorithm. To improve the clarity of motion-blurred images, the DA-Multi-DCGAN algorithm is proposed. It comprises an embedded dynamic adjustment mechanism and a multimodal multiscale fusion structure based on robot motion and surrounding environmental information, enabling the deblurring processing of images that are captured under different motion states. Compared with DeblurGAN, DA-Multi-DCGAN had a 5.07 improvement in Peak Signal-to-Noise Ratio (PSNR) and a 0.022 improvement in Structural Similarity (SSIM). An AT-LI-YOLO method is proposed for small and occluded object detection. Based on depthwise separable convolution, this method highlights key areas and integrates salient features by embedding the attention module in the AT-Resblock to improve the sensitivity and detection precision of small objects and partially occluded objects. It also employs a lightweight network unit Lightblock to reduce the network’s parameters and computational complexity, which improves its computational efficiency. Compared with YOLOv3, the mean average precision (mAP) of AT-LI-YOLO increased by 3.19%, and the detection precision of small objects, such as apples and oranges and partially occluded objects, increased by 19.12% and 29.52%, respectively. Moreover, the model inference efficiency had a 7 ms reduction in processing time. Based on the typical home activities of older people and children, the dataset Grasp-17 was established for the training and testing of the proposed method. Using the TensorRT neural network inference engine of the developed service robot prototype, the proposed dynamic and real-time object detection algorithm required 29 ms, which meets the real-time requirement of smooth vision. |
| first_indexed | 2024-03-09T01:43:16Z |
| format | Article |
| id | doaj.art-a3cb9f5b2c00474989601d1c5d2e2607 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T01:43:16Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-a3cb9f5b2c00474989601d1c5d2e26072023-12-08T15:26:10ZengMDPI AGSensors1424-82202023-11-012323948210.3390/s23239482Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service RobotsYangqing Ye0Xiaolon Ma1Xuanyi Zhou2Guanjun Bao3Weiwei Wan4Shibo Cai5College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, ChinaCollege of Mechanical and Electrical Engineering, China Jiliang University, Hangzhou 310018, ChinaCollege of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, ChinaCollege of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, ChinaGraduate School of Engineering Science, Osaka University, Suita 562-0045, JapanCollege of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, ChinaHome service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurriness, presenting a significant challenge for object detection. In particular, daily life scenes contain small objects like fruits and tableware, which are often occluded, further complicating object recognition and positioning. A dynamic and real-time object detection algorithm is proposed for home service robots. This is composed of an image deblurring algorithm and an object detection algorithm. To improve the clarity of motion-blurred images, the DA-Multi-DCGAN algorithm is proposed. It comprises an embedded dynamic adjustment mechanism and a multimodal multiscale fusion structure based on robot motion and surrounding environmental information, enabling the deblurring processing of images that are captured under different motion states. Compared with DeblurGAN, DA-Multi-DCGAN had a 5.07 improvement in Peak Signal-to-Noise Ratio (PSNR) and a 0.022 improvement in Structural Similarity (SSIM). An AT-LI-YOLO method is proposed for small and occluded object detection. Based on depthwise separable convolution, this method highlights key areas and integrates salient features by embedding the attention module in the AT-Resblock to improve the sensitivity and detection precision of small objects and partially occluded objects. It also employs a lightweight network unit Lightblock to reduce the network’s parameters and computational complexity, which improves its computational efficiency. Compared with YOLOv3, the mean average precision (mAP) of AT-LI-YOLO increased by 3.19%, and the detection precision of small objects, such as apples and oranges and partially occluded objects, increased by 19.12% and 29.52%, respectively. Moreover, the model inference efficiency had a 7 ms reduction in processing time. Based on the typical home activities of older people and children, the dataset Grasp-17 was established for the training and testing of the proposed method. Using the TensorRT neural network inference engine of the developed service robot prototype, the proposed dynamic and real-time object detection algorithm required 29 ms, which meets the real-time requirement of smooth vision.https://www.mdpi.com/1424-8220/23/23/9482real-time object detectionindoor service robotsDA-Multi-DCGANAT-LI-YOLO |
| spellingShingle | Yangqing Ye Xiaolon Ma Xuanyi Zhou Guanjun Bao Weiwei Wan Shibo Cai Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots Sensors real-time object detection indoor service robots DA-Multi-DCGAN AT-LI-YOLO |
| title | Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots |
| title_full | Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots |
| title_fullStr | Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots |
| title_full_unstemmed | Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots |
| title_short | Dynamic and Real-Time Object Detection Based on Deep Learning for Home Service Robots |
| title_sort | dynamic and real time object detection based on deep learning for home service robots |
| topic | real-time object detection indoor service robots DA-Multi-DCGAN AT-LI-YOLO |
| url | https://www.mdpi.com/1424-8220/23/23/9482 |
| work_keys_str_mv | AT yangqingye dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots AT xiaolonma dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots AT xuanyizhou dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots AT guanjunbao dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots AT weiweiwan dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots AT shibocai dynamicandrealtimeobjectdetectionbasedondeeplearningforhomeservicerobots |