Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles
When humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions whil...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9885187/ |
| _version_ | 1828174739927990272 |
|---|---|
| author | Samuel Karlsson Bjorn Lindqvist George Nikolakopulos |
| author_facet | Samuel Karlsson Bjorn Lindqvist George Nikolakopulos |
| author_sort | Samuel Karlsson |
| collection | DOAJ |
| description | When humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions while operating in full autonomy mode becomes a critical gate-keeping technology for trust in robotic workers. Towards that problem, this article proposes a track-and-avoid architecture for legged robots that combines a visual object detection and estimation pipeline with a Nonlinear Model Predictive Controller (NMPC) based on the Optimization Engine, capable of generating trajectories that satisfy the avoidance and tracking problems in real-time operations where the computation time never exceeded 40 ms. The system is experimentally evaluated using the BD Spot, in a custom sensor and computational suite, and in fully autonomous operational conditions, for the robot-human safety scenario of quickly moving noncooperative obstacles. The results demonstrate the efficacy of the scheme in multiple scenarios where the maximum safety distance violation was only 9 cm for an obstacle moving at 2.5 m/s while affected by both state estimation and object detection uncertainty and noise. |
| first_indexed | 2024-04-12T04:16:39Z |
| format | Article |
| id | doaj.art-67d72991c65e4f399b791904e39ec022 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T04:16:39Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-67d72991c65e4f399b791904e39ec0222022-12-22T03:48:22ZengIEEEIEEE Access2169-35362022-01-011010022410023310.1109/ACCESS.2022.32056119885187Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity ObstaclesSamuel Karlsson0https://orcid.org/0000-0002-1046-0305Bjorn Lindqvist1https://orcid.org/0000-0003-3922-1735George Nikolakopulos2https://orcid.org/0000-0003-0126-1897Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology, Luleå, SwedenDepartment of Computer Science, Electrical and Space Engineering, Luleå University of Technology, Luleå, SwedenDepartment of Computer Science, Electrical and Space Engineering, Luleå University of Technology, Luleå, SwedenWhen humans and robots operate in and occupy the same local space, proximity detection and proactive collision avoidance is of high importance. As legged robots, such as the Boston Dynamics (BD) Spot, start to appear in real-world application environments, ensuring safe robot-human interactions while operating in full autonomy mode becomes a critical gate-keeping technology for trust in robotic workers. Towards that problem, this article proposes a track-and-avoid architecture for legged robots that combines a visual object detection and estimation pipeline with a Nonlinear Model Predictive Controller (NMPC) based on the Optimization Engine, capable of generating trajectories that satisfy the avoidance and tracking problems in real-time operations where the computation time never exceeded 40 ms. The system is experimentally evaluated using the BD Spot, in a custom sensor and computational suite, and in fully autonomous operational conditions, for the robot-human safety scenario of quickly moving noncooperative obstacles. The results demonstrate the efficacy of the scheme in multiple scenarios where the maximum safety distance violation was only 9 cm for an obstacle moving at 2.5 m/s while affected by both state estimation and object detection uncertainty and noise.https://ieeexplore.ieee.org/document/9885187/Human robot interactionNMPCobject detectionobject trackingspotvelocity obstacle |
| spellingShingle | Samuel Karlsson Bjorn Lindqvist George Nikolakopulos Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles IEEE Access Human robot interaction NMPC object detection object tracking spot velocity obstacle |
| title | Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles |
| title_full | Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles |
| title_fullStr | Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles |
| title_full_unstemmed | Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles |
| title_short | Ensuring Robot-Human Safety for the BD Spot Using Active Visual Tracking and NMPC With Velocity Obstacles |
| title_sort | ensuring robot human safety for the bd spot using active visual tracking and nmpc with velocity obstacles |
| topic | Human robot interaction NMPC object detection object tracking spot velocity obstacle |
| url | https://ieeexplore.ieee.org/document/9885187/ |
| work_keys_str_mv | AT samuelkarlsson ensuringrobothumansafetyforthebdspotusingactivevisualtrackingandnmpcwithvelocityobstacles AT bjornlindqvist ensuringrobothumansafetyforthebdspotusingactivevisualtrackingandnmpcwithvelocityobstacles AT georgenikolakopulos ensuringrobothumansafetyforthebdspotusingactivevisualtrackingandnmpcwithvelocityobstacles |