DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction
With the growth of sensing, control and robotic technologies, autonomous underwater vehicles (AUVs) have become useful assistants to human divers for performing various underwater operations. In the current practice, the divers are required to carry expensive, bulky, and waterproof keyboards or joys...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10190624/ |
| _version_ | 1797768445493772288 |
|---|---|
| author | Jing Yang James P. Wilson Shalabh Gupta |
| author_facet | Jing Yang James P. Wilson Shalabh Gupta |
| author_sort | Jing Yang |
| collection | DOAJ |
| description | With the growth of sensing, control and robotic technologies, autonomous underwater vehicles (AUVs) have become useful assistants to human divers for performing various underwater operations. In the current practice, the divers are required to carry expensive, bulky, and waterproof keyboards or joystick-based controllers for the supervision and control of AUVs. Therefore, diver action-based supervision is becoming increasingly popular because it is convenient, easier to use, faster, and cost effective. However, various environmental, diver, and sensing uncertainties make the underwater diver action recognition problem challenging. In this regard, this paper presents DARE, a diver action recognition encoder, which is robust to underwater uncertainties and classifies various diver actions including sixteen gestures and three poses with high accuracy. DARE is based on the fusion of stereo-pairs of underwater camera images using bi-channel convolutional layers for feature extraction followed by a systematically designed decision tree of neural network classifiers. DARE is trained using the Cognitive Autonomous Diving Buddy (CADDY) dataset, which consists of a rich set of images of different diver actions in real underwater environments. DARE requires only a few milliseconds to classify one stereo-pair, thus making it suitable for real-time implementation. The results show that DARE achieves up to 95.87% overall accuracy and 92% minimum class accuracy, thus verifying its robustnesss and reliability. Furthermore, a comparative evaluation against existing deep transfer learning architectures reveals that DARE improves the performance of baseline classifiers by up to 3.44% in the overall accuracy and 30% in the minimum class accuracy. |
| first_indexed | 2024-03-12T20:53:37Z |
| format | Article |
| id | doaj.art-e7b3c111dbfe4333b7bbf0ace72a5909 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T20:53:37Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e7b3c111dbfe4333b7bbf0ace72a59092023-07-31T23:00:50ZengIEEEIEEE Access2169-35362023-01-0111769267694010.1109/ACCESS.2023.329830410190624DARE: Diver Action Recognition Encoder for Underwater Human–Robot InteractionJing Yang0James P. Wilson1https://orcid.org/0000-0002-0563-8025Shalabh Gupta2https://orcid.org/0000-0003-0438-7700Department of Electrical and Computer Engineering, University of Connecticut, Storrs, CT, USADepartment of Electrical and Computer Engineering, University of Connecticut, Storrs, CT, USADepartment of Electrical and Computer Engineering, University of Connecticut, Storrs, CT, USAWith the growth of sensing, control and robotic technologies, autonomous underwater vehicles (AUVs) have become useful assistants to human divers for performing various underwater operations. In the current practice, the divers are required to carry expensive, bulky, and waterproof keyboards or joystick-based controllers for the supervision and control of AUVs. Therefore, diver action-based supervision is becoming increasingly popular because it is convenient, easier to use, faster, and cost effective. However, various environmental, diver, and sensing uncertainties make the underwater diver action recognition problem challenging. In this regard, this paper presents DARE, a diver action recognition encoder, which is robust to underwater uncertainties and classifies various diver actions including sixteen gestures and three poses with high accuracy. DARE is based on the fusion of stereo-pairs of underwater camera images using bi-channel convolutional layers for feature extraction followed by a systematically designed decision tree of neural network classifiers. DARE is trained using the Cognitive Autonomous Diving Buddy (CADDY) dataset, which consists of a rich set of images of different diver actions in real underwater environments. DARE requires only a few milliseconds to classify one stereo-pair, thus making it suitable for real-time implementation. The results show that DARE achieves up to 95.87% overall accuracy and 92% minimum class accuracy, thus verifying its robustnesss and reliability. Furthermore, a comparative evaluation against existing deep transfer learning architectures reveals that DARE improves the performance of baseline classifiers by up to 3.44% in the overall accuracy and 30% in the minimum class accuracy.https://ieeexplore.ieee.org/document/10190624/Autonomous underwater vehiclesdiver action recognitionhuman-robot interactionbi-channel convolutional neural networkstransfer learning |
| spellingShingle | Jing Yang James P. Wilson Shalabh Gupta DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction IEEE Access Autonomous underwater vehicles diver action recognition human-robot interaction bi-channel convolutional neural networks transfer learning |
| title | DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction |
| title_full | DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction |
| title_fullStr | DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction |
| title_full_unstemmed | DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction |
| title_short | DARE: Diver Action Recognition Encoder for Underwater Human–Robot Interaction |
| title_sort | dare diver action recognition encoder for underwater human x2013 robot interaction |
| topic | Autonomous underwater vehicles diver action recognition human-robot interaction bi-channel convolutional neural networks transfer learning |
| url | https://ieeexplore.ieee.org/document/10190624/ |
| work_keys_str_mv | AT jingyang darediveractionrecognitionencoderforunderwaterhumanx2013robotinteraction AT jamespwilson darediveractionrecognitionencoderforunderwaterhumanx2013robotinteraction AT shalabhgupta darediveractionrecognitionencoderforunderwaterhumanx2013robotinteraction |