Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)
This work presents a multi-year study conducted at the University of Toledo, aimed at improving human–machine teaming (HMT) methods and technologies. With the advent of artificial intelligence (AI) in 21st-century machines, collaboration between humans and machines has become highly complicated for...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/23/8390 |
| _version_ | 1797546767727722496 |
|---|---|
| author | Praveen Damacharla Parashar Dhakal Jyothi Priyanka Bandreddi Ahmad Y. Javaid Jennie J. Gallimore Colin Elkin Vijay K. Devabhaktuni |
| author_facet | Praveen Damacharla Parashar Dhakal Jyothi Priyanka Bandreddi Ahmad Y. Javaid Jennie J. Gallimore Colin Elkin Vijay K. Devabhaktuni |
| author_sort | Praveen Damacharla |
| collection | DOAJ |
| description | This work presents a multi-year study conducted at the University of Toledo, aimed at improving human–machine teaming (HMT) methods and technologies. With the advent of artificial intelligence (AI) in 21st-century machines, collaboration between humans and machines has become highly complicated for real-time applications. The penetration of intelligent and synthetic assistants (IA/SA) in virtually every field has opened up a path to the area of HMT. When it comes to crucial tasks such as patient treatment/care, industrial production, and defense, the use of non-standardized HMT technologies may pose a risk to human lives and cost billions of taxpayer dollars. A thorough literature survey revealed that there are not many established standards or benchmarks for HMT. In this paper, we propose a method to design an HMT based on a generalized architecture. This design includes the development of an intelligent collaborative system and the human team. Followed by the identification of processes and metrics to test and validate the proposed model, we present a novel human-in-the-loop (HIL) simulation method. The effectiveness of this method is demonstrated using two controlled HMT scenarios: Emergency care provider (ECP) training and patient treatment by an experienced medic. Both scenarios include humans processing visual data and performing actions that represent real-world applications while responding to a Voice-Based Synthetic Assistant (VBSA) as a collaborator that keeps track of actions. The impact of various machines, humans, and HMT parameters is presented from the perspective of performance, rules, roles, and operational limitations. The proposed HIL method was found to assist in standardization studies in the pursuit of HMT benchmarking for critical applications. Finally, we present guidelines for designing and benchmarking HMTs based on the case studies’ results analysis. |
| first_indexed | 2024-03-10T14:34:48Z |
| format | Article |
| id | doaj.art-4ad75ce608844542aface862eb8ee605 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T14:34:48Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-4ad75ce608844542aface862eb8ee6052023-11-20T22:17:36ZengMDPI AGApplied Sciences2076-34172020-11-011023839010.3390/app10238390Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT)Praveen Damacharla0Parashar Dhakal1Jyothi Priyanka Bandreddi2Ahmad Y. Javaid3Jennie J. Gallimore4Colin Elkin5Vijay K. Devabhaktuni6KineticAI Inc., Crown Point, IN 46307, USAElectrical Engineering and Computer Science Department, The University of Toledo, Toledo, OH 43606, USAElectrical Engineering and Computer Science Department, The University of Toledo, Toledo, OH 43606, USAElectrical Engineering and Computer Science Department, The University of Toledo, Toledo, OH 43606, USAThe College of Technology, Architecture and Applied Engineering, Bowling Green State University, Bowling Green, OH 43403, USAECE Department, Purdue University Northwest, Hammond, IN 46323, USAECE Department, Purdue University Northwest, Hammond, IN 46323, USAThis work presents a multi-year study conducted at the University of Toledo, aimed at improving human–machine teaming (HMT) methods and technologies. With the advent of artificial intelligence (AI) in 21st-century machines, collaboration between humans and machines has become highly complicated for real-time applications. The penetration of intelligent and synthetic assistants (IA/SA) in virtually every field has opened up a path to the area of HMT. When it comes to crucial tasks such as patient treatment/care, industrial production, and defense, the use of non-standardized HMT technologies may pose a risk to human lives and cost billions of taxpayer dollars. A thorough literature survey revealed that there are not many established standards or benchmarks for HMT. In this paper, we propose a method to design an HMT based on a generalized architecture. This design includes the development of an intelligent collaborative system and the human team. Followed by the identification of processes and metrics to test and validate the proposed model, we present a novel human-in-the-loop (HIL) simulation method. The effectiveness of this method is demonstrated using two controlled HMT scenarios: Emergency care provider (ECP) training and patient treatment by an experienced medic. Both scenarios include humans processing visual data and performing actions that represent real-world applications while responding to a Voice-Based Synthetic Assistant (VBSA) as a collaborator that keeps track of actions. The impact of various machines, humans, and HMT parameters is presented from the perspective of performance, rules, roles, and operational limitations. The proposed HIL method was found to assist in standardization studies in the pursuit of HMT benchmarking for critical applications. Finally, we present guidelines for designing and benchmarking HMTs based on the case studies’ results analysis.https://www.mdpi.com/2076-3417/10/23/8390artificial agentshuman factorshuman–machine teamingmetricssynthetic agents |
| spellingShingle | Praveen Damacharla Parashar Dhakal Jyothi Priyanka Bandreddi Ahmad Y. Javaid Jennie J. Gallimore Colin Elkin Vijay K. Devabhaktuni Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) Applied Sciences artificial agents human factors human–machine teaming metrics synthetic agents |
| title | Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) |
| title_full | Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) |
| title_fullStr | Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) |
| title_full_unstemmed | Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) |
| title_short | Novel Human-in-the-Loop (HIL) Simulation Method to Study Synthetic Agents and Standardize Human–Machine Teams (HMT) |
| title_sort | novel human in the loop hil simulation method to study synthetic agents and standardize human machine teams hmt |
| topic | artificial agents human factors human–machine teaming metrics synthetic agents |
| url | https://www.mdpi.com/2076-3417/10/23/8390 |
| work_keys_str_mv | AT praveendamacharla novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT parashardhakal novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT jyothipriyankabandreddi novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT ahmadyjavaid novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT jenniejgallimore novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT colinelkin novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt AT vijaykdevabhaktuni novelhumanintheloophilsimulationmethodtostudysyntheticagentsandstandardizehumanmachineteamshmt |