Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles
This paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders;...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Frontiers Media
2023
|
| _version_ | 1797111775111413760 |
|---|---|
| author | Salvini, P Reinmund, T Hardin, B Grieman, K Ten Holter, C Johnson, A Kunze, L Winfield, A Jirotka, M |
| author_facet | Salvini, P Reinmund, T Hardin, B Grieman, K Ten Holter, C Johnson, A Kunze, L Winfield, A Jirotka, M |
| author_sort | Salvini, P |
| collection | OXFORD |
| description | This paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders; (ii) potential implications for stakeholders from regulation such as the General Data Protection Regulation (GDPR); (iii) the different values that stakeholder groups bring to the “decision” question; (iv) how complex pre-existing webs of stakeholders and decision-making authorities may be disrupted or disempowered by the use of an automated system and the lack of evaluation of possible consequences; (v) how ADMS for non-technical users can lead to circumvention of the boundaries of intended system use. We illustrate these challenges through case studies in three domains: adult social care, aviation, and vehicle driver monitoring systems. The paper closes with recommendations for both practice and policy in the deployment of ADMS. |
| first_indexed | 2024-03-07T08:15:07Z |
| format | Journal article |
| id | oxford-uuid:1347f5b3-850f-4bd1-9317-1e9ed9ad19b7 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:15:07Z |
| publishDate | 2023 |
| publisher | Frontiers Media |
| record_format | dspace |
| spelling | oxford-uuid:1347f5b3-850f-4bd1-9317-1e9ed9ad19b72024-01-03T13:52:57ZHuman involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehiclesJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:1347f5b3-850f-4bd1-9317-1e9ed9ad19b7EnglishSymplectic ElementsFrontiers Media2023Salvini, PReinmund, THardin, BGrieman, KTen Holter, CJohnson, AKunze, LWinfield, AJirotka, MThis paper draws on three case studies to examine some of the challenges and tensions involved in the use of Autonomous Decision-Making Systems (ADMS). In particular, the paper highlights: (i) challenges around the shifting “locale” of the decision, and the associated consequences for stakeholders; (ii) potential implications for stakeholders from regulation such as the General Data Protection Regulation (GDPR); (iii) the different values that stakeholder groups bring to the “decision” question; (iv) how complex pre-existing webs of stakeholders and decision-making authorities may be disrupted or disempowered by the use of an automated system and the lack of evaluation of possible consequences; (v) how ADMS for non-technical users can lead to circumvention of the boundaries of intended system use. We illustrate these challenges through case studies in three domains: adult social care, aviation, and vehicle driver monitoring systems. The paper closes with recommendations for both practice and policy in the deployment of ADMS. |
| spellingShingle | Salvini, P Reinmund, T Hardin, B Grieman, K Ten Holter, C Johnson, A Kunze, L Winfield, A Jirotka, M Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title | Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title_full | Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title_fullStr | Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title_full_unstemmed | Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title_short | Human involvement in autonomous decision-making systems. Lessons learned from three case studies in aviation, social care and road vehicles |
| title_sort | human involvement in autonomous decision making systems lessons learned from three case studies in aviation social care and road vehicles |
| work_keys_str_mv | AT salvinip humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT reinmundt humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT hardinb humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT griemank humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT tenholterc humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT johnsona humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT kunzel humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT winfielda humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles AT jirotkam humaninvolvementinautonomousdecisionmakingsystemslessonslearnedfromthreecasestudiesinaviationsocialcareandroadvehicles |