Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions
Quality-sensitive applications of machine learning (ML) require quality assurance (QA) by humans before the predictions of an ML model can be deployed. QA for ML (QA4ML) interfaces require users to view a large amount of data and perform many interactions to correct errors made by the ML model. An o...
| Main Authors: | , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Association for Computing Machinery
2024
|
| _version_ | 1797113111048617984 |
|---|---|
| author | Zhang, Y Tennekes, M De Jong, T Curier, L Coecke, B Chen, M |
| author_facet | Zhang, Y Tennekes, M De Jong, T Curier, L Coecke, B Chen, M |
| author_sort | Zhang, Y |
| collection | OXFORD |
| description | Quality-sensitive applications of machine learning (ML) require quality assurance (QA) by humans before the predictions of an ML model can be deployed. QA for ML (QA4ML) interfaces require users to view a large amount of data and perform many interactions to correct errors made by the ML model. An optimized user interface (UI) can significantly reduce interaction costs. While UI optimization can be informed by user studies evaluating design options, this approach is not scalable because there are typically numerous small variations that can affect the efficiency of a QA4ML interface. Hence, we propose using simulation to evaluate and aid the optimization of QA4ML interfaces. In particular, we focus on simulating the combined effects of human intelligence in initiating appropriate interaction commands and machine intelligence in providing algorithmic assistance for accelerating QA4ML processes. As QA4ML is usually labor-intensive, we use the simulated task completion time as the metric for UI optimization under different interface and algorithm setups. We demonstrate the usage of this UI design method in several QA4ML applications. |
| first_indexed | 2024-03-07T08:12:17Z |
| format | Journal article |
| id | oxford-uuid:2de0d150-64c6-49e8-9ff7-79647139a823 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-04-09T03:58:52Z |
| publishDate | 2024 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | oxford-uuid:2de0d150-64c6-49e8-9ff7-79647139a8232024-04-08T11:58:00ZSimulation-based optimization of user interfaces for quality-assuring machine learning model predictionsJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:2de0d150-64c6-49e8-9ff7-79647139a823EnglishSymplectic ElementsAssociation for Computing Machinery2024Zhang, YTennekes, MDe Jong, TCurier, LCoecke, BChen, MQuality-sensitive applications of machine learning (ML) require quality assurance (QA) by humans before the predictions of an ML model can be deployed. QA for ML (QA4ML) interfaces require users to view a large amount of data and perform many interactions to correct errors made by the ML model. An optimized user interface (UI) can significantly reduce interaction costs. While UI optimization can be informed by user studies evaluating design options, this approach is not scalable because there are typically numerous small variations that can affect the efficiency of a QA4ML interface. Hence, we propose using simulation to evaluate and aid the optimization of QA4ML interfaces. In particular, we focus on simulating the combined effects of human intelligence in initiating appropriate interaction commands and machine intelligence in providing algorithmic assistance for accelerating QA4ML processes. As QA4ML is usually labor-intensive, we use the simulated task completion time as the metric for UI optimization under different interface and algorithm setups. We demonstrate the usage of this UI design method in several QA4ML applications. |
| spellingShingle | Zhang, Y Tennekes, M De Jong, T Curier, L Coecke, B Chen, M Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title | Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title_full | Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title_fullStr | Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title_full_unstemmed | Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title_short | Simulation-based optimization of user interfaces for quality-assuring machine learning model predictions |
| title_sort | simulation based optimization of user interfaces for quality assuring machine learning model predictions |
| work_keys_str_mv | AT zhangy simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions AT tennekesm simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions AT dejongt simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions AT curierl simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions AT coeckeb simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions AT chenm simulationbasedoptimizationofuserinterfacesforqualityassuringmachinelearningmodelpredictions |