ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing
The AI-in-the-loop system (AIS) has been widely used in various autonomous decision and control systems, such as computing vision, autonomous vehicle, and collision avoidance systems. AIS generates and updates control strategies through learning algorithms, which make the control behaviors non-deter...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/2/518 |
| _version_ | 1818899521512407040 |
|---|---|
| author | Tingting Wu Yunwei Dong Yu Zhang Aziz Singa |
| author_facet | Tingting Wu Yunwei Dong Yu Zhang Aziz Singa |
| author_sort | Tingting Wu |
| collection | DOAJ |
| description | The AI-in-the-loop system (AIS) has been widely used in various autonomous decision and control systems, such as computing vision, autonomous vehicle, and collision avoidance systems. AIS generates and updates control strategies through learning algorithms, which make the control behaviors non-deterministic and bring about the test oracle problem in AIS testing procedure. The traditional system mainly concerns about properties of safety, reliability, and real-time, while AIS concerns more about the correctness, robustness, and stiffness of system. To perform an AIS testing with the existing testing techniques according to the testing requirements, this paper presents an extendable framework of AI-in-the-loop system testing by exploring the key steps involved in the testing procedure, named ExtendAIST, which contributes to define the execution steps of ExtendAIST and design space of testing techniques. Furthermore, the ExtendAIST framework provides three concerns for AIS testing, which include: (a) the extension points; (b) sub-extension points; and (c) existing techniques commonly present in each point. Therefore, testers can obtain the testing strategy using existing techniques directly for corresponding testing requirements or extend more techniques based on these extension points. |
| first_indexed | 2024-12-19T19:49:17Z |
| format | Article |
| id | doaj.art-ccdb3a45ad754709a5c78e6454c6f3c9 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-12-19T19:49:17Z |
| publishDate | 2020-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-ccdb3a45ad754709a5c78e6454c6f3c92022-12-21T20:08:03ZengMDPI AGApplied Sciences2076-34172020-01-0110251810.3390/app10020518app10020518ExtendAIST: Exploring the Space of AI-in-the-Loop System TestingTingting Wu0Yunwei Dong1Yu Zhang2Aziz Singa3School of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, ChinaSchool of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an 710072, ChinaThe AI-in-the-loop system (AIS) has been widely used in various autonomous decision and control systems, such as computing vision, autonomous vehicle, and collision avoidance systems. AIS generates and updates control strategies through learning algorithms, which make the control behaviors non-deterministic and bring about the test oracle problem in AIS testing procedure. The traditional system mainly concerns about properties of safety, reliability, and real-time, while AIS concerns more about the correctness, robustness, and stiffness of system. To perform an AIS testing with the existing testing techniques according to the testing requirements, this paper presents an extendable framework of AI-in-the-loop system testing by exploring the key steps involved in the testing procedure, named ExtendAIST, which contributes to define the execution steps of ExtendAIST and design space of testing techniques. Furthermore, the ExtendAIST framework provides three concerns for AIS testing, which include: (a) the extension points; (b) sub-extension points; and (c) existing techniques commonly present in each point. Therefore, testers can obtain the testing strategy using existing techniques directly for corresponding testing requirements or extend more techniques based on these extension points.https://www.mdpi.com/2076-3417/10/2/518ai-in-the-loop systemmachine learningais testingais testing strategy |
| spellingShingle | Tingting Wu Yunwei Dong Yu Zhang Aziz Singa ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing Applied Sciences ai-in-the-loop system machine learning ais testing ais testing strategy |
| title | ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing |
| title_full | ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing |
| title_fullStr | ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing |
| title_full_unstemmed | ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing |
| title_short | ExtendAIST: Exploring the Space of AI-in-the-Loop System Testing |
| title_sort | extendaist exploring the space of ai in the loop system testing |
| topic | ai-in-the-loop system machine learning ais testing ais testing strategy |
| url | https://www.mdpi.com/2076-3417/10/2/518 |
| work_keys_str_mv | AT tingtingwu extendaistexploringthespaceofaiintheloopsystemtesting AT yunweidong extendaistexploringthespaceofaiintheloopsystemtesting AT yuzhang extendaistexploringthespaceofaiintheloopsystemtesting AT azizsinga extendaistexploringthespaceofaiintheloopsystemtesting |