Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan
Abstract The development of computer-aided detection (CAD) using artificial intelligence (AI) and machine learning (ML) is rapidly evolving. Submission of AI/ML-based CAD devices for regulatory approval requires information about clinical trial design and performance criteria, but the requirements v...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-10-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-21426-7 |
| _version_ | 1828194567708475392 |
|---|---|
| author | Mitsuru Yuba Kiyotaka Iwasaki |
| author_facet | Mitsuru Yuba Kiyotaka Iwasaki |
| author_sort | Mitsuru Yuba |
| collection | DOAJ |
| description | Abstract The development of computer-aided detection (CAD) using artificial intelligence (AI) and machine learning (ML) is rapidly evolving. Submission of AI/ML-based CAD devices for regulatory approval requires information about clinical trial design and performance criteria, but the requirements vary between countries. This study compares the requirements for AI/ML-based CAD devices approved by the US Food and Drug Administration (FDA) and the Pharmaceuticals and Medical Devices Agency (PMDA) in Japan. A list of 45 FDA-approved and 12 PMDA-approved AI/ML-based CAD devices was compiled. In the USA, devices classified as computer-aided simple triage were approved based on standalone software testing, whereas devices classified as computer-aided detection/diagnosis were approved based on reader study testing. In Japan, however, there was no clear distinction between evaluation methods according to the category. In the USA, a prospective randomized controlled trial was conducted for AI/ML-based CAD devices used for the detection of colorectal polyps, whereas in Japan, such devices were approved based on standalone software testing. This study indicated that the different viewpoints of AI/ML-based CAD in the two countries influenced the selection of different evaluation methods. This study’s findings may be useful for defining a unified global development and approval standard for AI/ML-based CAD. |
| first_indexed | 2024-04-12T09:28:05Z |
| format | Article |
| id | doaj.art-917ea2893c9546a8a3ad2bd5ba32641f |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-12T09:28:05Z |
| publishDate | 2022-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-917ea2893c9546a8a3ad2bd5ba32641f2022-12-22T03:38:26ZengNature PortfolioScientific Reports2045-23222022-10-0112111310.1038/s41598-022-21426-7Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and JapanMitsuru Yuba0Kiyotaka Iwasaki1Cooperative Major in Advanced Biomedical Sciences, Joint Graduate School of Tokyo Women’s Medical University and Waseda University, Waseda UniversityCooperative Major in Advanced Biomedical Sciences, Joint Graduate School of Tokyo Women’s Medical University and Waseda University, Waseda UniversityAbstract The development of computer-aided detection (CAD) using artificial intelligence (AI) and machine learning (ML) is rapidly evolving. Submission of AI/ML-based CAD devices for regulatory approval requires information about clinical trial design and performance criteria, but the requirements vary between countries. This study compares the requirements for AI/ML-based CAD devices approved by the US Food and Drug Administration (FDA) and the Pharmaceuticals and Medical Devices Agency (PMDA) in Japan. A list of 45 FDA-approved and 12 PMDA-approved AI/ML-based CAD devices was compiled. In the USA, devices classified as computer-aided simple triage were approved based on standalone software testing, whereas devices classified as computer-aided detection/diagnosis were approved based on reader study testing. In Japan, however, there was no clear distinction between evaluation methods according to the category. In the USA, a prospective randomized controlled trial was conducted for AI/ML-based CAD devices used for the detection of colorectal polyps, whereas in Japan, such devices were approved based on standalone software testing. This study indicated that the different viewpoints of AI/ML-based CAD in the two countries influenced the selection of different evaluation methods. This study’s findings may be useful for defining a unified global development and approval standard for AI/ML-based CAD.https://doi.org/10.1038/s41598-022-21426-7 |
| spellingShingle | Mitsuru Yuba Kiyotaka Iwasaki Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan Scientific Reports |
| title | Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan |
| title_full | Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan |
| title_fullStr | Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan |
| title_full_unstemmed | Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan |
| title_short | Systematic analysis of the test design and performance of AI/ML-based medical devices approved for triage/detection/diagnosis in the USA and Japan |
| title_sort | systematic analysis of the test design and performance of ai ml based medical devices approved for triage detection diagnosis in the usa and japan |
| url | https://doi.org/10.1038/s41598-022-21426-7 |
| work_keys_str_mv | AT mitsuruyuba systematicanalysisofthetestdesignandperformanceofaimlbasedmedicaldevicesapprovedfortriagedetectiondiagnosisintheusaandjapan AT kiyotakaiwasaki systematicanalysisofthetestdesignandperformanceofaimlbasedmedicaldevicesapprovedfortriagedetectiondiagnosisintheusaandjapan |