Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal
Background and purpose — Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in general. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competen...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Medical Journals Sweden
2021-10-01
|
| Series: | Acta Orthopaedica |
| Online Access: | http://dx.doi.org/10.1080/17453674.2021.1918389 |
| _version_ | 1819280986690551808 |
|---|---|
| author | Jakub Olczak John Pavlopoulos Jasper Prijs Frank F A Ijpma Job N Doornberg Claes Lundström Joel Hedlund Max Gordon |
| author_facet | Jakub Olczak John Pavlopoulos Jasper Prijs Frank F A Ijpma Job N Doornberg Claes Lundström Joel Hedlund Max Gordon |
| author_sort | Jakub Olczak |
| collection | DOAJ |
| description | Background and purpose — Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in general. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competencies and goals creates challenges for collaboration and knowledge exchange. We aim to provide clinicians with a context and understanding of AI research by facilitating communication between creators, researchers, clinicians, and readers of medical AI and ML research. Methods and results — We present the common tasks, considerations, and pitfalls (both methodological and ethical) that clinicians will encounter in AI research. We discuss the following topics: labeling, missing data, training, testing, and overfitting. Common performance and outcome measures for various AI and ML tasks are presented, including accuracy, precision, recall, F1 score, Dice score, the area under the curve, and ROC curves. We also discuss ethical considerations in terms of privacy, fairness, autonomy, safety, responsibility, and liability regarding data collecting or sharing. Interpretation — We have developed guidelines for reporting medical AI research to clinicians in the run-up to a broader consensus process. The proposed guidelines consist of a Clinical Artificial Intelligence Research (CAIR) checklist and specific performance metrics guidelines to present and evaluate research using AI components. Researchers, engineers, clinicians, and other stakeholders can use these proposal guidelines and the CAIR checklist to read, present, and evaluate AI research geared towards a healthcare setting. |
| first_indexed | 2024-12-24T00:52:31Z |
| format | Article |
| id | doaj.art-d5f89633677b492791e7c073220f9c52 |
| institution | Directory Open Access Journal |
| issn | 1745-3674 1745-3682 |
| language | English |
| last_indexed | 2024-12-24T00:52:31Z |
| publishDate | 2021-10-01 |
| publisher | Medical Journals Sweden |
| record_format | Article |
| series | Acta Orthopaedica |
| spelling | doaj.art-d5f89633677b492791e7c073220f9c522022-12-21T17:23:33ZengMedical Journals SwedenActa Orthopaedica1745-36741745-36822021-10-0192551352510.1080/17453674.2021.19183891918389Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposalJakub Olczak0John Pavlopoulos1Jasper Prijs2Frank F A Ijpma3Job N Doornberg4Claes Lundström5Joel Hedlund6Max Gordon7Institute of Clinical Sciences, Danderyd University Hospital, Karolinska InstituteDepartment of Computer and System Sciences, Stockholm UniversityFlinders UniversityDepartment of Trauma Surgery, University Medical Center Groningen, University of GroningenFlinders UniversityCenter for Medical Image Science and Visualization, Linköping UniversityCenter for Medical Image Science and Visualization, Linköping UniversityInstitute of Clinical Sciences, Danderyd University Hospital, Karolinska InstituteBackground and purpose — Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in general. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competencies and goals creates challenges for collaboration and knowledge exchange. We aim to provide clinicians with a context and understanding of AI research by facilitating communication between creators, researchers, clinicians, and readers of medical AI and ML research. Methods and results — We present the common tasks, considerations, and pitfalls (both methodological and ethical) that clinicians will encounter in AI research. We discuss the following topics: labeling, missing data, training, testing, and overfitting. Common performance and outcome measures for various AI and ML tasks are presented, including accuracy, precision, recall, F1 score, Dice score, the area under the curve, and ROC curves. We also discuss ethical considerations in terms of privacy, fairness, autonomy, safety, responsibility, and liability regarding data collecting or sharing. Interpretation — We have developed guidelines for reporting medical AI research to clinicians in the run-up to a broader consensus process. The proposed guidelines consist of a Clinical Artificial Intelligence Research (CAIR) checklist and specific performance metrics guidelines to present and evaluate research using AI components. Researchers, engineers, clinicians, and other stakeholders can use these proposal guidelines and the CAIR checklist to read, present, and evaluate AI research geared towards a healthcare setting.http://dx.doi.org/10.1080/17453674.2021.1918389 |
| spellingShingle | Jakub Olczak John Pavlopoulos Jasper Prijs Frank F A Ijpma Job N Doornberg Claes Lundström Joel Hedlund Max Gordon Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal Acta Orthopaedica |
| title | Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal |
| title_full | Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal |
| title_fullStr | Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal |
| title_full_unstemmed | Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal |
| title_short | Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal |
| title_sort | presenting artificial intelligence deep learning and machine learning studies to clinicians and healthcare stakeholders an introductory reference with a guideline and a clinical ai research cair checklist proposal |
| url | http://dx.doi.org/10.1080/17453674.2021.1918389 |
| work_keys_str_mv | AT jakubolczak presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT johnpavlopoulos presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT jasperprijs presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT frankfaijpma presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT jobndoornberg presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT claeslundstrom presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT joelhedlund presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal AT maxgordon presentingartificialintelligencedeeplearningandmachinelearningstudiestocliniciansandhealthcarestakeholdersanintroductoryreferencewithaguidelineandaclinicalairesearchcairchecklistproposal |