Evaluation of standard and semantically-augmented distance metrics for neurology patients
Abstract Background Patient distances can be calculated based on signs and symptoms derived from an ontological hierarchy. There is controversy as to whether patient distance metrics that consider the semantic similarity between concepts can outperform standard patient distance metrics that are agno...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2020-08-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12911-020-01217-8 |
| _version_ | 1818335160423153664 |
|---|---|
| author | Daniel B. Hier Jonathan Kopel Steven U. Brint Donald C. Wunsch Gayla R. Olbricht Sima Azizi Blaine Allen |
| author_facet | Daniel B. Hier Jonathan Kopel Steven U. Brint Donald C. Wunsch Gayla R. Olbricht Sima Azizi Blaine Allen |
| author_sort | Daniel B. Hier |
| collection | DOAJ |
| description | Abstract Background Patient distances can be calculated based on signs and symptoms derived from an ontological hierarchy. There is controversy as to whether patient distance metrics that consider the semantic similarity between concepts can outperform standard patient distance metrics that are agnostic to concept similarity. The choice of distance metric can dominate the performance of classification or clustering algorithms. Our objective was to determine if semantically augmented distance metrics would outperform standard metrics on machine learning tasks. Methods We converted the neurological findings from 382 published neurology cases into sets of concepts with corresponding machine-readable codes. We calculated patient distances by four different metrics (cosine distance, a semantically augmented cosine distance, Jaccard distance, and a semantically augmented bipartite distance). Semantic augmentation for two of the metrics depended on concept similarities from a hierarchical neuro-ontology. For machine learning algorithms, we used the patient diagnosis as the ground truth label and patient findings as machine learning features. We assessed classification accuracy for four classifiers and cluster quality for two clustering algorithms for each of the distance metrics. Results Inter-patient distances were smaller when the distance metric was semantically augmented. Classification accuracy and cluster quality were not significantly different by distance metric. Conclusion Although semantic augmentation reduced inter-patient distances, we did not find improved classification accuracy or improved cluster quality with semantically augmented patient distance metrics when applied to a dataset of neurology patients. Further work is needed to assess the utility of semantically augmented patient distances. |
| first_indexed | 2024-12-13T14:19:01Z |
| format | Article |
| id | doaj.art-5f4b14967e54409992ae66e3d1dbba4b |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-12-13T14:19:01Z |
| publishDate | 2020-08-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-5f4b14967e54409992ae66e3d1dbba4b2022-12-21T23:42:09ZengBMCBMC Medical Informatics and Decision Making1472-69472020-08-0120111510.1186/s12911-020-01217-8Evaluation of standard and semantically-augmented distance metrics for neurology patientsDaniel B. Hier0Jonathan Kopel1Steven U. Brint2Donald C. Wunsch3Gayla R. Olbricht4Sima Azizi5Blaine Allen6Department of Neurology and Rehabilitation, University of Illinois at ChicagoDepartment of Internal Medicine, Texas Tech University Health Sciences CenterDepartment of Neurology and Rehabilitation, University of Illinois at ChicagoDepartment of Electrical and Computer Engineering, Missouri University of Science and TechnologyDepartment of Mathematics and Statistics, Missouri University of Science and TechnologyDepartment of Electrical and Computer Engineering, Missouri University of Science and TechnologyDepartment of Electrical and Computer Engineering, Missouri University of Science and TechnologyAbstract Background Patient distances can be calculated based on signs and symptoms derived from an ontological hierarchy. There is controversy as to whether patient distance metrics that consider the semantic similarity between concepts can outperform standard patient distance metrics that are agnostic to concept similarity. The choice of distance metric can dominate the performance of classification or clustering algorithms. Our objective was to determine if semantically augmented distance metrics would outperform standard metrics on machine learning tasks. Methods We converted the neurological findings from 382 published neurology cases into sets of concepts with corresponding machine-readable codes. We calculated patient distances by four different metrics (cosine distance, a semantically augmented cosine distance, Jaccard distance, and a semantically augmented bipartite distance). Semantic augmentation for two of the metrics depended on concept similarities from a hierarchical neuro-ontology. For machine learning algorithms, we used the patient diagnosis as the ground truth label and patient findings as machine learning features. We assessed classification accuracy for four classifiers and cluster quality for two clustering algorithms for each of the distance metrics. Results Inter-patient distances were smaller when the distance metric was semantically augmented. Classification accuracy and cluster quality were not significantly different by distance metric. Conclusion Although semantic augmentation reduced inter-patient distances, we did not find improved classification accuracy or improved cluster quality with semantically augmented patient distance metrics when applied to a dataset of neurology patients. Further work is needed to assess the utility of semantically augmented patient distances.http://link.springer.com/article/10.1186/s12911-020-01217-8Patient distancesSemantic augmentationOntologiesMachine learningPatient clusteringPatient classification |
| spellingShingle | Daniel B. Hier Jonathan Kopel Steven U. Brint Donald C. Wunsch Gayla R. Olbricht Sima Azizi Blaine Allen Evaluation of standard and semantically-augmented distance metrics for neurology patients BMC Medical Informatics and Decision Making Patient distances Semantic augmentation Ontologies Machine learning Patient clustering Patient classification |
| title | Evaluation of standard and semantically-augmented distance metrics for neurology patients |
| title_full | Evaluation of standard and semantically-augmented distance metrics for neurology patients |
| title_fullStr | Evaluation of standard and semantically-augmented distance metrics for neurology patients |
| title_full_unstemmed | Evaluation of standard and semantically-augmented distance metrics for neurology patients |
| title_short | Evaluation of standard and semantically-augmented distance metrics for neurology patients |
| title_sort | evaluation of standard and semantically augmented distance metrics for neurology patients |
| topic | Patient distances Semantic augmentation Ontologies Machine learning Patient clustering Patient classification |
| url | http://link.springer.com/article/10.1186/s12911-020-01217-8 |
| work_keys_str_mv | AT danielbhier evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT jonathankopel evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT stevenubrint evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT donaldcwunsch evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT gaylarolbricht evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT simaazizi evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients AT blaineallen evaluationofstandardandsemanticallyaugmenteddistancemetricsforneurologypatients |