Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study
BackgroundElectronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigat...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2020-04-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2020/4/e15876 |
| _version_ | 1819241621676359680 |
|---|---|
| author | King, Andrew J Cooper, Gregory F Clermont, Gilles Hochheiser, Harry Hauskrecht, Milos Sittig, Dean F Visweswaran, Shyam |
| author_facet | King, Andrew J Cooper, Gregory F Clermont, Gilles Hochheiser, Harry Hauskrecht, Milos Sittig, Dean F Visweswaran, Shyam |
| author_sort | King, Andrew J |
| collection | DOAJ |
| description | BackgroundElectronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigate cognitive overload, a Learning EMR (LEMR) system prioritizes the display of relevant medical record data. Relevant data are those that are pertinent to a context—defined as the combination of the user, clinical task, and patient case. To determine which data are relevant in a specific context, a LEMR system uses supervised machine learning models of physician information-seeking behavior. Since obtaining information-seeking behavior data via manual annotation is slow and expensive, automatic methods for capturing such data are needed.
ObjectiveThe goal of the research was to propose and evaluate eye tracking as a high-throughput method to automatically acquire physician information-seeking behavior useful for training models for a LEMR system.
MethodsCritical care medicine physicians reviewed intensive care unit patient cases in an EMR interface developed for the study. Participants manually identified patient data that were relevant in the context of a clinical task: preparing a patient summary to present at morning rounds. We used eye tracking to capture each physician’s gaze dwell time on each data item (eg, blood glucose measurements). Manual annotations and gaze dwell times were used to define target variables for developing supervised machine learning models of physician information-seeking behavior. We compared the performance of manual selection and gaze-derived models on an independent set of patient cases.
ResultsA total of 68 pairs of manual selection and gaze-derived machine learning models were developed from training data and evaluated on an independent evaluation data set. A paired Wilcoxon signed-rank test showed similar performance of manual selection and gaze-derived models on area under the receiver operating characteristic curve (P=.40).
ConclusionsWe used eye tracking to automatically capture physician information-seeking behavior and used it to train models for a LEMR system. The models that were trained using eye tracking performed like models that were trained using manual annotations. These results support further development of eye tracking as a high-throughput method for training clinical decision support systems that prioritize the display of relevant medical record data. |
| first_indexed | 2024-12-23T14:26:49Z |
| format | Article |
| id | doaj.art-35e47b39991e49f09aa8d55680b03bb7 |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-12-23T14:26:49Z |
| publishDate | 2020-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-35e47b39991e49f09aa8d55680b03bb72022-12-21T17:43:39ZengJMIR PublicationsJournal of Medical Internet Research1438-88712020-04-01224e1587610.2196/15876Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning StudyKing, Andrew JCooper, Gregory FClermont, GillesHochheiser, HarryHauskrecht, MilosSittig, Dean FVisweswaran, ShyamBackgroundElectronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigate cognitive overload, a Learning EMR (LEMR) system prioritizes the display of relevant medical record data. Relevant data are those that are pertinent to a context—defined as the combination of the user, clinical task, and patient case. To determine which data are relevant in a specific context, a LEMR system uses supervised machine learning models of physician information-seeking behavior. Since obtaining information-seeking behavior data via manual annotation is slow and expensive, automatic methods for capturing such data are needed. ObjectiveThe goal of the research was to propose and evaluate eye tracking as a high-throughput method to automatically acquire physician information-seeking behavior useful for training models for a LEMR system. MethodsCritical care medicine physicians reviewed intensive care unit patient cases in an EMR interface developed for the study. Participants manually identified patient data that were relevant in the context of a clinical task: preparing a patient summary to present at morning rounds. We used eye tracking to capture each physician’s gaze dwell time on each data item (eg, blood glucose measurements). Manual annotations and gaze dwell times were used to define target variables for developing supervised machine learning models of physician information-seeking behavior. We compared the performance of manual selection and gaze-derived models on an independent set of patient cases. ResultsA total of 68 pairs of manual selection and gaze-derived machine learning models were developed from training data and evaluated on an independent evaluation data set. A paired Wilcoxon signed-rank test showed similar performance of manual selection and gaze-derived models on area under the receiver operating characteristic curve (P=.40). ConclusionsWe used eye tracking to automatically capture physician information-seeking behavior and used it to train models for a LEMR system. The models that were trained using eye tracking performed like models that were trained using manual annotations. These results support further development of eye tracking as a high-throughput method for training clinical decision support systems that prioritize the display of relevant medical record data.https://www.jmir.org/2020/4/e15876 |
| spellingShingle | King, Andrew J Cooper, Gregory F Clermont, Gilles Hochheiser, Harry Hauskrecht, Milos Sittig, Dean F Visweswaran, Shyam Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study Journal of Medical Internet Research |
| title | Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study |
| title_full | Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study |
| title_fullStr | Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study |
| title_full_unstemmed | Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study |
| title_short | Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study |
| title_sort | leveraging eye tracking to prioritize relevant medical record data comparative machine learning study |
| url | https://www.jmir.org/2020/4/e15876 |
| work_keys_str_mv | AT kingandrewj leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT coopergregoryf leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT clermontgilles leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT hochheiserharry leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT hauskrechtmilos leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT sittigdeanf leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy AT visweswaranshyam leveragingeyetrackingtoprioritizerelevantmedicalrecorddatacomparativemachinelearningstudy |