Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video
Abstract Major vascular injury resulting in uncontrolled bleeding is a catastrophic and often fatal complication of minimally invasive surgery. At the outset of these events, surgeons do not know how much blood will be lost or whether they will successfully control the hemorrhage (achieve hemostasis...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-05-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-11549-2 |
| _version_ | 1811338892852330496 |
|---|---|
| author | Dhiraj J. Pangal Guillaume Kugener Yichao Zhu Aditya Sinha Vyom Unadkat David J. Cote Ben Strickland Martin Rutkowski Andrew Hung Animashree Anandkumar X. Y. Han Vardan Papyan Bozena Wrobel Gabriel Zada Daniel A. Donoho |
| author_facet | Dhiraj J. Pangal Guillaume Kugener Yichao Zhu Aditya Sinha Vyom Unadkat David J. Cote Ben Strickland Martin Rutkowski Andrew Hung Animashree Anandkumar X. Y. Han Vardan Papyan Bozena Wrobel Gabriel Zada Daniel A. Donoho |
| author_sort | Dhiraj J. Pangal |
| collection | DOAJ |
| description | Abstract Major vascular injury resulting in uncontrolled bleeding is a catastrophic and often fatal complication of minimally invasive surgery. At the outset of these events, surgeons do not know how much blood will be lost or whether they will successfully control the hemorrhage (achieve hemostasis). We evaluate the ability of a deep learning neural network (DNN) to predict hemostasis control ability using the first minute of surgical video and compare model performance with human experts viewing the same video. The publicly available SOCAL dataset contains 147 videos of attending and resident surgeons managing hemorrhage in a validated, high-fidelity cadaveric simulator. Videos are labeled with outcome and blood loss (mL). The first minute of 20 videos was shown to four, blinded, fellowship trained skull-base neurosurgery instructors, and to SOCALNet (a DNN trained on SOCAL videos). SOCALNet architecture included a convolutional network (ResNet) identifying spatial features and a recurrent network identifying temporal features (LSTM). Experts independently assessed surgeon skill, predicted outcome and blood loss (mL). Outcome and blood loss predictions were compared with SOCALNet. Expert inter-rater reliability was 0.95. Experts correctly predicted 14/20 trials (Sensitivity: 82%, Specificity: 55%, Positive Predictive Value (PPV): 69%, Negative Predictive Value (NPV): 71%). SOCALNet correctly predicted 17/20 trials (Sensitivity 100%, Specificity 66%, PPV 79%, NPV 100%) and correctly identified all successful attempts. Expert predictions of the highest and lowest skill surgeons and expert predictions reported with maximum confidence were more accurate. Experts systematically underestimated blood loss (mean error − 131 mL, RMSE 350 mL, R2 0.70) and fewer than half of expert predictions identified blood loss > 500 mL (47.5%, 19/40). SOCALNet had superior performance (mean error − 57 mL, RMSE 295 mL, R2 0.74) and detected most episodes of blood loss > 500 mL (80%, 8/10). In validation experiments, SOCALNet evaluation of a critical on-screen surgical maneuver and high/low-skill composite videos were concordant with expert evaluation. Using only the first minute of video, experts and SOCALNet can predict outcome and blood loss during surgical hemorrhage. Experts systematically underestimated blood loss, and SOCALNet had no false negatives. DNNs can provide accurate, meaningful assessments of surgical video. We call for the creation of datasets of surgical adverse events for quality improvement research. |
| first_indexed | 2024-04-13T18:17:21Z |
| format | Article |
| id | doaj.art-c964f61c59dd458fa1fe9545d78e34ea |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T18:17:21Z |
| publishDate | 2022-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-c964f61c59dd458fa1fe9545d78e34ea2022-12-22T02:35:37ZengNature PortfolioScientific Reports2045-23222022-05-0112111010.1038/s41598-022-11549-2Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of videoDhiraj J. Pangal0Guillaume Kugener1Yichao Zhu2Aditya Sinha3Vyom Unadkat4David J. Cote5Ben Strickland6Martin Rutkowski7Andrew Hung8Animashree Anandkumar9X. Y. Han10Vardan Papyan11Bozena Wrobel12Gabriel Zada13Daniel A. Donoho14Department of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaDepartment of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaViterbi School of Engineering, University of Southern CaliforniaDepartment of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaViterbi School of Engineering, University of Southern CaliforniaDepartment of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaDepartment of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaDepartment of Neurosurgery, Medical College of GeorgiaCenter for Robotic Simulation and Education, USC Institute of Urology, Keck School of Medicine of the University of Southern CaliforniaDepartment of Computer Science + Mathematics, California Institute of TechnologyDepartment of Operations Research and Information Engineering, Cornell UniversityDepartment of Mathematics, University of TorontoDepartment of Otolaryngology, Keck School of Medicine of the University of Southern CaliforniaDepartment of Neurosurgery, Keck School of Medicine of the University of Southern CaliforniaDivision of Neurosurgery, Center for Neuroscience, Children’s National HospitalAbstract Major vascular injury resulting in uncontrolled bleeding is a catastrophic and often fatal complication of minimally invasive surgery. At the outset of these events, surgeons do not know how much blood will be lost or whether they will successfully control the hemorrhage (achieve hemostasis). We evaluate the ability of a deep learning neural network (DNN) to predict hemostasis control ability using the first minute of surgical video and compare model performance with human experts viewing the same video. The publicly available SOCAL dataset contains 147 videos of attending and resident surgeons managing hemorrhage in a validated, high-fidelity cadaveric simulator. Videos are labeled with outcome and blood loss (mL). The first minute of 20 videos was shown to four, blinded, fellowship trained skull-base neurosurgery instructors, and to SOCALNet (a DNN trained on SOCAL videos). SOCALNet architecture included a convolutional network (ResNet) identifying spatial features and a recurrent network identifying temporal features (LSTM). Experts independently assessed surgeon skill, predicted outcome and blood loss (mL). Outcome and blood loss predictions were compared with SOCALNet. Expert inter-rater reliability was 0.95. Experts correctly predicted 14/20 trials (Sensitivity: 82%, Specificity: 55%, Positive Predictive Value (PPV): 69%, Negative Predictive Value (NPV): 71%). SOCALNet correctly predicted 17/20 trials (Sensitivity 100%, Specificity 66%, PPV 79%, NPV 100%) and correctly identified all successful attempts. Expert predictions of the highest and lowest skill surgeons and expert predictions reported with maximum confidence were more accurate. Experts systematically underestimated blood loss (mean error − 131 mL, RMSE 350 mL, R2 0.70) and fewer than half of expert predictions identified blood loss > 500 mL (47.5%, 19/40). SOCALNet had superior performance (mean error − 57 mL, RMSE 295 mL, R2 0.74) and detected most episodes of blood loss > 500 mL (80%, 8/10). In validation experiments, SOCALNet evaluation of a critical on-screen surgical maneuver and high/low-skill composite videos were concordant with expert evaluation. Using only the first minute of video, experts and SOCALNet can predict outcome and blood loss during surgical hemorrhage. Experts systematically underestimated blood loss, and SOCALNet had no false negatives. DNNs can provide accurate, meaningful assessments of surgical video. We call for the creation of datasets of surgical adverse events for quality improvement research.https://doi.org/10.1038/s41598-022-11549-2 |
| spellingShingle | Dhiraj J. Pangal Guillaume Kugener Yichao Zhu Aditya Sinha Vyom Unadkat David J. Cote Ben Strickland Martin Rutkowski Andrew Hung Animashree Anandkumar X. Y. Han Vardan Papyan Bozena Wrobel Gabriel Zada Daniel A. Donoho Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video Scientific Reports |
| title | Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| title_full | Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| title_fullStr | Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| title_full_unstemmed | Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| title_short | Expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| title_sort | expert surgeons and deep learning models can predict the outcome of surgical hemorrhage from 1 min of video |
| url | https://doi.org/10.1038/s41598-022-11549-2 |
| work_keys_str_mv | AT dhirajjpangal expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT guillaumekugener expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT yichaozhu expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT adityasinha expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT vyomunadkat expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT davidjcote expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT benstrickland expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT martinrutkowski expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT andrewhung expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT animashreeanandkumar expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT xyhan expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT vardanpapyan expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT bozenawrobel expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT gabrielzada expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo AT danieladonoho expertsurgeonsanddeeplearningmodelscanpredicttheoutcomeofsurgicalhemorrhagefrom1minofvideo |