Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera
We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale an...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/9/4416 |
| _version_ | 1797505711294382080 |
|---|---|
| author | Kaiwen Guo Tianqu Zhai Manoj H. Purushothama Alexander Dobre Shawn Meah Elton Pashollari Aabhaas Vaish Carl DeWilde Mohammed N. Islam |
| author_facet | Kaiwen Guo Tianqu Zhai Manoj H. Purushothama Alexander Dobre Shawn Meah Elton Pashollari Aabhaas Vaish Carl DeWilde Mohammed N. Islam |
| author_sort | Kaiwen Guo |
| collection | DOAJ |
| description | We demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS. |
| first_indexed | 2024-03-10T04:22:17Z |
| format | Article |
| id | doaj.art-cc1ec93ddc10432c90db12248bf732c1 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T04:22:17Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-cc1ec93ddc10432c90db12248bf732c12023-11-23T07:48:42ZengMDPI AGApplied Sciences2076-34172022-04-01129441610.3390/app12094416Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight CameraKaiwen Guo0Tianqu Zhai1Manoj H. Purushothama2Alexander Dobre3Shawn Meah4Elton Pashollari5Aabhaas Vaish6Carl DeWilde7Mohammed N. Islam8Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USAOmni Science Inc., Dexter, MI 48130, USADepartment of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USAWe demonstrate a Contactless Vital Sign Monitoring (CVSM) system and road-test the system for in-cabin driver monitoring using a near-infrared indirect Time-of-Flight (ToF) camera. The CVSM measures both heart rate (HR) and respiration rate (RR) by leveraging the simultaneously measured grayscale and depth information from a ToF camera. For a camera-based driver monitoring system (DMS), key challenges from varying background illumination and motion-induced artifacts need to be addressed. In this study, active illumination and depth-based motion compensation are used to mitigate these two challenges. For HR measurements, active illumination allows the system to work under various lighting conditions, while our depth-based motion compensation has the advantage of directly measuring the motion of the driver without making prior assumptions about the motion artifacts. In addition, we can extract RR directly from the chest wall motion, circumventing the challenge of acquiring RR from the near-infrared photoplethysmography (PPG) signal of low signal quality. We investigate the system’s performance in various scenarios, including monitoring both drivers and passengers while driving on highways and local roads. Our results show that our CVSM system is ambient light agnostic, and the success rates of HR measurements on the highway are 82% and 71.9% for the passenger and driver, respectively. At the same time, we show that the system can measure RR on users driving on a highway with a mean deviation of −1.4 breaths per minute (BPM). With reliable HR and RR measurement in the vehicle, the CVSM system could one day be a key enabler to sudden sickness or drowsiness detection in DMS.https://www.mdpi.com/2076-3417/12/9/4416Time-of-Flightcontactless physiological measurementheart rate monitoringmotion artifacts compensationdriver monitoring system |
| spellingShingle | Kaiwen Guo Tianqu Zhai Manoj H. Purushothama Alexander Dobre Shawn Meah Elton Pashollari Aabhaas Vaish Carl DeWilde Mohammed N. Islam Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera Applied Sciences Time-of-Flight contactless physiological measurement heart rate monitoring motion artifacts compensation driver monitoring system |
| title | Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera |
| title_full | Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera |
| title_fullStr | Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera |
| title_full_unstemmed | Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera |
| title_short | Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera |
| title_sort | contactless vital sign monitoring system for in vehicle driver monitoring using a near infrared time of flight camera |
| topic | Time-of-Flight contactless physiological measurement heart rate monitoring motion artifacts compensation driver monitoring system |
| url | https://www.mdpi.com/2076-3417/12/9/4416 |
| work_keys_str_mv | AT kaiwenguo contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT tianquzhai contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT manojhpurushothama contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT alexanderdobre contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT shawnmeah contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT eltonpashollari contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT aabhaasvaish contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT carldewilde contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera AT mohammednislam contactlessvitalsignmonitoringsystemforinvehicledrivermonitoringusinganearinfraredtimeofflightcamera |