IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm
Abstract Machine learning has been applied in recent years to categorize sleep stages (NREM, REM, and wake) using electroencephalogram (EEG) recordings; however, a well-validated sleep scoring automatic pipeline in rodent research is still not publicly available. Here, we present IntelliSleepScorer,...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-31288-2 |
| _version_ | 1797864898737209344 |
|---|---|
| author | Lei A. Wang Ryan Kern Eunah Yu Soonwook Choi Jen Q. Pan |
| author_facet | Lei A. Wang Ryan Kern Eunah Yu Soonwook Choi Jen Q. Pan |
| author_sort | Lei A. Wang |
| collection | DOAJ |
| description | Abstract Machine learning has been applied in recent years to categorize sleep stages (NREM, REM, and wake) using electroencephalogram (EEG) recordings; however, a well-validated sleep scoring automatic pipeline in rodent research is still not publicly available. Here, we present IntelliSleepScorer, a software package with a graphic user interface to score sleep stages automatically in mice. IntelliSleepScorer uses the light gradient boosting machine (LightGBM) to score sleep stages for each epoch of recordings. We developed LightGBM models using a large cohort of data, which consisted of 5776 h of sleep EEG and electromyogram (EMG) signals across 519 unique recordings from 124 mice. The LightGBM model achieved an overall accuracy of 95.2% and a Cohen’s kappa of 0.91, which outperforms the baseline models such as the logistic regression model (accuracy = 93.3%, kappa = 0.88) and the random forest model (accuracy = 94.3%, kappa = 0.89). The overall performance of the LightGBM model as well as the performance across different sleep stages are on par with that of the human experts. Most importantly, we validated the generalizability of the LightGBM models: (1) The LightGBM model performed well on two publicly available, independent datasets (kappa > = 0.80), which have different sampling frequency and epoch lengths; (2) The LightGBM model performed well on data recorded at a lower sampling frequency (kappa = 0.90); (3) The performance of the LightGBM model is not affected by the light/dark cycle; and (4) A modified LightGBM model performed well on data containing only one EEG and one EMG electrode (kappa > = 0.89). Taken together, the LightGBM models offer state-of-the-art performance for automatic sleep stage scoring in mice. Last, we implemented the IntelliSleepScorer software package based on the validated model to provide an out-of-box solution to sleep researchers (available for download at https://sites.broadinstitute.org/pan-lab/resources ). |
| first_indexed | 2024-04-09T23:00:34Z |
| format | Article |
| id | doaj.art-4d3a0f7ed4d04eb3956c3d2d09973d71 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-09T23:00:34Z |
| publishDate | 2023-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-4d3a0f7ed4d04eb3956c3d2d09973d712023-03-22T11:00:58ZengNature PortfolioScientific Reports2045-23222023-03-0113111110.1038/s41598-023-31288-2IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithmLei A. Wang0Ryan Kern1Eunah Yu2Soonwook Choi3Jen Q. Pan4Stanley Center for Psychiatric Research, Broad Institute of MIT and HarvardTroy High SchoolStanley Center for Psychiatric Research, Broad Institute of MIT and HarvardStanley Center for Psychiatric Research, Broad Institute of MIT and HarvardStanley Center for Psychiatric Research, Broad Institute of MIT and HarvardAbstract Machine learning has been applied in recent years to categorize sleep stages (NREM, REM, and wake) using electroencephalogram (EEG) recordings; however, a well-validated sleep scoring automatic pipeline in rodent research is still not publicly available. Here, we present IntelliSleepScorer, a software package with a graphic user interface to score sleep stages automatically in mice. IntelliSleepScorer uses the light gradient boosting machine (LightGBM) to score sleep stages for each epoch of recordings. We developed LightGBM models using a large cohort of data, which consisted of 5776 h of sleep EEG and electromyogram (EMG) signals across 519 unique recordings from 124 mice. The LightGBM model achieved an overall accuracy of 95.2% and a Cohen’s kappa of 0.91, which outperforms the baseline models such as the logistic regression model (accuracy = 93.3%, kappa = 0.88) and the random forest model (accuracy = 94.3%, kappa = 0.89). The overall performance of the LightGBM model as well as the performance across different sleep stages are on par with that of the human experts. Most importantly, we validated the generalizability of the LightGBM models: (1) The LightGBM model performed well on two publicly available, independent datasets (kappa > = 0.80), which have different sampling frequency and epoch lengths; (2) The LightGBM model performed well on data recorded at a lower sampling frequency (kappa = 0.90); (3) The performance of the LightGBM model is not affected by the light/dark cycle; and (4) A modified LightGBM model performed well on data containing only one EEG and one EMG electrode (kappa > = 0.89). Taken together, the LightGBM models offer state-of-the-art performance for automatic sleep stage scoring in mice. Last, we implemented the IntelliSleepScorer software package based on the validated model to provide an out-of-box solution to sleep researchers (available for download at https://sites.broadinstitute.org/pan-lab/resources ).https://doi.org/10.1038/s41598-023-31288-2 |
| spellingShingle | Lei A. Wang Ryan Kern Eunah Yu Soonwook Choi Jen Q. Pan IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm Scientific Reports |
| title | IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| title_full | IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| title_fullStr | IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| title_full_unstemmed | IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| title_short | IntelliSleepScorer, a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| title_sort | intellisleepscorer a software package with a graphic user interface for automated sleep stage scoring in mice based on a light gradient boosting machine algorithm |
| url | https://doi.org/10.1038/s41598-023-31288-2 |
| work_keys_str_mv | AT leiawang intellisleepscorerasoftwarepackagewithagraphicuserinterfaceforautomatedsleepstagescoringinmicebasedonalightgradientboostingmachinealgorithm AT ryankern intellisleepscorerasoftwarepackagewithagraphicuserinterfaceforautomatedsleepstagescoringinmicebasedonalightgradientboostingmachinealgorithm AT eunahyu intellisleepscorerasoftwarepackagewithagraphicuserinterfaceforautomatedsleepstagescoringinmicebasedonalightgradientboostingmachinealgorithm AT soonwookchoi intellisleepscorerasoftwarepackagewithagraphicuserinterfaceforautomatedsleepstagescoringinmicebasedonalightgradientboostingmachinealgorithm AT jenqpan intellisleepscorerasoftwarepackagewithagraphicuserinterfaceforautomatedsleepstagescoringinmicebasedonalightgradientboostingmachinealgorithm |