In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus
In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level contr...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-09-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/18/6350 |
| _version_ | 1797553875447709696 |
|---|---|
| author | Jonas Nordhaug Myhre Miguel Tejedor Ilkka Kalervo Launonen Anas El Fathi Fred Godtliebsen |
| author_facet | Jonas Nordhaug Myhre Miguel Tejedor Ilkka Kalervo Launonen Anas El Fathi Fred Godtliebsen |
| author_sort | Jonas Nordhaug Myhre |
| collection | DOAJ |
| description | In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation. |
| first_indexed | 2024-03-10T16:23:50Z |
| format | Article |
| id | doaj.art-b95994f2fdfd4a7891c86f46676733ff |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T16:23:50Z |
| publishDate | 2020-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-b95994f2fdfd4a7891c86f46676733ff2023-11-20T13:29:19ZengMDPI AGApplied Sciences2076-34172020-09-011018635010.3390/app10186350In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes MellitusJonas Nordhaug Myhre0Miguel Tejedor1Ilkka Kalervo Launonen2Anas El Fathi3Fred Godtliebsen4Department of Physics and Technology, UiT-The Arctic University of Norway, 9019 Tromso, NorwayDepartment of Computer Science, UiT-The Arctic University of Norway, 9019 Tromso, NorwayDepartment of Clinical Research, The University Hospital of North-Norway, 9019 Tromso, NorwayThe McGill Artificial Pancreas Lab, McGill University, Montreal, QC H3A 2B4, CanadaDepartment of Mathematics and Statistics, UiT-The Arctic University of Norway, 9019 Tromso, NorwayIn this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.https://www.mdpi.com/2076-3417/10/18/6350reinforcement learningType 1 Diabetes Mellituspolicy gradientdeep learningartificial pancreas |
| spellingShingle | Jonas Nordhaug Myhre Miguel Tejedor Ilkka Kalervo Launonen Anas El Fathi Fred Godtliebsen In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus Applied Sciences reinforcement learning Type 1 Diabetes Mellitus policy gradient deep learning artificial pancreas |
| title | In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus |
| title_full | In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus |
| title_fullStr | In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus |
| title_full_unstemmed | In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus |
| title_short | In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus |
| title_sort | in silico evaluation of glucose regulation using policy gradient reinforcement learning for patients with type 1 diabetes mellitus |
| topic | reinforcement learning Type 1 Diabetes Mellitus policy gradient deep learning artificial pancreas |
| url | https://www.mdpi.com/2076-3417/10/18/6350 |
| work_keys_str_mv | AT jonasnordhaugmyhre insilicoevaluationofglucoseregulationusingpolicygradientreinforcementlearningforpatientswithtype1diabetesmellitus AT migueltejedor insilicoevaluationofglucoseregulationusingpolicygradientreinforcementlearningforpatientswithtype1diabetesmellitus AT ilkkakalervolaunonen insilicoevaluationofglucoseregulationusingpolicygradientreinforcementlearningforpatientswithtype1diabetesmellitus AT anaselfathi insilicoevaluationofglucoseregulationusingpolicygradientreinforcementlearningforpatientswithtype1diabetesmellitus AT fredgodtliebsen insilicoevaluationofglucoseregulationusingpolicygradientreinforcementlearningforpatientswithtype1diabetesmellitus |