A reinforcement learning agent for head and neck intensity-modulated radiation therapy
Head and neck (HN) cancers pose a difficult problem in the planning of intensity-modulated radiation therapy (IMRT) treatment. The primary tumor can be large and asymmetrical, and multiple organs at risk (OARs) with varying dose-sparing goals lie close to the target volume. Currently, there is no sy...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-02-01
|
| Series: | Frontiers in Physics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fphy.2024.1331849/full |
| _version_ | 1797337305347784704 |
|---|---|
| author | Hunter Stephens Xinyi Li Yang Sheng Qiuwen Wu Yaorong Ge Q. Jackie Wu |
| author_facet | Hunter Stephens Xinyi Li Yang Sheng Qiuwen Wu Yaorong Ge Q. Jackie Wu |
| author_sort | Hunter Stephens |
| collection | DOAJ |
| description | Head and neck (HN) cancers pose a difficult problem in the planning of intensity-modulated radiation therapy (IMRT) treatment. The primary tumor can be large and asymmetrical, and multiple organs at risk (OARs) with varying dose-sparing goals lie close to the target volume. Currently, there is no systematic way of automating the generation of IMRT plans, and the manual options face planning quality and long planning time challenges. In this article, we present a reinforcement learning (RL) model for the purposes of providing automated treatment planning to reduce clinical workflow time as well as providing a better starting point for human planners to modify and build upon. Several models with progressing complexity are presented, including the relevant plan dosimetry analysis and model interpretations of the resulting strategies learned by the auto-planning agent. Models were trained on a set of 40 patients and validated on a set of 20 patients. The presented models are shown to be consistent with the requirements of an RL model to be underpinned by a Markov decision process (MDP). In-depth interpretability of the models is presented by examination of the decision space using action hyperplanes. The auto-planning agent was able to generate plans with superior reduction in the mean dose of the left and right parotid glands by approximately 7 Gy ± 2.5 Gy (p < 0.01) over a starting, static template plan with only pre-defined general prescription information. RL plans were comparable to a human expert’s clinical plans for the primary (44 Gy), boost (26 Gy) , and the summed plans (70 Gy) with p-values of 0.43, 0.72, and 0.67, respectively, for the dosimetric endpoints and uniform target coverage normalization. The RL planning agent was able to produce the plans used in validation in an average of 13.58 min, with a minimum and a maximum planning time of 2.27 and 44.82 min, respectively. |
| first_indexed | 2024-03-08T09:07:37Z |
| format | Article |
| id | doaj.art-a99eb6d2ab604d5ca94234c3e1c6343c |
| institution | Directory Open Access Journal |
| issn | 2296-424X |
| language | English |
| last_indexed | 2024-03-08T09:07:37Z |
| publishDate | 2024-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Physics |
| spelling | doaj.art-a99eb6d2ab604d5ca94234c3e1c6343c2024-02-01T04:20:51ZengFrontiers Media S.A.Frontiers in Physics2296-424X2024-02-011210.3389/fphy.2024.13318491331849A reinforcement learning agent for head and neck intensity-modulated radiation therapyHunter Stephens0Xinyi Li1Yang Sheng2Qiuwen Wu3Yaorong Ge4Q. Jackie Wu5Department of Radiation Oncology, Duke University, Durham, NC, United StatesDepartment of Radiation Oncology, Duke University, Durham, NC, United StatesDepartment of Radiation Oncology, Duke University, Durham, NC, United StatesDepartment of Radiation Oncology, Duke University, Durham, NC, United StatesDepartment of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United StatesDepartment of Radiation Oncology, Duke University, Durham, NC, United StatesHead and neck (HN) cancers pose a difficult problem in the planning of intensity-modulated radiation therapy (IMRT) treatment. The primary tumor can be large and asymmetrical, and multiple organs at risk (OARs) with varying dose-sparing goals lie close to the target volume. Currently, there is no systematic way of automating the generation of IMRT plans, and the manual options face planning quality and long planning time challenges. In this article, we present a reinforcement learning (RL) model for the purposes of providing automated treatment planning to reduce clinical workflow time as well as providing a better starting point for human planners to modify and build upon. Several models with progressing complexity are presented, including the relevant plan dosimetry analysis and model interpretations of the resulting strategies learned by the auto-planning agent. Models were trained on a set of 40 patients and validated on a set of 20 patients. The presented models are shown to be consistent with the requirements of an RL model to be underpinned by a Markov decision process (MDP). In-depth interpretability of the models is presented by examination of the decision space using action hyperplanes. The auto-planning agent was able to generate plans with superior reduction in the mean dose of the left and right parotid glands by approximately 7 Gy ± 2.5 Gy (p < 0.01) over a starting, static template plan with only pre-defined general prescription information. RL plans were comparable to a human expert’s clinical plans for the primary (44 Gy), boost (26 Gy) , and the summed plans (70 Gy) with p-values of 0.43, 0.72, and 0.67, respectively, for the dosimetric endpoints and uniform target coverage normalization. The RL planning agent was able to produce the plans used in validation in an average of 13.58 min, with a minimum and a maximum planning time of 2.27 and 44.82 min, respectively.https://www.frontiersin.org/articles/10.3389/fphy.2024.1331849/fullreinforcement learningradiation therapyautomated treatment planninghead and neck cancerinterpretable machine learning |
| spellingShingle | Hunter Stephens Xinyi Li Yang Sheng Qiuwen Wu Yaorong Ge Q. Jackie Wu A reinforcement learning agent for head and neck intensity-modulated radiation therapy Frontiers in Physics reinforcement learning radiation therapy automated treatment planning head and neck cancer interpretable machine learning |
| title | A reinforcement learning agent for head and neck intensity-modulated radiation therapy |
| title_full | A reinforcement learning agent for head and neck intensity-modulated radiation therapy |
| title_fullStr | A reinforcement learning agent for head and neck intensity-modulated radiation therapy |
| title_full_unstemmed | A reinforcement learning agent for head and neck intensity-modulated radiation therapy |
| title_short | A reinforcement learning agent for head and neck intensity-modulated radiation therapy |
| title_sort | reinforcement learning agent for head and neck intensity modulated radiation therapy |
| topic | reinforcement learning radiation therapy automated treatment planning head and neck cancer interpretable machine learning |
| url | https://www.frontiersin.org/articles/10.3389/fphy.2024.1331849/full |
| work_keys_str_mv | AT hunterstephens areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT xinyili areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT yangsheng areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT qiuwenwu areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT yaorongge areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT qjackiewu areinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT hunterstephens reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT xinyili reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT yangsheng reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT qiuwenwu reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT yaorongge reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy AT qjackiewu reinforcementlearningagentforheadandneckintensitymodulatedradiationtherapy |