Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems
Advances in technology have enabled the use of sensors with varied modalities to monitor different parts of systems, each providing diverse levels of information about the underlying system. However, resource limitations and computational power restrict the number of sensors/data that can be process...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2024-12-01
|
| Series: | Systems Science & Control Engineering |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/21642583.2024.2329260 |
| _version_ | 1826930066596036608 |
|---|---|
| author | Mohammad Alali Armita Kazeminajafabadi Mahdi Imani |
| author_facet | Mohammad Alali Armita Kazeminajafabadi Mahdi Imani |
| author_sort | Mohammad Alali |
| collection | DOAJ |
| description | Advances in technology have enabled the use of sensors with varied modalities to monitor different parts of systems, each providing diverse levels of information about the underlying system. However, resource limitations and computational power restrict the number of sensors/data that can be processed in real-time in most complex systems. These challenges necessitate the need for selecting/scheduling a subset of sensors to obtain measurements that guarantee the best monitoring objectives. This paper focuses on sensor scheduling for systems modeled by hidden Markov models. Despite the development of several sensor selection and scheduling methods, existing methods tend to be greedy and do not take into account the long-term impact of selected sensors on monitoring objectives. This paper formulates optimal sensor scheduling as a reinforcement learning problem defined over the posterior distribution of system states. Further, the paper derives a deep reinforcement learning policy for offline learning of the sensor scheduling policy, which can then be executed in real-time as new information unfolds. The proposed method applies to any monitoring objective that can be expressed in terms of the posterior distribution of the states (e.g. state estimation, information gain, etc.). The performance of the proposed method in terms of accuracy and robustness is investigated for monitoring the security of networked systems and the health monitoring of gene regulatory networks. |
| first_indexed | 2024-04-24T05:50:44Z |
| format | Article |
| id | doaj.art-01b539f6ae18431aa65012b184950c5f |
| institution | Directory Open Access Journal |
| issn | 2164-2583 |
| language | English |
| last_indexed | 2025-02-17T16:21:09Z |
| publishDate | 2024-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Systems Science & Control Engineering |
| spelling | doaj.art-01b539f6ae18431aa65012b184950c5f2024-12-17T09:06:12ZengTaylor & Francis GroupSystems Science & Control Engineering2164-25832024-12-0112110.1080/21642583.2024.2329260Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systemsMohammad Alali0Armita Kazeminajafabadi1Mahdi Imani2Northeastern University, Boston, MA, USANortheastern University, Boston, MA, USANortheastern University, Boston, MA, USAAdvances in technology have enabled the use of sensors with varied modalities to monitor different parts of systems, each providing diverse levels of information about the underlying system. However, resource limitations and computational power restrict the number of sensors/data that can be processed in real-time in most complex systems. These challenges necessitate the need for selecting/scheduling a subset of sensors to obtain measurements that guarantee the best monitoring objectives. This paper focuses on sensor scheduling for systems modeled by hidden Markov models. Despite the development of several sensor selection and scheduling methods, existing methods tend to be greedy and do not take into account the long-term impact of selected sensors on monitoring objectives. This paper formulates optimal sensor scheduling as a reinforcement learning problem defined over the posterior distribution of system states. Further, the paper derives a deep reinforcement learning policy for offline learning of the sensor scheduling policy, which can then be executed in real-time as new information unfolds. The proposed method applies to any monitoring objective that can be expressed in terms of the posterior distribution of the states (e.g. state estimation, information gain, etc.). The performance of the proposed method in terms of accuracy and robustness is investigated for monitoring the security of networked systems and the health monitoring of gene regulatory networks.https://www.tandfonline.com/doi/10.1080/21642583.2024.2329260Sensor schedulingmonitoringhidden Markov modelsstate estimationreinforcement learning |
| spellingShingle | Mohammad Alali Armita Kazeminajafabadi Mahdi Imani Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems Systems Science & Control Engineering Sensor scheduling monitoring hidden Markov models state estimation reinforcement learning |
| title | Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| title_full | Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| title_fullStr | Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| title_full_unstemmed | Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| title_short | Deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| title_sort | deep reinforcement learning sensor scheduling for effective monitoring of dynamical systems |
| topic | Sensor scheduling monitoring hidden Markov models state estimation reinforcement learning |
| url | https://www.tandfonline.com/doi/10.1080/21642583.2024.2329260 |
| work_keys_str_mv | AT mohammadalali deepreinforcementlearningsensorschedulingforeffectivemonitoringofdynamicalsystems AT armitakazeminajafabadi deepreinforcementlearningsensorschedulingforeffectivemonitoringofdynamicalsystems AT mahdiimani deepreinforcementlearningsensorschedulingforeffectivemonitoringofdynamicalsystems |