A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach
Learning automaton (LA), a powerful tool in reinforcement learning, is of crucial importance for its adaptivity in the stochastic environment and its applicability in various engineering fields. In particular, the LA adaptively explores the optimal action that maximizes the reward among all possible...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8653809/ |
| _version_ | 1818912984773165056 |
|---|---|
| author | Chong Di Shenghong Li Fangqi Li Kaiyue Qi |
| author_facet | Chong Di Shenghong Li Fangqi Li Kaiyue Qi |
| author_sort | Chong Di |
| collection | DOAJ |
| description | Learning automaton (LA), a powerful tool in reinforcement learning, is of crucial importance for its adaptivity in the stochastic environment and its applicability in various engineering fields. In particular, the LA adaptively explores the optimal action that maximizes the reward among all possible choices by interacting with the environment. However, the traditional frameworks for LA have several limitations in practical applications, e.g., the cost of parameter tuning and predicaments in massive-action environments, preventing them from being applied to time-sensitive and resources-restricted tasks. In this paper, we propose a novel LA framework based on the statistical hypothesis testing, where the actions are compared by statistical hypothesis iteratively and the suboptimal ones are dismissed, and the estimated optimal action is attained. Apart from the proposal, the theoretical analyses for the framework are given to reveal its e-optimality. The proposed framework also features efficiency in massive-action environments and the parameter-free property. The comprehensive simulations are conducted in both benchmark and massiveaction environments to demonstrate the superiority of the proposed framework over the ordinary schemes. |
| first_indexed | 2024-12-19T23:23:17Z |
| format | Article |
| id | doaj.art-6e2e03f562994abba95fca8fc1c7dbdf |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T23:23:17Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-6e2e03f562994abba95fca8fc1c7dbdf2022-12-21T20:01:55ZengIEEEIEEE Access2169-35362019-01-017279112792210.1109/ACCESS.2019.29019418653809A Novel Framework for Learning Automata: A Statistical Hypothesis Testing ApproachChong Di0https://orcid.org/0000-0002-6008-1813Shenghong Li1Fangqi Li2Kaiyue Qi3School of Cyber Security, School of Electronic Information and Electrical Engineering, Artificial Intelligence Institute, Shanghai Jiao Tong University, Shanghai, ChinaSchool of Cyber Security, School of Electronic Information and Electrical Engineering, Artificial Intelligence Institute, Shanghai Jiao Tong University, Shanghai, ChinaSchool of Cyber Security, School of Electronic Information and Electrical Engineering, Artificial Intelligence Institute, Shanghai Jiao Tong University, Shanghai, ChinaSchool of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, ChinaLearning automaton (LA), a powerful tool in reinforcement learning, is of crucial importance for its adaptivity in the stochastic environment and its applicability in various engineering fields. In particular, the LA adaptively explores the optimal action that maximizes the reward among all possible choices by interacting with the environment. However, the traditional frameworks for LA have several limitations in practical applications, e.g., the cost of parameter tuning and predicaments in massive-action environments, preventing them from being applied to time-sensitive and resources-restricted tasks. In this paper, we propose a novel LA framework based on the statistical hypothesis testing, where the actions are compared by statistical hypothesis iteratively and the suboptimal ones are dismissed, and the estimated optimal action is attained. Apart from the proposal, the theoretical analyses for the framework are given to reveal its e-optimality. The proposed framework also features efficiency in massive-action environments and the parameter-free property. The comprehensive simulations are conducted in both benchmark and massiveaction environments to demonstrate the superiority of the proposed framework over the ordinary schemes.https://ieeexplore.ieee.org/document/8653809/Learning automatareinforcement learningstatistical inferenceparameter-free |
| spellingShingle | Chong Di Shenghong Li Fangqi Li Kaiyue Qi A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach IEEE Access Learning automata reinforcement learning statistical inference parameter-free |
| title | A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach |
| title_full | A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach |
| title_fullStr | A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach |
| title_full_unstemmed | A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach |
| title_short | A Novel Framework for Learning Automata: A Statistical Hypothesis Testing Approach |
| title_sort | novel framework for learning automata a statistical hypothesis testing approach |
| topic | Learning automata reinforcement learning statistical inference parameter-free |
| url | https://ieeexplore.ieee.org/document/8653809/ |
| work_keys_str_mv | AT chongdi anovelframeworkforlearningautomataastatisticalhypothesistestingapproach AT shenghongli anovelframeworkforlearningautomataastatisticalhypothesistestingapproach AT fangqili anovelframeworkforlearningautomataastatisticalhypothesistestingapproach AT kaiyueqi anovelframeworkforlearningautomataastatisticalhypothesistestingapproach AT chongdi novelframeworkforlearningautomataastatisticalhypothesistestingapproach AT shenghongli novelframeworkforlearningautomataastatisticalhypothesistestingapproach AT fangqili novelframeworkforlearningautomataastatisticalhypothesistestingapproach AT kaiyueqi novelframeworkforlearningautomataastatisticalhypothesistestingapproach |