Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View
An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrat...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/12/1709 |
| _version_ | 1797459353584795648 |
|---|---|
| author | Xiaochuan Sun Mingxiang Hao Yutong Wang Yu Wang Zhigang Li Yingqi Li |
| author_facet | Xiaochuan Sun Mingxiang Hao Yutong Wang Yu Wang Zhigang Li Yingqi Li |
| author_sort | Xiaochuan Sun |
| collection | DOAJ |
| description | An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework. |
| first_indexed | 2024-03-09T16:51:15Z |
| format | Article |
| id | doaj.art-d05c00a2dc3c46a39045af7278ccf0d3 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T16:51:15Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-d05c00a2dc3c46a39045af7278ccf0d32023-11-24T14:41:33ZengMDPI AGEntropy1099-43002022-11-012412170910.3390/e24121709Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy ViewXiaochuan Sun0Mingxiang Hao1Yutong Wang2Yu Wang3Zhigang Li4Yingqi Li5College of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, ChinaCollege of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, ChinaCollege of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, ChinaCollege of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, ChinaCollege of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, ChinaSchool of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, ChinaAn echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the “black-box” nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity–entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.https://www.mdpi.com/1099-4300/24/12/1709echo state networktime series predictioninterpretabilityPEreservoir richnessprojection capability |
| spellingShingle | Xiaochuan Sun Mingxiang Hao Yutong Wang Yu Wang Zhigang Li Yingqi Li Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View Entropy echo state network time series prediction interpretability PE reservoir richness projection capability |
| title | Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View |
| title_full | Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View |
| title_fullStr | Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View |
| title_full_unstemmed | Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View |
| title_short | Reservoir Dynamic Interpretability for Time Series Prediction: A Permutation Entropy View |
| title_sort | reservoir dynamic interpretability for time series prediction a permutation entropy view |
| topic | echo state network time series prediction interpretability PE reservoir richness projection capability |
| url | https://www.mdpi.com/1099-4300/24/12/1709 |
| work_keys_str_mv | AT xiaochuansun reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview AT mingxianghao reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview AT yutongwang reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview AT yuwang reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview AT zhigangli reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview AT yingqili reservoirdynamicinterpretabilityfortimeseriespredictionapermutationentropyview |