Development of finite memory neural fuzzy networks for lag-free improved time series forecasting
Time series modelling/ forecasting is one of the most popular areas of research in the machine learning and data science community. It has widespread applications across various domains from different sectors such as energy demand prediction, weather forecasting, wind speed forecasting, stock pri...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Doctor of Philosophy |
| Language: | English |
| Published: |
Nanyang Technological University
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/146938 |
| _version_ | 1811687949729792000 |
|---|---|
| author | Subhrajit Samanta |
| author2 | Jun Zhao |
| author_facet | Jun Zhao Subhrajit Samanta |
| author_sort | Subhrajit Samanta |
| collection | NTU |
| description | Time series modelling/ forecasting is one of the most popular areas of research in
the machine learning and data science community. It has widespread applications
across various domains from different sectors such as energy demand prediction,
weather forecasting, wind speed forecasting, stock price prediction, Exchange rate
prediction, pandemic growth forecasting etc. to name a few and the list goes on.
This ever-growing field of study has seen some significant improvements over the
past few decades since its inception starting from the classical statistical methods
such as Auto-Regressive Moving Average (ARMA). The recent advents in artificial
neural network based deep learning algorithms have propelled this research to a
new high. However, with ever increasing amount of data every hour of each day,
there are always more challenges and improvements to be made in this field. This
thesis addresses a few of the major challenges associated with time series modelling
which are well known to the community. I also look at some of the problems which
are relatively under-explored.
One of the major known challenges of time series modelling is capturing the inherent
temporal characteristics of the data. Most time series data are non-stationary in
nature i.e. their statistical properties change over time. To model such data, it is
of utmost importance to trace the underlying dynamics i.e. the temporal behavior.
Another, big problem of time series modelling is to handle the uncertainty. Realworld time series data are often riddled with noise and volatility. Hence it is crucial
to address the issue of uncertainty associated with such real-world data. Online
learning from a time series data can also pose a novel issue as it requires the model
to learn in a single epoch without any repetition. In the first two chapters, this
thesis proposes two novel neuro-fuzzy systems to address these three challenges.
A Spatio-Temporal Fuzzy Inference System (or SPATFIS) with memory type neurons is proposed first which retains all past information to capture the system
dynamics. It utilizes a new self-adaptive learning mechanism to add, eliminate and
unify its fuzzy rules which helps it to attain a parsimonious structure. At the same
time, the linguistic nature of the fuzzy inference system makes it well capable of
handling the uncertainty. SPATFIS also adopts a projection-based algorithm to
update its parameters in a sequential manner thereby it is capable of online time
series forecasting. However, when the system dynamics changes rapidly then memory neurons fail to adapt to the quickly changing dynamics (as they retain the effect
of all past instances, thereby its output takes longer to get adjusted to a sudden
shock). Therefore, it is very important to ensure that the temporal output is finite
in nature and it is not depending on all pasts.
The Neuro-Fuzzy Inference System with Dynamic Neurons (NFIS-DN) addresses
the aforementioned problem. The proposed Dynamic Neuron (DN) is designed in
such way, that its temporal output considers only the effect of finite past instances,
enabling the system with finite memory. The benefits of dynamic neurons (DNs)
become more apparent in areas where there is a sudden change in the system
dynamics as commonly seen in non-linear dynamic system identification problems.
However, there is one drawback to the implementation of DN. All the samples need
to be present to perform an empirical test to choose the number of past instances
required for the model (i.e. temporal order or temporality: N), thereby making
this process offline in nature. Hence in case of an online learning scenario, this
method of choosing N will not hold.
Hence, a novel Bayesian method of online temporality analysis is proposed to estimate how many past instances a model requires for an improved time series
forecasting; this results in finite memory. Temporality change or drift can be a
common occurrence in online time series, hence a drift detection mechanism is also
developed. The entire mechanism is termed as Learning Elastic Memory Online
or LEMON. This method is agnostic of the underlying network or learning mechanism and can be utilized in any time series model. A neuro-fuzzy adaptation of
LEMON is also developed by the name of Bayesian Neuro-Fuzzy Inference System
(BaNFIS) to handle the problem of online temporality estimation and uncertainty
handling simultaneously.
NFIS-DN, LEMON, and BaNFIS, all three of these models utilize only finite past
information (hence finite memory) within their respective neural fuzzy frameworks
to provide superior prediction performance. Apart from the aforementioned challenges, there is one more challenge i.e. ‘Lag in predicted sequence’ which is often
overlooked in the time series literature as it does not contribute to a high mean
squared prediction error. Most neural network and neuro-fuzzy based time series
models in literature (including proposed SPATFIS, NFIS-DN, LEMON and BaNFIS), trained with historical data alone can often lead to a lagged time series where
the predicted sequence is always trailing the original sequence. This leads to poor
forecast in terms of movement prediction.
In the final chapter of this thesis a novel trend driven Dual Network Solution (DNS)
is proposed. Trend, defined as the inherent pattern of the data, is extracted here
and utilized next to perform lag-free forecasting. DNS exhibits a substantially
improved performance compared to more complex and resource-intensive state-ofthe-art algorithms in large scale time series problems. Apart from the traditional
Mean Squared Error (MSE), a new Movement Prediction Metric or MPM (for detection of lag in time series) is also developed as a new complementary performance
metric to evaluate the efficacy of DNS better.
The performance of each of these proposed methods is evaluated meticulously on
several benchmark data sets as well as real-world problems with a main focus on
energy demand prediction and wind speed forecasting along with financial indicator
prediction. The superior performance of the proposed methods further indicates
their applicability across domains, especially in the energy sector. These methods could be instrumental for producing better renewable energy, reducing energy
wastage, better planning and grid management etc. for a sustainable earth.
Overall, this thesis tackles several problems of time series forecasting, starting with
the challenging tasks of capturing system dynamics, online learning and handling
uncertainty of real-world data. It also addresses some under-explored issues of
this domain including estimation of temporal order and the problem of lag. Future
research in this domain, can benefit from this thesis and improve upon the presented
methods to address other fundamental challenges of time series modelling |
| first_indexed | 2024-10-01T05:24:26Z |
| format | Thesis-Doctor of Philosophy |
| id | ntu-10356/146938 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:24:26Z |
| publishDate | 2021 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1469382023-03-05T16:35:06Z Development of finite memory neural fuzzy networks for lag-free improved time series forecasting Subhrajit Samanta Jun Zhao Interdisciplinary Graduate School (IGS) Energy Research Institute @ NTU (ERI@N) Narasimalu Srikanth junzhao@ntu.edu.sg, nsrikanth@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Time series modelling/ forecasting is one of the most popular areas of research in the machine learning and data science community. It has widespread applications across various domains from different sectors such as energy demand prediction, weather forecasting, wind speed forecasting, stock price prediction, Exchange rate prediction, pandemic growth forecasting etc. to name a few and the list goes on. This ever-growing field of study has seen some significant improvements over the past few decades since its inception starting from the classical statistical methods such as Auto-Regressive Moving Average (ARMA). The recent advents in artificial neural network based deep learning algorithms have propelled this research to a new high. However, with ever increasing amount of data every hour of each day, there are always more challenges and improvements to be made in this field. This thesis addresses a few of the major challenges associated with time series modelling which are well known to the community. I also look at some of the problems which are relatively under-explored. One of the major known challenges of time series modelling is capturing the inherent temporal characteristics of the data. Most time series data are non-stationary in nature i.e. their statistical properties change over time. To model such data, it is of utmost importance to trace the underlying dynamics i.e. the temporal behavior. Another, big problem of time series modelling is to handle the uncertainty. Realworld time series data are often riddled with noise and volatility. Hence it is crucial to address the issue of uncertainty associated with such real-world data. Online learning from a time series data can also pose a novel issue as it requires the model to learn in a single epoch without any repetition. In the first two chapters, this thesis proposes two novel neuro-fuzzy systems to address these three challenges. A Spatio-Temporal Fuzzy Inference System (or SPATFIS) with memory type neurons is proposed first which retains all past information to capture the system dynamics. It utilizes a new self-adaptive learning mechanism to add, eliminate and unify its fuzzy rules which helps it to attain a parsimonious structure. At the same time, the linguistic nature of the fuzzy inference system makes it well capable of handling the uncertainty. SPATFIS also adopts a projection-based algorithm to update its parameters in a sequential manner thereby it is capable of online time series forecasting. However, when the system dynamics changes rapidly then memory neurons fail to adapt to the quickly changing dynamics (as they retain the effect of all past instances, thereby its output takes longer to get adjusted to a sudden shock). Therefore, it is very important to ensure that the temporal output is finite in nature and it is not depending on all pasts. The Neuro-Fuzzy Inference System with Dynamic Neurons (NFIS-DN) addresses the aforementioned problem. The proposed Dynamic Neuron (DN) is designed in such way, that its temporal output considers only the effect of finite past instances, enabling the system with finite memory. The benefits of dynamic neurons (DNs) become more apparent in areas where there is a sudden change in the system dynamics as commonly seen in non-linear dynamic system identification problems. However, there is one drawback to the implementation of DN. All the samples need to be present to perform an empirical test to choose the number of past instances required for the model (i.e. temporal order or temporality: N), thereby making this process offline in nature. Hence in case of an online learning scenario, this method of choosing N will not hold. Hence, a novel Bayesian method of online temporality analysis is proposed to estimate how many past instances a model requires for an improved time series forecasting; this results in finite memory. Temporality change or drift can be a common occurrence in online time series, hence a drift detection mechanism is also developed. The entire mechanism is termed as Learning Elastic Memory Online or LEMON. This method is agnostic of the underlying network or learning mechanism and can be utilized in any time series model. A neuro-fuzzy adaptation of LEMON is also developed by the name of Bayesian Neuro-Fuzzy Inference System (BaNFIS) to handle the problem of online temporality estimation and uncertainty handling simultaneously. NFIS-DN, LEMON, and BaNFIS, all three of these models utilize only finite past information (hence finite memory) within their respective neural fuzzy frameworks to provide superior prediction performance. Apart from the aforementioned challenges, there is one more challenge i.e. ‘Lag in predicted sequence’ which is often overlooked in the time series literature as it does not contribute to a high mean squared prediction error. Most neural network and neuro-fuzzy based time series models in literature (including proposed SPATFIS, NFIS-DN, LEMON and BaNFIS), trained with historical data alone can often lead to a lagged time series where the predicted sequence is always trailing the original sequence. This leads to poor forecast in terms of movement prediction. In the final chapter of this thesis a novel trend driven Dual Network Solution (DNS) is proposed. Trend, defined as the inherent pattern of the data, is extracted here and utilized next to perform lag-free forecasting. DNS exhibits a substantially improved performance compared to more complex and resource-intensive state-ofthe-art algorithms in large scale time series problems. Apart from the traditional Mean Squared Error (MSE), a new Movement Prediction Metric or MPM (for detection of lag in time series) is also developed as a new complementary performance metric to evaluate the efficacy of DNS better. The performance of each of these proposed methods is evaluated meticulously on several benchmark data sets as well as real-world problems with a main focus on energy demand prediction and wind speed forecasting along with financial indicator prediction. The superior performance of the proposed methods further indicates their applicability across domains, especially in the energy sector. These methods could be instrumental for producing better renewable energy, reducing energy wastage, better planning and grid management etc. for a sustainable earth. Overall, this thesis tackles several problems of time series forecasting, starting with the challenging tasks of capturing system dynamics, online learning and handling uncertainty of real-world data. It also addresses some under-explored issues of this domain including estimation of temporal order and the problem of lag. Future research in this domain, can benefit from this thesis and improve upon the presented methods to address other fundamental challenges of time series modelling Doctor of Philosophy 2021-03-15T12:58:25Z 2021-03-15T12:58:25Z 2021 Thesis-Doctor of Philosophy Subhrajit Samanta (2021). Development of finite memory neural fuzzy networks for lag-free improved time series forecasting. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/146938 https://hdl.handle.net/10356/146938 10.32657/10356/146938 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Subhrajit Samanta Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title | Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title_full | Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title_fullStr | Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title_full_unstemmed | Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title_short | Development of finite memory neural fuzzy networks for lag-free improved time series forecasting |
| title_sort | development of finite memory neural fuzzy networks for lag free improved time series forecasting |
| topic | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence |
| url | https://hdl.handle.net/10356/146938 |
| work_keys_str_mv | AT subhrajitsamanta developmentoffinitememoryneuralfuzzynetworksforlagfreeimprovedtimeseriesforecasting |