An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams
Existing fuzzy neural networks (FNNs) are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This article proposes a novel self-organizing deep FNN, namely deep evolving fuzzy neural network (DEVFNN). Fuzzy rules can be automatical...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/161032 |
| _version_ | 1826121344694091776 |
|---|---|
| author | Pratama, Mahardhika Pedrycz, Witold Webb, Geoffrey I. |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Pratama, Mahardhika Pedrycz, Witold Webb, Geoffrey I. |
| author_sort | Pratama, Mahardhika |
| collection | NTU |
| description | Existing fuzzy neural networks (FNNs) are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This article proposes a novel self-organizing deep FNN, namely deep evolving fuzzy neural network (DEVFNN). Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method, which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. The DEVFNN is developed under the stacked generalization principle via the feature augmentation concept, where a recently developed algorithm, namely generic classifier, drives the hidden layer. It is equipped by an automatic feature selection method, which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent the uncontrollable growth of dimensionality of input space due to the nature of the feature augmentation approach in building a deep network structure. The DEVFNN works in the samplewise fashion and is compatible for data stream applications. The efficacy of the DEVFNN has been thoroughly evaluated using seven datasets with nonstationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart, where the DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept of the drift detection method is an effective tool to control the depth of the network structure, while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance. |
| first_indexed | 2024-10-01T05:30:48Z |
| format | Journal Article |
| id | ntu-10356/161032 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:30:48Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1610322022-08-12T04:11:26Z An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams Pratama, Mahardhika Pedrycz, Witold Webb, Geoffrey I. School of Computer Science and Engineering Engineering::Computer science and engineering Fuzzy Neural Networks Merging Existing fuzzy neural networks (FNNs) are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This article proposes a novel self-organizing deep FNN, namely deep evolving fuzzy neural network (DEVFNN). Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method, which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. The DEVFNN is developed under the stacked generalization principle via the feature augmentation concept, where a recently developed algorithm, namely generic classifier, drives the hidden layer. It is equipped by an automatic feature selection method, which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent the uncontrollable growth of dimensionality of input space due to the nature of the feature augmentation approach in building a deep network structure. The DEVFNN works in the samplewise fashion and is compatible for data stream applications. The efficacy of the DEVFNN has been thoroughly evaluated using seven datasets with nonstationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart, where the DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept of the drift detection method is an effective tool to control the depth of the network structure, while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance. Ministry of Education (MOE) This work was supported by the MOE Tier 1 Research Grant (RG130/17). 2022-08-12T04:11:26Z 2022-08-12T04:11:26Z 2019 Journal Article Pratama, M., Pedrycz, W. & Webb, G. I. (2019). An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams. IEEE Transactions On Fuzzy Systems, 28(7), 1315-1328. https://dx.doi.org/10.1109/TFUZZ.2019.2939993 1063-6706 https://hdl.handle.net/10356/161032 10.1109/TFUZZ.2019.2939993 2-s2.0-85089481234 7 28 1315 1328 en RG130/17 IEEE Transactions on Fuzzy Systems © 2019 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Fuzzy Neural Networks Merging Pratama, Mahardhika Pedrycz, Witold Webb, Geoffrey I. An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title | An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title_full | An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title_fullStr | An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title_full_unstemmed | An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title_short | An incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| title_sort | incremental construction of deep neuro fuzzy system for continual learning of nonstationary data streams |
| topic | Engineering::Computer science and engineering Fuzzy Neural Networks Merging |
| url | https://hdl.handle.net/10356/161032 |
| work_keys_str_mv | AT pratamamahardhika anincrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams AT pedryczwitold anincrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams AT webbgeoffreyi anincrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams AT pratamamahardhika incrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams AT pedryczwitold incrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams AT webbgeoffreyi incrementalconstructionofdeepneurofuzzysystemforcontinuallearningofnonstationarydatastreams |