Online and continual learning using randomization based deep neural networks
Deep neural networks have shown their promise in recent years with their state-of-the-art results. Yet, they suffer from some issues such as the time-consuming training process and catastrophic forgetting. In this work we look to overcome them by combining the advantages of an online learning pro...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Research |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/165774 |
| _version_ | 1811679379560857600 |
|---|---|
| author | Sreenivasan, Shiva |
| author2 | Radhakrishnan K |
| author_facet | Radhakrishnan K Sreenivasan, Shiva |
| author_sort | Sreenivasan, Shiva |
| collection | NTU |
| description | Deep neural networks have shown their promise in recent years with their state-of-the-art results.
Yet, they suffer from some issues such as the time-consuming training process and catastrophic
forgetting. In this work we look to overcome them by combining the advantages of an
online learning process as new data arrives and a system with fast and effective learning capability
such as the Random Vector Functional Link (RVFL) which is a Randomization based Deep
Neural Network. Our approach involves allowing the model to grow incrementally as new data
is made available so that it can more resemble real-life learning scenarios. Although RVFL
network was proposed as a single-hidden layer feedforward neural networks (SLFNs), deep
variants have been recently developed. As opposed to conventional neural networks adjusting
network weights iteratively, RVFL uses a simple learning method without iterative parameter
learning.
Keywords: RVFL, Online Learning, Continual Learning. |
| first_indexed | 2024-10-01T03:08:13Z |
| format | Thesis-Master by Research |
| id | ntu-10356/165774 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:08:13Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1657742023-07-04T16:17:23Z Online and continual learning using randomization based deep neural networks Sreenivasan, Shiva Radhakrishnan K School of Electrical and Electronic Engineering ERADHA@ntu.edu.sg Engineering::Electrical and electronic engineering Deep neural networks have shown their promise in recent years with their state-of-the-art results. Yet, they suffer from some issues such as the time-consuming training process and catastrophic forgetting. In this work we look to overcome them by combining the advantages of an online learning process as new data arrives and a system with fast and effective learning capability such as the Random Vector Functional Link (RVFL) which is a Randomization based Deep Neural Network. Our approach involves allowing the model to grow incrementally as new data is made available so that it can more resemble real-life learning scenarios. Although RVFL network was proposed as a single-hidden layer feedforward neural networks (SLFNs), deep variants have been recently developed. As opposed to conventional neural networks adjusting network weights iteratively, RVFL uses a simple learning method without iterative parameter learning. Keywords: RVFL, Online Learning, Continual Learning. Master of Engineering 2023-04-10T05:21:26Z 2023-04-10T05:21:26Z 2022 Thesis-Master by Research Sreenivasan, S. (2022). Online and continual learning using randomization based deep neural networks. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/165774 https://hdl.handle.net/10356/165774 10.32657/10356/165774 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering Sreenivasan, Shiva Online and continual learning using randomization based deep neural networks |
| title | Online and continual learning using randomization based deep neural networks |
| title_full | Online and continual learning using randomization based deep neural networks |
| title_fullStr | Online and continual learning using randomization based deep neural networks |
| title_full_unstemmed | Online and continual learning using randomization based deep neural networks |
| title_short | Online and continual learning using randomization based deep neural networks |
| title_sort | online and continual learning using randomization based deep neural networks |
| topic | Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/165774 |
| work_keys_str_mv | AT sreenivasanshiva onlineandcontinuallearningusingrandomizationbaseddeepneuralnetworks |