A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting
Abstract In continual learning, previously learnt knowledge tends to be overlapped by the subsequent training tasks. This bottleneck, known as catastrophic forgetting, has recently been relieved between vision tasks involving pixel shuffles etc. Nevertheless, the challenge lies in the continuous cla...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-02-01
|
| Series: | IET Computer Vision |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cvi2.12068 |
| _version_ | 1798041293500186624 |
|---|---|
| author | Meng Wang Zhengbing Guo Huafeng Li |
| author_facet | Meng Wang Zhengbing Guo Huafeng Li |
| author_sort | Meng Wang |
| collection | DOAJ |
| description | Abstract In continual learning, previously learnt knowledge tends to be overlapped by the subsequent training tasks. This bottleneck, known as catastrophic forgetting, has recently been relieved between vision tasks involving pixel shuffles etc. Nevertheless, the challenge lies in the continuous classification of the sequential sets discriminated by global transformations, such as excessively spatial rotations. Aim at this, a novel strategy of dynamic memory routing is proposed to dominate the forward paths of capsule network (CapsNet) according to the current input sets. To recall previous knowledge, a binary routing table is maintained among these sequential tasks. Then, an increment procedure of competitive prototype clustering is integrated to update the routing of the current task. Moreover, a sparsity measurement is employed to decouple the salient routing among the different learnt tasks. The experimental results demonstrate the superiority of the proposed memory network over the state–of–the–art approaches by the recalling evaluations on extended sets of Cifar–100, CelebA and Tiny ImageNet etc. |
| first_indexed | 2024-04-11T22:19:30Z |
| format | Article |
| id | doaj.art-cae1e03ceba5405c8e85f7f1b92d817f |
| institution | Directory Open Access Journal |
| issn | 1751-9632 1751-9640 |
| language | English |
| last_indexed | 2024-04-11T22:19:30Z |
| publishDate | 2022-02-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Computer Vision |
| spelling | doaj.art-cae1e03ceba5405c8e85f7f1b92d817f2022-12-22T04:00:14ZengWileyIET Computer Vision1751-96321751-96402022-02-01161839710.1049/cvi2.12068A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgettingMeng Wang0Zhengbing Guo1Huafeng Li2Information Engineering and Automation Kunming University of Science and Technology Kunming Yunnan ChinaInformation Engineering and Automation Kunming University of Science and Technology Kunming Yunnan ChinaInformation Engineering and Automation Kunming University of Science and Technology Kunming Yunnan ChinaAbstract In continual learning, previously learnt knowledge tends to be overlapped by the subsequent training tasks. This bottleneck, known as catastrophic forgetting, has recently been relieved between vision tasks involving pixel shuffles etc. Nevertheless, the challenge lies in the continuous classification of the sequential sets discriminated by global transformations, such as excessively spatial rotations. Aim at this, a novel strategy of dynamic memory routing is proposed to dominate the forward paths of capsule network (CapsNet) according to the current input sets. To recall previous knowledge, a binary routing table is maintained among these sequential tasks. Then, an increment procedure of competitive prototype clustering is integrated to update the routing of the current task. Moreover, a sparsity measurement is employed to decouple the salient routing among the different learnt tasks. The experimental results demonstrate the superiority of the proposed memory network over the state–of–the–art approaches by the recalling evaluations on extended sets of Cifar–100, CelebA and Tiny ImageNet etc.https://doi.org/10.1049/cvi2.12068computer visionlearning (artificial intelligence)image classificationneural netspattern clustering |
| spellingShingle | Meng Wang Zhengbing Guo Huafeng Li A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting IET Computer Vision computer vision learning (artificial intelligence) image classification neural nets pattern clustering |
| title | A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting |
| title_full | A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting |
| title_fullStr | A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting |
| title_full_unstemmed | A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting |
| title_short | A dynamic routing CapsNet based on increment prototype clustering for overcoming catastrophic forgetting |
| title_sort | dynamic routing capsnet based on increment prototype clustering for overcoming catastrophic forgetting |
| topic | computer vision learning (artificial intelligence) image classification neural nets pattern clustering |
| url | https://doi.org/10.1049/cvi2.12068 |
| work_keys_str_mv | AT mengwang adynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting AT zhengbingguo adynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting AT huafengli adynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting AT mengwang dynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting AT zhengbingguo dynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting AT huafengli dynamicroutingcapsnetbasedonincrementprototypeclusteringforovercomingcatastrophicforgetting |