Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding
Aspect detection systems for online reviews, especially based on unsupervised models, are considered better strategically to process online reviews, generally a very large collection of unstructured data. Aspect embedding-based deep learning models are designed for this problem however they still r...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Universitas Gadjah Mada
2022-10-01
|
| Series: | IJCCS (Indonesian Journal of Computing and Cybernetics Systems) |
| Subjects: | |
| Online Access: | https://jurnal.ugm.ac.id/ijccs/article/view/72981 |
| _version_ | 1797681730053734400 |
|---|---|
| author | Muhammad Haris Maulana Masayu Leylia Khodra |
| author_facet | Muhammad Haris Maulana Masayu Leylia Khodra |
| author_sort | Muhammad Haris Maulana |
| collection | DOAJ |
| description | Aspect detection systems for online reviews, especially based on unsupervised models, are considered better strategically to process online reviews, generally a very large collection of unstructured data. Aspect embedding-based deep learning models are designed for this problem however they still rely on redundant word embedding and they are sensitive to initialization which may have a significant impact on model performance. In this research, a pruning approach is used to reduce the redundancy of deep learning model connections and is expected to produce a model with similar or better performance. This research includes several experiments and comparisons of the results of pruning the model network weights based on the general neural network pruning strategy and the lottery ticket hypothesis. The result of this research is that pruning of the unsupervised aspect detection model, in general, can produce smaller submodels with similar performance even with a significant amount of weights pruned. Our sparse model with 80% of its total weight pruned has a similar performance to the original model. Our current pruning implementation, however, has not been able to produce sparse models with better performance. |
| first_indexed | 2024-03-11T23:49:14Z |
| format | Article |
| id | doaj.art-8458ace10195497da8efad5e216bd979 |
| institution | Directory Open Access Journal |
| issn | 1978-1520 2460-7258 |
| language | English |
| last_indexed | 2024-03-11T23:49:14Z |
| publishDate | 2022-10-01 |
| publisher | Universitas Gadjah Mada |
| record_format | Article |
| series | IJCCS (Indonesian Journal of Computing and Cybernetics Systems) |
| spelling | doaj.art-8458ace10195497da8efad5e216bd9792023-09-19T08:33:45ZengUniversitas Gadjah MadaIJCCS (Indonesian Journal of Computing and Cybernetics Systems)1978-15202460-72582022-10-0116436737810.22146/ijccs.7298132650Neural Network Pruning in Unsupervised Aspect Detection based on Aspect EmbeddingMuhammad Haris Maulana0Masayu Leylia Khodra1Master Program of Informatics, SEEI; ITB, BandungSchool of Electrical Engineering and Informatics, Institut Teknologi Bandung, BandungAspect detection systems for online reviews, especially based on unsupervised models, are considered better strategically to process online reviews, generally a very large collection of unstructured data. Aspect embedding-based deep learning models are designed for this problem however they still rely on redundant word embedding and they are sensitive to initialization which may have a significant impact on model performance. In this research, a pruning approach is used to reduce the redundancy of deep learning model connections and is expected to produce a model with similar or better performance. This research includes several experiments and comparisons of the results of pruning the model network weights based on the general neural network pruning strategy and the lottery ticket hypothesis. The result of this research is that pruning of the unsupervised aspect detection model, in general, can produce smaller submodels with similar performance even with a significant amount of weights pruned. Our sparse model with 80% of its total weight pruned has a similar performance to the original model. Our current pruning implementation, however, has not been able to produce sparse models with better performance.https://jurnal.ugm.ac.id/ijccs/article/view/72981aspect embeddingunsupervised aspect detectionpruninglottery ticket hypothesis |
| spellingShingle | Muhammad Haris Maulana Masayu Leylia Khodra Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding IJCCS (Indonesian Journal of Computing and Cybernetics Systems) aspect embedding unsupervised aspect detection pruning lottery ticket hypothesis |
| title | Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding |
| title_full | Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding |
| title_fullStr | Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding |
| title_full_unstemmed | Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding |
| title_short | Neural Network Pruning in Unsupervised Aspect Detection based on Aspect Embedding |
| title_sort | neural network pruning in unsupervised aspect detection based on aspect embedding |
| topic | aspect embedding unsupervised aspect detection pruning lottery ticket hypothesis |
| url | https://jurnal.ugm.ac.id/ijccs/article/view/72981 |
| work_keys_str_mv | AT muhammadharismaulana neuralnetworkpruninginunsupervisedaspectdetectionbasedonaspectembedding AT masayuleyliakhodra neuralnetworkpruninginunsupervisedaspectdetectionbasedonaspectembedding |