Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, w...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/141680 |
| _version_ | 1826127945258762240 |
|---|---|
| author | Lauren, Paula Qu, Guangzhi Yang, Jucheng Watta, Paul Huang, Guang-Bin Lendasse, Amaury |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Lauren, Paula Qu, Guangzhi Yang, Jucheng Watta, Paul Huang, Guang-Bin Lendasse, Amaury |
| author_sort | Lauren, Paula |
| collection | NTU |
| description | Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, we proposed using an Extreme Learning Machine (ELM) for generating word embeddings. In this research, we apply the ELM-based Word Embeddings to the NLP task of Text Categorization, specifically Sentiment Analysis and Sequence Labeling. The ELM-based Word Embeddings utilizes a count-based approach similar to the Global Vectors (GloVe) model, where the word-context matrix is computed then matrix factorization is applied. A comparative study is done with Word2Vec and GloVe, which are the two popular state-of-the-art models. The results show that ELM-based Word Embeddings slightly outperforms the aforementioned two methods in the Sentiment Analysis and Sequence Labeling tasks.In addition, only one hyperparameter is needed using ELM whereas several are utilized for the other methods. ELM-based Word Embeddings are comparable to the state-of-the-art methods: Word2Vec and GloVe models. In addition, the count-based ELM model have word similarities to both the count-based GloVe and the predict-based Word2Vec models, with subtle differences. |
| first_indexed | 2024-10-01T07:17:08Z |
| format | Journal Article |
| id | ntu-10356/141680 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:17:08Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1416802020-06-10T02:46:10Z Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks Lauren, Paula Qu, Guangzhi Yang, Jucheng Watta, Paul Huang, Guang-Bin Lendasse, Amaury School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Word Embeddings Extreme Learning Machine (ELM) Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, we proposed using an Extreme Learning Machine (ELM) for generating word embeddings. In this research, we apply the ELM-based Word Embeddings to the NLP task of Text Categorization, specifically Sentiment Analysis and Sequence Labeling. The ELM-based Word Embeddings utilizes a count-based approach similar to the Global Vectors (GloVe) model, where the word-context matrix is computed then matrix factorization is applied. A comparative study is done with Word2Vec and GloVe, which are the two popular state-of-the-art models. The results show that ELM-based Word Embeddings slightly outperforms the aforementioned two methods in the Sentiment Analysis and Sequence Labeling tasks.In addition, only one hyperparameter is needed using ELM whereas several are utilized for the other methods. ELM-based Word Embeddings are comparable to the state-of-the-art methods: Word2Vec and GloVe models. In addition, the count-based ELM model have word similarities to both the count-based GloVe and the predict-based Word2Vec models, with subtle differences. 2020-06-10T02:46:10Z 2020-06-10T02:46:10Z 2018 Journal Article Lauren, P., Qu, G., Yang, J., Watta, P., Huang, G.-B., & Lendasse, A. (2018). Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cognitive Computation, 10(4), 625-638. doi:10.1007/s12559-018-9548-y 1866-9956 https://hdl.handle.net/10356/141680 10.1007/s12559-018-9548-y 2-s2.0-85045130357 4 10 625 638 en Cognitive Computation © 2018 Springer Science+Business Media, LLC, part of Springer Nature. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Word Embeddings Extreme Learning Machine (ELM) Lauren, Paula Qu, Guangzhi Yang, Jucheng Watta, Paul Huang, Guang-Bin Lendasse, Amaury Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title | Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title_full | Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title_fullStr | Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title_full_unstemmed | Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title_short | Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| title_sort | generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks |
| topic | Engineering::Electrical and electronic engineering Word Embeddings Extreme Learning Machine (ELM) |
| url | https://hdl.handle.net/10356/141680 |
| work_keys_str_mv | AT laurenpaula generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks AT quguangzhi generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks AT yangjucheng generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks AT wattapaul generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks AT huangguangbin generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks AT lendasseamaury generatingwordembeddingsfromanextremelearningmachineforsentimentanalysisandsequencelabelingtasks |