Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets
The availability of different pre-trained semantic models has enabled the quick development of machine learning components for downstream applications. However, even if texts are abundant for low-resource languages, there are very few semantic models publicly available. Most of the publicly availabl...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Future Internet |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-5903/13/11/275 |
| _version_ | 1797510217486827520 |
|---|---|
| author | Seid Muhie Yimam Abinew Ali Ayele Gopalakrishnan Venkatesh Ibrahim Gashaw Chris Biemann |
| author_facet | Seid Muhie Yimam Abinew Ali Ayele Gopalakrishnan Venkatesh Ibrahim Gashaw Chris Biemann |
| author_sort | Seid Muhie Yimam |
| collection | DOAJ |
| description | The availability of different pre-trained semantic models has enabled the quick development of machine learning components for downstream applications. However, even if texts are abundant for low-resource languages, there are very few semantic models publicly available. Most of the publicly available pre-trained models are usually built as a multilingual version of semantic models that will not fit well with the need for low-resource languages. We introduce different semantic models for Amharic, a morphologically complex Ethio-Semitic language. After we investigate the publicly available pre-trained semantic models, we fine-tune two pre-trained models and train seven new different models. The models include Word2Vec embeddings, distributional thesaurus (DT), BERT-like contextual embeddings, and DT embeddings obtained via network embedding algorithms. Moreover, we employ these models for different NLP tasks and study their impact. We find that newly-trained models perform better than pre-trained multilingual models. Furthermore, models based on contextual embeddings from FLAIR and RoBERTa perform better than word2Vec models for the NER and POS tagging tasks. DT-based network embeddings are suitable for the sentiment classification task. We publicly release all the semantic models, machine learning components, and several benchmark datasets such as NER, POS tagging, sentiment classification, as well as Amharic versions of WordSim353 and SimLex999. |
| first_indexed | 2024-03-10T05:28:27Z |
| format | Article |
| id | doaj.art-544f83bbce6749a7907372d31f485747 |
| institution | Directory Open Access Journal |
| issn | 1999-5903 |
| language | English |
| last_indexed | 2024-03-10T05:28:27Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Future Internet |
| spelling | doaj.art-544f83bbce6749a7907372d31f4857472023-11-22T23:26:23ZengMDPI AGFuture Internet1999-59032021-10-01131127510.3390/fi13110275Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and DatasetsSeid Muhie Yimam0Abinew Ali Ayele1Gopalakrishnan Venkatesh2Ibrahim Gashaw3Chris Biemann4Language Technology Group, Universität Hamburg, Grindelallee 117, 20146 Hamburg, GermanyLanguage Technology Group, Universität Hamburg, Grindelallee 117, 20146 Hamburg, GermanyInternational Institute of Information Technology, Bangalore 560100, IndiaCollege of Informatics, University of Gondar, Gondar 6200, EthiopiaLanguage Technology Group, Universität Hamburg, Grindelallee 117, 20146 Hamburg, GermanyThe availability of different pre-trained semantic models has enabled the quick development of machine learning components for downstream applications. However, even if texts are abundant for low-resource languages, there are very few semantic models publicly available. Most of the publicly available pre-trained models are usually built as a multilingual version of semantic models that will not fit well with the need for low-resource languages. We introduce different semantic models for Amharic, a morphologically complex Ethio-Semitic language. After we investigate the publicly available pre-trained semantic models, we fine-tune two pre-trained models and train seven new different models. The models include Word2Vec embeddings, distributional thesaurus (DT), BERT-like contextual embeddings, and DT embeddings obtained via network embedding algorithms. Moreover, we employ these models for different NLP tasks and study their impact. We find that newly-trained models perform better than pre-trained multilingual models. Furthermore, models based on contextual embeddings from FLAIR and RoBERTa perform better than word2Vec models for the NER and POS tagging tasks. DT-based network embeddings are suitable for the sentiment classification task. We publicly release all the semantic models, machine learning components, and several benchmark datasets such as NER, POS tagging, sentiment classification, as well as Amharic versions of WordSim353 and SimLex999.https://www.mdpi.com/1999-5903/13/11/275datasetsneural networkssemantic modelsAmharic NLPlow-resource languagetext tagging |
| spellingShingle | Seid Muhie Yimam Abinew Ali Ayele Gopalakrishnan Venkatesh Ibrahim Gashaw Chris Biemann Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets Future Internet datasets neural networks semantic models Amharic NLP low-resource language text tagging |
| title | Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets |
| title_full | Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets |
| title_fullStr | Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets |
| title_full_unstemmed | Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets |
| title_short | Introducing Various Semantic Models for Amharic: Experimentation and Evaluation with Multiple Tasks and Datasets |
| title_sort | introducing various semantic models for amharic experimentation and evaluation with multiple tasks and datasets |
| topic | datasets neural networks semantic models Amharic NLP low-resource language text tagging |
| url | https://www.mdpi.com/1999-5903/13/11/275 |
| work_keys_str_mv | AT seidmuhieyimam introducingvarioussemanticmodelsforamharicexperimentationandevaluationwithmultipletasksanddatasets AT abinewaliayele introducingvarioussemanticmodelsforamharicexperimentationandevaluationwithmultipletasksanddatasets AT gopalakrishnanvenkatesh introducingvarioussemanticmodelsforamharicexperimentationandevaluationwithmultipletasksanddatasets AT ibrahimgashaw introducingvarioussemanticmodelsforamharicexperimentationandevaluationwithmultipletasksanddatasets AT chrisbiemann introducingvarioussemanticmodelsforamharicexperimentationandevaluationwithmultipletasksanddatasets |