Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods
The paper describes experiments performed on two sets of manually annotated data. The task of irony and sarcasm detection in Russian sentences was solved using baseline classifiers, i. e., BERT, Bi-LSTM, SVM, Random Forest, Logistic Regression. The best achieved F1-score for each classifier was 0.76...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
FRUCT
2023-05-01
|
| Series: | Proceedings of the XXth Conference of Open Innovations Association FRUCT |
| Subjects: | |
| Online Access: | https://www.fruct.org/publications/volume-33/fruct33/files/Kos.pdf |
| _version_ | 1827929473627979776 |
|---|---|
| author | Maksim Kosterin Ilya Paramonov Nadezhda Lagutina |
| author_facet | Maksim Kosterin Ilya Paramonov Nadezhda Lagutina |
| author_sort | Maksim Kosterin |
| collection | DOAJ |
| description | The paper describes experiments performed on two sets of manually annotated data. The task of irony and sarcasm detection in Russian sentences was solved using baseline classifiers, i. e., BERT, Bi-LSTM, SVM, Random Forest, Logistic Regression. The best achieved F1-score for each classifier was 0.76, 0.73, 0.66, 0.64, 0.68 respectively. The results achieved by BERT and Bi-LSTM classifiers are comparable with the results from the articles describing the application of similar approaches for English language. Analysis of the results allowed to conclude that transferring the word context improves classification metrics and refinement of training data allows to improve the classifier's performance. |
| first_indexed | 2024-03-13T06:23:47Z |
| format | Article |
| id | doaj.art-6f24f5f47ad14e68be7650bfb2749ce6 |
| institution | Directory Open Access Journal |
| issn | 2305-7254 2343-0737 |
| language | English |
| last_indexed | 2024-03-13T06:23:47Z |
| publishDate | 2023-05-01 |
| publisher | FRUCT |
| record_format | Article |
| series | Proceedings of the XXth Conference of Open Innovations Association FRUCT |
| spelling | doaj.art-6f24f5f47ad14e68be7650bfb2749ce62023-06-09T11:41:51ZengFRUCTProceedings of the XXth Conference of Open Innovations Association FRUCT2305-72542343-07372023-05-0133114815410.23919/FRUCT58615.2023.10142992Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline MethodsMaksim Kosterin0Ilya Paramonov1Nadezhda Lagutina2P.G. Demidov Yaroslavl State UniversityP.G. Demidov Yaroslavl State UniversityP.G. Demidov Yaroslavl State UniversityThe paper describes experiments performed on two sets of manually annotated data. The task of irony and sarcasm detection in Russian sentences was solved using baseline classifiers, i. e., BERT, Bi-LSTM, SVM, Random Forest, Logistic Regression. The best achieved F1-score for each classifier was 0.76, 0.73, 0.66, 0.64, 0.68 respectively. The results achieved by BERT and Bi-LSTM classifiers are comparable with the results from the articles describing the application of similar approaches for English language. Analysis of the results allowed to conclude that transferring the word context improves classification metrics and refinement of training data allows to improve the classifier's performance.https://www.fruct.org/publications/volume-33/fruct33/files/Kos.pdfsarcasm detectionnlptext classification |
| spellingShingle | Maksim Kosterin Ilya Paramonov Nadezhda Lagutina Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods Proceedings of the XXth Conference of Open Innovations Association FRUCT sarcasm detection nlp text classification |
| title | Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods |
| title_full | Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods |
| title_fullStr | Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods |
| title_full_unstemmed | Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods |
| title_short | Automatic Irony and Sarcasm Detection in Russian Sentences: Baseline Methods |
| title_sort | automatic irony and sarcasm detection in russian sentences baseline methods |
| topic | sarcasm detection nlp text classification |
| url | https://www.fruct.org/publications/volume-33/fruct33/files/Kos.pdf |
| work_keys_str_mv | AT maksimkosterin automaticironyandsarcasmdetectioninrussiansentencesbaselinemethods AT ilyaparamonov automaticironyandsarcasmdetectioninrussiansentencesbaselinemethods AT nadezhdalagutina automaticironyandsarcasmdetectioninrussiansentencesbaselinemethods |