Building generalizable models for discourse phenomena evaluation and machine translation
The neural revolution in machine translation has made it easier to model larger contexts beyond the sentence-level, which can potentially help resolve some discourse-level ambiguities and enable better translations. Despite increasing instances of machine translation systems including contextual...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Doctor of Philosophy |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/165027 |
| _version_ | 1826125519572172800 |
|---|---|
| author | Jwalapuram, Prathyusha |
| author2 | Joty Shafiq Rayhan |
| author_facet | Joty Shafiq Rayhan Jwalapuram, Prathyusha |
| author_sort | Jwalapuram, Prathyusha |
| collection | NTU |
| description | The neural revolution in machine translation has made it easier to model larger contexts beyond
the sentence-level, which can potentially help resolve some discourse-level ambiguities and
enable better translations. Despite increasing instances of machine translation systems
including contextual information, the evidence for translation quality improvement is sparse,
especially for discourse phenomena. Most of these phenomena go virtually unnoticed by
traditional automatic evaluation measures such as BLEU. This work presents testsets and
evaluation measures for four discourse phenomena: anaphora, lexical consistency, discourse
connectives, and coherence, and highlights the need for performing such fine-grained
evaluation. We present benchmarking results for several context-aware machine translation
models using these testsets and evaluation measures, showing that the performance is not
always consistent across languages. We also present a targeted fine-tuning strategy which
improves pronoun translations by leveraging errors in already seen training data and additional
losses, instead of building specialized architectures that do not generalize across languages. |
| first_indexed | 2024-10-01T06:38:00Z |
| format | Thesis-Doctor of Philosophy |
| id | ntu-10356/165027 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:38:00Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1650272023-04-04T02:58:00Z Building generalizable models for discourse phenomena evaluation and machine translation Jwalapuram, Prathyusha Joty Shafiq Rayhan School of Computer Science and Engineering srjoty@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Document and text processing Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence The neural revolution in machine translation has made it easier to model larger contexts beyond the sentence-level, which can potentially help resolve some discourse-level ambiguities and enable better translations. Despite increasing instances of machine translation systems including contextual information, the evidence for translation quality improvement is sparse, especially for discourse phenomena. Most of these phenomena go virtually unnoticed by traditional automatic evaluation measures such as BLEU. This work presents testsets and evaluation measures for four discourse phenomena: anaphora, lexical consistency, discourse connectives, and coherence, and highlights the need for performing such fine-grained evaluation. We present benchmarking results for several context-aware machine translation models using these testsets and evaluation measures, showing that the performance is not always consistent across languages. We also present a targeted fine-tuning strategy which improves pronoun translations by leveraging errors in already seen training data and additional losses, instead of building specialized architectures that do not generalize across languages. Doctor of Philosophy 2023-03-08T04:37:38Z 2023-03-08T04:37:38Z 2022 Thesis-Doctor of Philosophy Jwalapuram, P. (2022). Building generalizable models for discourse phenomena evaluation and machine translation. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/165027 https://hdl.handle.net/10356/165027 10.32657/10356/165027 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Document and text processing Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Jwalapuram, Prathyusha Building generalizable models for discourse phenomena evaluation and machine translation |
| title | Building generalizable models for discourse phenomena evaluation and machine translation |
| title_full | Building generalizable models for discourse phenomena evaluation and machine translation |
| title_fullStr | Building generalizable models for discourse phenomena evaluation and machine translation |
| title_full_unstemmed | Building generalizable models for discourse phenomena evaluation and machine translation |
| title_short | Building generalizable models for discourse phenomena evaluation and machine translation |
| title_sort | building generalizable models for discourse phenomena evaluation and machine translation |
| topic | Engineering::Computer science and engineering::Computing methodologies::Document and text processing Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence |
| url | https://hdl.handle.net/10356/165027 |
| work_keys_str_mv | AT jwalapuramprathyusha buildinggeneralizablemodelsfordiscoursephenomenaevaluationandmachinetranslation |