The role of automated evaluation techniques in online professional translator training
The rapid technologisation of translation has influenced the translation industry’s direction towards machine translation, post-editing, subtitling services and video content translation. Besides, the pandemic situation associated with COVID-19 has rapidly increased the transfer of business and educ...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2021-10-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-706.pdf |
| _version_ | 1818399644277800960 |
|---|---|
| author | Dasa Munkova Michal Munk Ľubomír Benko Petr Hajek |
| author_facet | Dasa Munkova Michal Munk Ľubomír Benko Petr Hajek |
| author_sort | Dasa Munkova |
| collection | DOAJ |
| description | The rapid technologisation of translation has influenced the translation industry’s direction towards machine translation, post-editing, subtitling services and video content translation. Besides, the pandemic situation associated with COVID-19 has rapidly increased the transfer of business and education to the virtual world. This situation has motivated us not only to look for new approaches to online translator training, which requires a different method than learning foreign languages but in particular to look for new approaches to assess translator performance within online educational environments. Translation quality assessment is a key task, as the concept of quality is closely linked to the concept of optimization. Automatic metrics are very good indicators of quality, but they do not provide sufficient and detailed linguistic information about translations or post-edited machine translations. However, using their residuals, we can identify the segments with the largest distances between the post-edited machine translations and machine translations, which allow us to focus on a more detailed textual analysis of suspicious segments. We introduce a unique online teaching and learning system, which is specifically “tailored” for online translators’ training and subsequently we focus on a new approach to assess translators’ competences using evaluation techniques—the metrics of automatic evaluation and their residuals. We show that the residuals of the metrics of accuracy (BLEU_n) and error rate (PER, WER, TER, CDER, and HTER) for machine translation post-editing are valid for translator assessment. Using the residuals of the metrics of accuracy and error rate, we can identify errors in post-editing (critical, major, and minor) and subsequently utilize them in more detailed linguistic analysis. |
| first_indexed | 2024-12-14T07:23:57Z |
| format | Article |
| id | doaj.art-48bc9bfc117a40c28f512fb5ea2e3e0f |
| institution | Directory Open Access Journal |
| issn | 2376-5992 |
| language | English |
| last_indexed | 2024-12-14T07:23:57Z |
| publishDate | 2021-10-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj.art-48bc9bfc117a40c28f512fb5ea2e3e0f2022-12-21T23:11:33ZengPeerJ Inc.PeerJ Computer Science2376-59922021-10-017e70610.7717/peerj-cs.706The role of automated evaluation techniques in online professional translator trainingDasa Munkova0Michal Munk1Ľubomír Benko2Petr Hajek3Department of Translation Studies, Constantine the Philosopher University in Nitra, Nitra, SlovakiaDepartment of Computer Science, Constantine the Philosopher University in Nitra, Nitra, SlovakiaDepartment of Computer Science, Constantine the Philosopher University in Nitra, Nitra, SlovakiaDepartment of Computer Science, Constantine the Philosopher University in Nitra, Nitra, SlovakiaThe rapid technologisation of translation has influenced the translation industry’s direction towards machine translation, post-editing, subtitling services and video content translation. Besides, the pandemic situation associated with COVID-19 has rapidly increased the transfer of business and education to the virtual world. This situation has motivated us not only to look for new approaches to online translator training, which requires a different method than learning foreign languages but in particular to look for new approaches to assess translator performance within online educational environments. Translation quality assessment is a key task, as the concept of quality is closely linked to the concept of optimization. Automatic metrics are very good indicators of quality, but they do not provide sufficient and detailed linguistic information about translations or post-edited machine translations. However, using their residuals, we can identify the segments with the largest distances between the post-edited machine translations and machine translations, which allow us to focus on a more detailed textual analysis of suspicious segments. We introduce a unique online teaching and learning system, which is specifically “tailored” for online translators’ training and subsequently we focus on a new approach to assess translators’ competences using evaluation techniques—the metrics of automatic evaluation and their residuals. We show that the residuals of the metrics of accuracy (BLEU_n) and error rate (PER, WER, TER, CDER, and HTER) for machine translation post-editing are valid for translator assessment. Using the residuals of the metrics of accuracy and error rate, we can identify errors in post-editing (critical, major, and minor) and subsequently utilize them in more detailed linguistic analysis.https://peerj.com/articles/cs-706.pdfOnline educationAutomatic MT metricsResidualsTranslator trainingPost-editingFormative assessment |
| spellingShingle | Dasa Munkova Michal Munk Ľubomír Benko Petr Hajek The role of automated evaluation techniques in online professional translator training PeerJ Computer Science Online education Automatic MT metrics Residuals Translator training Post-editing Formative assessment |
| title | The role of automated evaluation techniques in online professional translator training |
| title_full | The role of automated evaluation techniques in online professional translator training |
| title_fullStr | The role of automated evaluation techniques in online professional translator training |
| title_full_unstemmed | The role of automated evaluation techniques in online professional translator training |
| title_short | The role of automated evaluation techniques in online professional translator training |
| title_sort | role of automated evaluation techniques in online professional translator training |
| topic | Online education Automatic MT metrics Residuals Translator training Post-editing Formative assessment |
| url | https://peerj.com/articles/cs-706.pdf |
| work_keys_str_mv | AT dasamunkova theroleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT michalmunk theroleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT lubomirbenko theroleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT petrhajek theroleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT dasamunkova roleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT michalmunk roleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT lubomirbenko roleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining AT petrhajek roleofautomatedevaluationtechniquesinonlineprofessionaltranslatortraining |