Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate
Purpose. The advent of Large Language Model (LLM), a generative artificial intelligence (AI) model, in November 2022 has had a profound impact on various domains, including the field of translation studies. This motivated this study to conduct a rigorous evaluation of the effectiveness and precision...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Pereiaslav-Khmelnytsky Hryhorii Skovoroda State Pedagogical University
2024-04-01
|
| Series: | Психолінгвістика |
| Subjects: | |
| Online Access: | https://psycholing-journal.com/index.php/journal/article/view/1473 |
| _version_ | 1797206848584024064 |
|---|---|
| author | Мохаммед Мохсен |
| author_facet | Мохаммед Мохсен |
| author_sort | Мохаммед Мохсен |
| collection | DOAJ |
| description | Purpose. The advent of Large Language Model (LLM), a generative artificial intelligence (AI) model, in November 2022 has had a profound impact on various domains, including the field of translation studies. This motivated this study to conduct a rigorous evaluation of the effectiveness and precision of machine translation, represented by Google Translate (GT), in comparison to Large Language Models (LLMs), specifically ChatGPT 3.5 and 4, when translating academic abstracts bidirectionally between English and Arabic.
Methods. Employing a mixed-design approach, this study utilizes a corpus comprising 20 abstracts sourced from peer-reviewed journals indexed in the Clarivate Web of Science, specifically the Journal of Arabic Literature and Al-Istihlal Journal. The abstracts are equally divided to represent both English-Arabic and Arabic-English translation directionality. The study’s design is rooted in a comprehensive evaluation rubric adapted from Hurtado Albir and Taylor (2015), focusing on semantic integrity, syntactic coherence, and technical adequacy. Three independent raters carried out assessments of the translation outputs generated by both GT and LLM models.
Results. Results from quantitative and qualitative analyses indicated that LLM tools significantly outperformed MT outputs in both Arabic and English translation directions. Additionally, ChatGPT 4 demonstrated a significant advantage over ChatGPT 3.5 in Arabic-English translation, while no statistically significant difference was observed in the English-Arabic translation directionality. Qualitative analysis findings indicated that AI tools exhibited the capacity to comprehend contextual nuances, recognize city names, and adapt to the target language's style. Conversely, GT displayed limitations in handling specific contextual aspects and often provided literal translations for certain terms. |
| first_indexed | 2024-04-24T09:13:32Z |
| format | Article |
| id | doaj.art-2f9b48ed36f34eb2a9d32cc5fbcd983c |
| institution | Directory Open Access Journal |
| issn | 2309-1797 2415-3397 |
| language | English |
| last_indexed | 2024-04-24T09:13:32Z |
| publishDate | 2024-04-01 |
| publisher | Pereiaslav-Khmelnytsky Hryhorii Skovoroda State Pedagogical University |
| record_format | Article |
| series | Психолінгвістика |
| spelling | doaj.art-2f9b48ed36f34eb2a9d32cc5fbcd983c2024-04-15T17:18:09ZengPereiaslav-Khmelnytsky Hryhorii Skovoroda State Pedagogical UniversityПсихолінгвістика2309-17972415-33972024-04-0135213415610.31470/2309-1797-2024-35-2-134-1561473Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google TranslateМохаммед Мохсен0Najran University (Saudi Arabia)Purpose. The advent of Large Language Model (LLM), a generative artificial intelligence (AI) model, in November 2022 has had a profound impact on various domains, including the field of translation studies. This motivated this study to conduct a rigorous evaluation of the effectiveness and precision of machine translation, represented by Google Translate (GT), in comparison to Large Language Models (LLMs), specifically ChatGPT 3.5 and 4, when translating academic abstracts bidirectionally between English and Arabic. Methods. Employing a mixed-design approach, this study utilizes a corpus comprising 20 abstracts sourced from peer-reviewed journals indexed in the Clarivate Web of Science, specifically the Journal of Arabic Literature and Al-Istihlal Journal. The abstracts are equally divided to represent both English-Arabic and Arabic-English translation directionality. The study’s design is rooted in a comprehensive evaluation rubric adapted from Hurtado Albir and Taylor (2015), focusing on semantic integrity, syntactic coherence, and technical adequacy. Three independent raters carried out assessments of the translation outputs generated by both GT and LLM models. Results. Results from quantitative and qualitative analyses indicated that LLM tools significantly outperformed MT outputs in both Arabic and English translation directions. Additionally, ChatGPT 4 demonstrated a significant advantage over ChatGPT 3.5 in Arabic-English translation, while no statistically significant difference was observed in the English-Arabic translation directionality. Qualitative analysis findings indicated that AI tools exhibited the capacity to comprehend contextual nuances, recognize city names, and adapt to the target language's style. Conversely, GT displayed limitations in handling specific contextual aspects and often provided literal translations for certain terms.https://psycholing-journal.com/index.php/journal/article/view/1473chatgpt, machine translation, google translate, articles’ abstract. |
| spellingShingle | Мохаммед Мохсен Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate Психолінгвістика chatgpt, machine translation, google translate, articles’ abstract. |
| title | Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate |
| title_full | Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate |
| title_fullStr | Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate |
| title_full_unstemmed | Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate |
| title_short | Artificial Intelligence in Academic Translation: A Comparative Study of Large Language Models and Google Translate |
| title_sort | artificial intelligence in academic translation a comparative study of large language models and google translate |
| topic | chatgpt, machine translation, google translate, articles’ abstract. |
| url | https://psycholing-journal.com/index.php/journal/article/view/1473 |
| work_keys_str_mv | AT mohammedmohsen artificialintelligenceinacademictranslationacomparativestudyoflargelanguagemodelsandgoogletranslate |