An Analysis of ChatGPT’s Language Translation Based on the Korean Film Minari

Article information

J Eng Teach Movie Media. 2023;24(4):1-14
Publication date (electronic) : 2023 November 30
doi : https://doi.org/10.16875/stem.2023.24.4.1
1Assistant professor, Dept. of College of General Education, Kookmin University, 77 Jeongeung-ro, Seongbuk-gu, Seoul, 02707, Korea
Corresponding author, Assistant professor, Dept. of College of General Education, Kookmin University, 77 Jeongeung-ro, Seongbuk-gu, Seoul, 02707, Korea (E-mail: sungran@kookmin.ac.kr)
Received 2023 October 14; Revised 2023 November 15; Accepted 2023 November 25.

Abstract

The emergence of ChatGPT as an AI chatbot has marked significant progress in multiple-language translation. However, concerns persist regarding its capability to precisely translate various languages. This research aims to find a more effective method for improving the quality of Korean-to-English translation by using ChatGPT for EFL learners. To achieve this, EFL learners were asked to make questionnaires to assess the overall awareness and understanding of ChatGPT among their peers. Subsequently, they were instructed to select among Korean-to-English translations generated by ChatGPT and Korean-to-English translations done by human translators, providing reasons for their choices. The script of the film Minari (Lee, 2021) was collected and ChatGPT was used to translate it from Korean to English. These translations, both by the human translator and ChatGPT, were manually evaluated and a comparative analysis of the two translations was conducted. Finally, any errors in ChatGPT’s Korean-to-English translations were addressed by providing additional prompts to achieve the best possible translations. The result showed that overall translations were significantly enhanced by the prompts and demonstrated accuracy in translation. This finding demonstrates that EFL learners should make the most use of ChatGPT as a language learning and translation tool to improve their language communication skills.

Keywords: secondary; tertiary

I. INTRODUCTION

With the ongoing trend of globalization and rapid technological advancements, there is a rising fascination with intercultural and cross-linguistic interaction. Consequently, there is an increasing demand for translation between different languages, and automated translation systems have become indispensable in facilitating communication across languages in people’s daily routines. Machine Translation (MT), commonly known as automated language translation, involves the utilization of computer programs to convert sentences from one natural language to another. As a significant field of study within the area of Artificial Intelligence, MT has attracted considerable interest from both the academic and industrial sectors (Yang et al., 2020). According to Ali (2016), “machine translation may play a pivotal role in helping language experts in their daily work in general and in aiding non-professionals to understand and create text in target languages in particular” (p. 55). MT is a crucial position in the field of natural language processing, and its significance has gathered substantial interest in recent times (Yuan et al., 2023). Prior research has demonstrated that Large Language Models (LLMs) can enhance the comprehension of source text within translation systems, but encounter challenges in elevating generation capabilities (Guo et al., 2020; Liu et al., 2019). In this context, ChatGPT has showcased remarkable proficiency in both comprehending and generating natural language.

Given the rise of sophisticated Artificial Intelligence (AI) models like ChatGPT’s translation, the process of achieving precise language translation has become faster and more streamlined. The newly launched ChatGPT, a chatbot based on the Generative Pre-training Transformer (GPT) stands as a potent pre-trained language model created by OpenAI. While ChatGPT was not specifically designed for translation, it is quickly establishing itself as a dependable tool for language translation. ChatGPT translation utilizes deep learning methods to perform language translation across multiple languages. ChatGPT is among the systems that have demonstrated exceptional proficiency not only in single-language translation but also in the translation of multiple languages. Its capacity to understand and craft human-like responses across diverse contexts stands as evidence of its impressive performance.

MT has attracted increased attention and has been the subject of research and development over the past few decades. Nevertheless, it remains a significant challenge for both developers and users. While MT can convert texts, it struggles to capture the nuances and underlying implications. Likewise, even though ChatGPT is an exceptional translation tool, it is still a machine that can provide inaccurate translation. Its proficiency in precisely translating various languages is still being questioned. Banu (2023) mentions that translating multiple languages is a task that presents considerable challenges and complexities.

When dealing with single-language translation, a system is trained to achieve precise translations in a specific language. Conversely, for multilingual translation, the system needs to possess the ability to comprehend and handle the nuances in each language. Furthermore, when tackling the task of translating multiple languages, an additional layer of complexity arises concerning the accurate rendition of idiomatic expressions and phrases, which may hold distinct interpretations in the target language.

Furthermore, it’s worth noting that languages do not universally share the same or even similar sentence structures. Many instances arise where the word arrangement in the source language diverges from that of the target language. A clear illustration of this is the contrast between English and Korean. Korean belongs to the category of agglutinative languages, characterized by intricate morphological attributes like languages such as Turkish and Mongolian. While languages following the SVO structure such as English incorporate semantic particles into nouns and verb stems, agglutinative languages employ intricate chains of particles affixed to complex suffixes. These languages exhibit morphemes per word structures, which manifest numerous syntactic traits owing to sparse word-form patterns and variable sentence structures. As a result, the task of translation between languages has proven to be challenging for machine translation systems. Despite these challenges, machine translations remain valuable and effective tools, offering communication aids and translation assistance for unfamiliar foreign languages.

Human translation (HT) has been shaped by several key factors: the transfer of meaning between source and target languages, the influence of cultural context, and the unique capabilities of individual translators (Li et al., 2014). Chun (2019) mentions that MT will not present a threat or obstacle to human intelligence (HI) in the field of literary translation. This domain demands elevated language proficiency, ability in textual analysis, an understanding of subtext, context, and situation, as well as intercultural communicative competence, which are qualities that MT currently lacks. Interestingly, the precision rate of machine translation displays notable variations across different target languages. For instance, Western European languages exhibit accuracy rates of 90-95% (Lee et al., 2016). Conversely, the Korean language continues to demonstrate a relatively low accuracy rate, possibly attributed to inadequate data accumulation and cultural disparities (Lee & Cha, 2022).

The adoption of AI-powered language models like ChatGPT has the potential to revolutionize the field of translation. Since ChatGPT has a human-like interactive nature and an intelligent conversational system, the translation is generated into appropriate responses and modifications as requested by prompt and context. No matter how many times you try to translate something using MT such as Google Translate, the translation it gives will remain consistent. This is because Google Translate lacks the ability to identify contextual subtleties. However, ChatGPT tries to offer the most precise translation based on the context provided. On the other hand, ChatGPT can make it closer to human-like communication. This quality makes ChatGPT an invaluable resource in seeking precise and reliable translations.

Furthermore, it’s worth acknowledging that AI-powered models such as ChatGPT hold the potential to aid in foreign language learners (Yan, 2023). Fate (2023) states that in the Google Translator (GT) system, an artificial neural network (ANN) has replaced conventional digital lexicographic dictionaries. This transition has led researchers to recommend the use of Google Translate, which employs AI, in an English as a Foreign Language (EFL) course to enhance students’ syntax and grammar skills (Cancino & Panes, 2021) This involves assimilating data, encoding it, placing it within a contextual framework, and subsequently re-encoding it for output (King, 2019).

EFL learners can engage in conversations with ChatGPT in English, allowing them to simulate situations that are closer to real-life scenarios and improve their communication skills. Furthermore, if the AI learning platform is intricately designed, it can generate conversations of varying levels of difficulty based on individuals’ prior knowledge, English proficiency, and level of interest. Through such conversation generation and practice, they can enhance their English conversational skills and acquire natural expressions, vocabulary, grammar, and more that are essential for this purpose.

This article will discuss the competence of human-like ChatGPT translation when it comes to Korean-English translation. It will also examine the differences between human interpretation (HI) and machine translation using ChatGPT and discuss both its advantages and limitations in this context. To do this, as a first step, a questionnaire was given to identify Participants’ overall awareness and knowledge of ChatGPT. Then, they were instructed to select among Korean-to-English translations generated by ChatGPT and Korean-to-English translations done by human translators, and provide an explanation of their choice. The next step was collecting the script of the film Minari (Lee, 2020) that was translated by ChatGPT and scripts by a human translator. These two versions of translations were manually scored by two Korean Americans who have substantial expertise and experience in English translation. Lastly, to enhance the accuracy of ChatGPT’s Korean-to-English translations, we resolved any mistakes by introducing supplementary prompts, striving for the most accurate translations possible.

II. LITERATURE REVIEW

1. Machine Translation and AI Translation

MT, referred to as AI translation, constitutes a subset within the area of computational linguistics. This uses software to render text or speech into diverse languages. It empowers computers to create translations autonomously, outdoing the speed at which humans can achieve the same task. The level of its technological advancement is remarkable, attributed to AI’s deep learning, natural machine translation (NMT), and algorithmic development. In recent times, AI-powered machine translators have achieved significant progress, becoming extensively utilized by numerous people globally due to their widespread accessibility. The improvements in artificial translation technology have not ably heightened the precision of machine translation, rendering it more readily accessible and economically viable (Park & Kim, 2021).

Pokomy (2023) states that MT provides five advantages: speed, cost, accessibility, consistency, and round-the-clock availability. Machines possess the ability to perform text translation more rapidly than humans, making it attractive by quickly translating extensive amounts of text within a brief period. Utilizing machine translation generally incurs lower expenses compared to human translation. Easily accessible online machine translation tools such as Google Translate and Microsoft Translator also enable anyone around the globe to translate text rapidly and effortlessly. In addition, due to their systematic operational approach, machine translation tools can consistently translate text according to specific rules. Since machines don’t require rest, machine translation services are accessible for utilization around the clock.

Nevertheless, since the beginning of MT, challenges pertaining to both syntactic and semantic aspects have been faced. Stankevičiūtė et al. (2017) state, “despite progress in the development of MT, its systems still fail to recognize which synonym, collocation or word meaning should be used” (p. 75). Given that translation errors continue to be common, users face challenges in fully relying on and utilizing machine translators. Accordingly, machine translation stands as a promising field within artificial intelligence, requiring further advancement and refinement.

Moulieswaran and Prasantha (2023) state AI-driven materials and activities can aid Participants in enhancing their language proficiency. Ribeiro (2020) states that AI in English Language Teaching (ELT) is the most realistic way English language. Thus, the reform of English teaching and learning can be effectively promoted through AI learning, intelligent search, and natural language processing (Wang, 2019). Ribeiro (2020) asserts that incorporating AI into ELT represents a practical and feasible approach for English language educators. Given the systematic grammatical structure of English, the learning process has historically posed significant challenges for Participants (Mehrotra, 2019). Therefore, the enhancement of English teaching and learning can be effectively facilitated through the implementation of AI (Wang, 2019).

The reason Korean-English machine translation’s effectiveness remains somewhat limited is due to the contrasting syntactic and morphological aspects of the English and Korean languages. Furthermore, the cultural distinctions inherent to these languages, along with the extent and diversity of data utilized by language algorithms, also contribute to this quality gap. Primarily, MT tends to render passages in a literal and surface-level manner, while human intelligence (HI) can employ the interpreter’s flexibility and creativity to enhance the reader/listener’s understanding within the given context. Because of this, post-editing is regarded as inevitable since a machine translation system needs to handle flexible sentence structures, a wide range of words, and various topics across diverse types of communication (Vasconcellos & Marjorie, 1988). At this point, OpenAI’s powerful language model, ChatGPT, utilizes deep learning techniques to analyze and generate natural language text. Having trained on a massive volume of data, ChatGPT has acquired the ability to comprehend and generate human-like text with remarkable precision.

2. ChatGPT as a Language Translation Tool

ChatGPT developed by OpenAI has undergone training on an extensive dataset, allowing it to comprehend and generate human-like language communication with impressive precision. Thus, it offers significant value for a wide variety of natural language processing (NLP) tasks. Fate (2023) argues that the adaptability of ChatGPT as a translator across various language combinations, including English and Arabic, has rendered it a precious tool for educational purposes, cross-cultural dialogues, and fostering mutual comprehension.

One of its most prominent characteristics is to make conversational learning possible. Since ChatGPT’s primary function is to serve as an intelligent conversational system, it possesses the ability to undertake a range of human-like tasks, including machine translation (Gao et al., 2023).

Pokomy (2023) argues the nature of ChatGPT allows it a helpful language translation tool such as contextual understanding, multifunctionality, interactivity, and informal and slang translation. Since ChatGPT is trained to produce text that sounds human-like, it might excel in delivering translations with finer nuances, especially in scenarios involving informal or conversational language. Traditional machine translation systems that focus on literal translation may struggle to capture the original sentiment or tone of such source text in their translations as adeptly.

Aside from translation, ChatGPT can undertake various language-related functions as well. For instance, you can request it to translate a sentence and subsequently pose a follow-up question related to the translation’s content. Besides, ChatGPT can interact with you while you engage in your translation tasks. For example, if you find a translation unsatisfactory or require an explanation, you can engage in a back-and-forth dialogue with ChatGPT to improve and explain its translations. Considering the extensive training ChatGPT has received on a wide array of internet text, it might have translation capabilities for rendering informal language, internet slang, and idiomatic expressions, surpassing the performance of traditional machine translation systems in these aspects.

Nevertheless, the limitations of ChatGPT have still been studied to its current capabilities. ChatGPT stands apart from conventional machine translation tools. Interestingly, it wasn’t primarily developed for translation purposes. Jiao et al. (2023) argue that ChatGPT displays a noticeable disparity in performance compared to other established commercial translation systems like Google Translate and DeepL Translate. This gap becomes even more noticeable when dealing with languages that have limited resources.

Therefore, the translation capabilities of ChatGPT by devising distinct translation prompts should be introduced (Gao et al., 2023). Timothy (2023) argues that the text to be translated should be specified and point out six methods to deal with ChatGPT as a language translation tool: provide context, declare the type of text, use style transfer, account for regional differences, use summarized translation and use a fine-tuned instance of ChatGPT.

III. METHODOLOGY

Turing (1950) introduced the idea of having a human evaluator assess conversations in natural language between a human and a machine that’s engineered to produce responses resembling those of a human. The evaluator would be informed that one of the conversational partners is a machine, and all participants would be physically separated. If the evaluator couldn’t consistently distinguish between the machine and the human, the machine would be considered to have successfully passed the test. The outcome of the test would not be determined by the machine’s accuracy in providing correct answers to questions but rather by how closely its responses resembled those that a human would give. Base on Turning (1950), this research begins with a preliminary investigation to determine whether Participants can differentiate between translations by ChatGPT and those by human translators. Participants (male: n = 26, female: n = 8) were freshman to junior students enrolled in English prerequisite courses in Seoul, Korea. They have different majors such as computer engineering, business administration, and music, etc. The first stage investigates the Participants’ opinion on the use of ChatGPT in language translation in a prior questionnaire. The next stage is to present a set of 25 sentences generated by human translators to Participants alongside sentences written by ChatGPT, have them choose which one is written by ChatGPT, and the justification for why they selected them. According to their opinion, the findings are collected and analyzed. This stage is to identify their recognition and understanding of the translation by ChatGPT. The questionnaire was designed in a descriptive format to allow participants to comfortably express their opinions in their native language, Korean.

In this study, the corpus data did not involve an original English-language movie but instead used a Korean-to-English translated film, Minari. The movie Minari is a drama film made in 2020, and it achieved significant success at the 93rd Academy Awards by winning six of the most prestigious awards: Best Picture, Best Director, Best Original Score, Best Original Screenplay, Best Actor, and Best Supporting Actress. The movie runs for around 1 hour and 55 minutes. Korean script from the movie Minari was gathered. The total number of sentences is 1031, which includes only sentences. and one or two more words such as yes/no, right, okay, and so on are exempt from the data. The sentences were translated into English using ChatGPT and the data was gathered and evaluated by a native English speaker and researchers for reliability. While manual evaluation of MT results has the drawback of being a subjective and time-consuming process, it possesses the advantage of offering a precise assessment of MT outputs and the ability to identify error issues, as opposed to automatic evaluation methods.

In the final phase, building on the above observations, the data were categorized into two main groups: general language translation errors in ChatGPT, encompassing issues related to context information, cultural references, nuances, slang, and idiomatic expressions; and frequent errors specific to Korean MT outputs, including homonyms, subject omission, and third-person subject representation of first-person subjects. The classified data were regenerated by applying immediate feedback and prompts through back-and-forth interaction. This stage is to demonstrate the improvement and development of ChatGPT’s language translation using repetitive conversation like humans.

IV. RESULTS AND DISCUSSION

1. Prior Questionnaire

As a prior study, a questionnaire was given to identify participants’ awareness and knowledge of ChatGPT. The questionnaire was composed of the following: (a) Have you ever used ChatGPT? (b) If you have ever used ChatGPT for language translation, which way have you used between Korean to English and English to Korean? (c) What is the most commonly used for English translation?

As shown in Figure 1, participants have experience in using ChatGPT over 90% which means they are interested in ChatGPT and conscious of the importance of ChatGPT these days whether or not in relation to language translation.

Fig. 1.

The Experiences in Using ChatGPT

The results of the second questionnaire indicate that participants do not often use ChatGPT as a means of language translation over the half of them, especially from English to Korean translation. This implies that they think ChatGPT is not the only way to translate language and they are not accustomed to using it as a translation tool (see Figure 2).

Fig. 2.

Language Translation Between Korean to English and English to Korean

As shown in Figure 3, the findings of the third questionnaire show that participants the most commonly used Papago at 88.2% for English translation. Papago, developed by the prominent South Korean IT company Naver Corporation, is a multilingual machine translation system that uses AI-driven neural technology to learn from its errors (Koh, 2022). It means Korean participants are used to using a Korean native tool. Google Translate by Google is the next commonly used tool, which is prominent in Korea and offers translations into numerous languages, relying on neural network technology. It is followed by ChatGPT. ChatGPT is recently developed and participants do not seem to have a lot of experience to use it. Therefore, it is clear that participants are in great need of learning how to use ChatGPT effectively and efficiently.

Fig. 3.

Language Translation Used the Most Commonly

2. The Translation Sentences Awareness Generated by ChatGPT

Participants are given a set of 25 sentences, some composed by human translators and others by ChatGPT. They are asked to identify the sentences written by ChatGPT and provide reasons for their choices. The study collects and analyzes the findings based on their explanations and opinions.

As shown in Table 1, participants more than 50% distinguish the output of ChatGPT from that of human translation. Even though they mentioned they did not utilize ChatGPT as the means of language translation, they were relatively able to tell the difference between them. The most interesting is that the longer the Korean sentence is, the higher the distinction ratio of the ChatGPT translation is. In Korean, “약 3주전에 설치된 이 이동식집은 채 고정을 안 시킨 상태였습니다” takes up the highest in distinction ratio by ChatGPT as one of the longest Korean sentences whereas “더 좋은 거 보여줄까?” as one of the shortest sentences is the lowest in ChatGPT translation. The result shows that ChatGPT excels at translating longer sentences rather than shorter ones (see Table 2).

Translation Distinction Ratio Between ChatGPT and Human Translation

Distinction Ratio Between ChatGPT and Human Translation

Participants’ opinions on ChatGPT were completely divided into positive and negative views of point. Some of them state that ChatGPT sounds to speak more like an English native speaker because it was trained and learned from English native speakers. On the contrary, others mention that ChatGPT is just like a machine translator, and it translates literally without capturing between lines and lacks context explanation and colloquial and idiomatic meaning (see Table 3).

The Positive and Negative Viewpoint in ChatGPT

3. Regenerating Translation Generated by ChatGPT

1) General Language Translation Errors of ChatGPT

Some prompts are used to improve the precision of translation generated by ChatGPT, employing sentences that were less accurate. It is categorized into five subcategories such as context information, cultural references, nuance, slang, and idiomatic expression.

(1) Context Information

In Korean “숫놈들” is used for all kinds of living things and “숫놈들” is translated as ‘male animals’. Considering context meaning, “숫놈들 means male chicks” is added and it is translated as “They are disposing of the male chicks over there,” similar to human translation “Male chicks are discarded there” (see Figure 4).

Fig. 4.

Context Information Prompt

(2) Cultural References

Although “화투” is a Korean gambling card game, it is not easy to recognize the word meaning without cultural background. ChatGPT translates it into a deck of cards and the meaning sounds obscure. Next prompt is put such as “화투 means gambling card game” and the meaning leads to obvious and clear (see Figure 5).

Fig. 5.

Cultural References Prompt

(3) Nuance

In Korean “저렇게 이쁜 새끼한테 무슨 회초리야” is explained two aspects of prompts such as cultural reference and nuance. “회초리” is translated by ChatGPT as sledgehammer, but it is traditionally used as a cane for educating kids. As a first step, “회초리” prompt is added and regenerated as “What kind of cane is that for such a pretty child?” The sentence is not a wh-question, which is a rhetorical question. Even so, ChatGPT as a kind of machine translation does not identify the context nuance and it is translated as “What kind of is that for such a pretty child?” Next prompt is added “회초리야 nuances there is no need for their kids” and the result shows as “Why would you need a cane like that for such a pretty child?” This conveys the nuance that it’s unnecessary to use such a cane on a child who looks so lovely. It is close meaning to human translation “Why would you hit such a sweet boy?” (see Figure 6).

Fig. 6.

Cultural References and Nuance Prompt

(4) Slang

“쌌지” in Korean has a few meanings such as “cheap, pack, wrap, or pee,” but in this situation, “쌌지” is a kind of slang. In the Korean gambling card game, it is used in a situation where the player seems to lose the game. ChatGPT could not notice the background and it is said, “There isn’t a direct one-word translation in English” at first. Then prompt for the slang is put in it such as “meaning someone is losing it in Korean gambling card game” and it is translated as “losing it” or “on a losing streak.” It conveys the idea that the person is not doing well in the game (see Figure 7).

Fig. 7.

Slang Prompt

(5) Idiomatic Expression

In Korean, “그럼 지하에 대고 들어 갈게” is at first translated into “Then I will enter holding it underground,” but “지하에 차를 대다” is a kind of idiomatic expression meaning “park a car in the underground parking lot.” As a next step, a prompt like “지하에 대다 means to park a car in the underground parking lot.” is put and it is translated into “Then I’ll go and park it in the underground parking lot” (see Figure 8).

Fig. 8.

Idiomatic Expression Prompt

2) Frequent Korean Errors of MT Output

Koh (2022) categorized the errors in Korean-to English translation into incorrect disambiguation (subject), wrong lexical choice, and missing words. Then, incorrect disambiguation (subject) is subcategorized into subject omission, third person subject representation of first-person subject, and indefinite question. Next, the wrong lexical choice is subcategorized into homonyms, the confusion of similar meaning or pronunciation, English used in Korean, missing words, and unknown words. Frequent Korean errors of MT output in this study are categorized into three groups: homonym, subject omission, third person subject representation of first-person subject. They are regenerated by ChatGPT, using additional prompts. Some detail points are depicted in the following.

(1) Homonym

In Korean, homonyms can lead to translation inaccuracy in machine translation. For example, the Korean word “밤” can refer to either “chestnut” or “night,” and this can result in incorrect translations, such as “It’s a chestnut” being mistakenly rendered as “It’s night” or ‘It’s nighttime” (Koh, 2022). Korean word “용” means “dragon” and “용도” can refer to either “deer antlers also” or “purpose.” In addition, it is a kind of cultural background where Koreans traditionally have the custom of eating deer antlers to boost their energy when they have low energy. Consequently, both the exact meaning “용도” and cultural reference are needed to put as prompts. The result is shown as “The deer antler is in, and everything is in,” which has a similar meaning to the human translation “It has everything, even deer antlers” (see Figure 9).

Fig. 9.

Homonym Prompt

(2) Subject Omission

In Korean, it’s common to omit subjects in sentences, whereas in English, subjects are typically specified in most sentences. This is why the most common errors in MT were identified in relation to this aspect in Korean. Koh (2022) found that subject omission ranks the highest errors among incorrect error classifications in MT. However, ChatGPT solved the problem by adding subjects as immediate feedback and improved the quality like human translation. “네 엄마 치마 뒤로 숨는 거야?” in Korean does not have subject. Then, as a next step, the subject “you” is put in ChatGPT and the result shows the exact meaning as shown in Figure 10.

Fig. 10.

Subject Omission Prompt

(3) Third Person Subject Representation of First-Person Subject

Korean frequently employs third-person subjects instead of first-person subjects, whereas English uses third-person verbs based on the subject when a third-person subject is used. As a result, translating from Korean to English can result in changes in meaning. Though the subject of “할머니가 좀 먹어 볼게” in Korean is “할머니(grandmother),” it is translated that subject is “I.” When the prompt subject is “할머니(grandmother)” is put in, the translation is improved (see Figure 11).

Fig. 11.

Third Person Subject Representation of First-Person Subject Prompt

V. CONCLUSION

As mentioned before, some research mentions that while ChatGPT is an outstanding translation tool, it’s important to remember that it’s a machine and may not always provide the absolute best translations. There are ongoing concerns about its ability to accurately translate different languages. It can have difficulties in fully understanding the context or in grasping nuances and subtle differences in the user’s questions. It also may provide inconsistent or inaccurate responses due to the difficulty in comprehending lengthy and complex conversations by referencing previous dialogues.

However, this study shows that the Korean-to-English translation ability of ChatGPT has improved and developed by certain prompts, an ability that is impossible for other machine translators to achieve when translating multiple languages. Other MTs repeat the same translation in the same sentences, but ChatGPT provides a variety of kinds of sentences by offering some prompts according to users’ needs. It has human-like conversation ability and as a result, it leads to accurate and precise translation like the human transitions. Still, even though it has advantages as a tool for foreign language translation, it takes time to have the best result that users want, and it should be considered as a point to be overcome.

The emergence of ChatGPT has generated significant reactions across the technology industry, academia, and society. While there is amazement at its technological advancements, it has also raised many negative educational influences such as its potential to facilitate academic dishonesty among students or hinder efforts to build abilities like creativity. Unsurprisingly, it causes a significant change in the foreign language learning in the classroom. It seems inevitable to change the objectives and methodologies of traditional foreign language learning. It could result in the reduction of the necessity of foreign language learning for the purpose of communication. For example, translation and conversation apps integrated into smart devices have been advancing and they now provide high-quality translation communicating with foreigners, without the need for actively learning a foreign language. In the not-so-distant future, the development of artificial intelligence may render it unnecessary for individuals to learn a foreign language unless they have specific goals.

In this present point, teachers hold a highly significant position as facilitators in foreign language instruction. It is essential for teachers to recognize that reducing the learners’ cognitive load not only captures their attention and stimulates their interest but also significantly amplifies their motivation to attain their ultimate objective of mastering English (Koh, 2020). In this age of technological advancement, the role of foreign language teachers and instructors is crucial in teaching how to use AI conversation tools like ChatGPT effectively and efficiently. Since it can interact with users, teachers and instructors should instruct how to provide proper prompts in foreign language translation and to have the best translation. Then it can be used in foreign language learning to foster language literacy and competence, and expand communication skills. Participants can engage in conversations with ChatGPT in a foreign language more freely and comfortably. They can also have ChatGPT assess their translations and then compare and correct errors. Additionally, ChatGPT can offer a wide variety of resources related to the target language, including grammar, vocabulary, sentence structures, and various information, making it easy for learners to utilize these resources in foreign language learning. Considering these possibilities and limitations of ChatGPT, it is suggested that the potential applications of ChatGPT as an AI chatbot for classroom instruction be explored and researched in a wide range of ways.

References

Ali A.. 2016;Exploring the problems of machine translation from Arabic into English language faced by Saudi university student of translation at the faculty of arts. Journal of Humanities and Social Science 21(4):55–66. https://doi.org/10.9790/0837-2104025566.
Banu, S. (2023, May 10). ChatGPT translation: Can it accurately translate multiple languages? AMBCrypto. https://ambcrypto.com/blog/chatgpt-translation-can-it-accurately-translate-multiple-languages/.
Cancino M., Panes J.. 2021;The impact of Google Translate on L2 writing quality measures: Evidence from Chilean EFL high school learners. System 98:102464. https://doi.org/10.1016/j.system.2021.102464.
Chun H.-J.. 2019;Comparative analysis of machine translation and human translation in literary and literary translation: Based on the analysis of the translation of “Childhood” by L. N. Tolstoy. Korean Journal of Russian Language and Literature 31(1):111–154.
Fate K.. 2023. ChatGPT for Arabic-English translation: Evaluating the accuracy Research Square. https://doi.org/10.21203/rs.3.rs-2814154/v2.
Gao Y., Wang R., Hou F.. 2023. How to design translation prompts for ChatGPT: An empirical study arXiv. https://doi.org/10.48550/arXiv.2304.02182.
Guo J., Zhang Z., Xu L., Wei H., Chen B., Chen E.. 2020;Incorporating BERT into parallel sequence decoding with adapters. Advances in Neural Information Processing Systems 33:10843–10854. https://doi.org/10.48550/arXiv.2010.06138.
Jiao W., Wang W., Huang J., Wang X., Tu Z.. 2023. Is Chatgpt a good translator? A preliminary study arXiv. https://doi.org/10.48550/arXiv.2301.08745.
King K.. 2019;Can Google translate be taught to translate literature? A case for humanists to collaborate in the future of machine translation. Translation Review 105(1):76–92. https://doi.org/10.1080/07374836.2019.1673268.
Koh S. R.. 2020;Initial formulaic sequence approach for EFL learners: Based on film corpus-based patterns. STEM Journal 21(4):1–17. https://doi.org/10.16875/stem.2020.21.4.1.
Koh S. R.. 2022;A comparative error analysis of neural machine translation output: Based on film corpus. The Linguistic Association of Korea Journal 30(1):157–177.
Lee, I. (Director). (2020). Minari [Film]. Plan B Entertainment.
Lee J.-H., Cha K.-W.. 2022;A study of error types in Korean-English translation from Korean spoken language to Papago. Modern English Education 23(1):56–65.
Lee N.-S., Lee S., Lee S.-J., Lee J.-Y., Lee J.-H.. 2016;Future horizons of translation & interpretation: Artificial intelligence and interactive-converged translation & interpretation. The Journal of Translation Studies 17(2):65–89. https://doi.org/10.15749/jts.2016.17.2.003.
Li H., Graesser A. C., Cai Z.. 2014. Comparison of Google translation with human translation. In W. Eberle & C. Boonthum-Denecke (Eds.), Proceedings of the 27th International Florida Artificial Intelligence Research Society Conference (pp. 21-23). AAAI.
Liu Z., Yan X., Genta I.-W., Pascale F.. 2019;Incorporating word and subword units in unsupervised machine translation using language model rescoring. Proceedings of the Fourth Conference on Machine Translation 2:275–282. https://doi.org/10.18653/v1/W19-5327.
Mehrotra D. D.. Basic of artificial intelligence & machine learning Notion Press.
Moulieswaran N., Prasantha K.. 2023;Investigating ESL learners’ perception and problem towards artificial intelligence (AI) - assisted English language learning and teaching. World Journal of English Language 13(5):290–298. https://doi.10.5430/wjel.v13n5p290.
Park K.-H., Kim J.-R.. 2021;Analysis of the usability of machine translators as an English learning tool: Through backtranslation of the as phrase. Journal of the Korea Contents Association 21(5):259–267. https://doi.org/10.5392/JKCA.2021.21.05.259.
Pokomy, E. (2023, July 26). ChatGPT vs translation services: A comparative study. WEGLOT. https://www.weglot.com/blog/chatgpt-translation.
Ribeiro, R. (2020, March 20). Artificial intelligence in English language learning. World of better learning. https://www.cambridge.org/elt/blog/2020/03/09/artificial-intelligence-english-language-learning/.
Stankevičiūtė G., Kasperaviciene R., Horbacauskiene J.. 2017;Issues in machine translation. International Journal on Language, Literature and Culture in Education 4(1):75–88. https://doi.org/10.1515/llce-2017-0005.
Timothy, M. (2023, August 13). How to use ChatGPT as a language translation tool. MakeUseOf. https://www.makeuseof.com/how-to-translate-with-chatgpt/.
Turing A. M.. 1950;Computing machinery and intelligence. Mind 59(236):433–460. https://doi.org/10.1093/mind/LIX.236.433.
Vasconcellos, M., & Marjorie, L. (1988). SPANAM and ENGSPAN: Machine translation at the pan American health organization. In J. Slocum (Ed.), Machine translation systems (pp. 187-235). Cambridge University Press.
Wang R.. 2019;Research on artificial intelligence promoting English learning change. Proceedings of the 3rd International Conference on Economics and Management, Education, Humanities and Social Sciences (EMEHSS 2019) https://doi.org/10.2991/emehss-19.2019.79.
Yan D.. 2023;Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Educationand Information Technologies 28:13943–13967. https://doi.org/10.1007/s10639-023-11742-4.
Yang S., Wang Y., Chu X.. 2020. A survey of deep learning techniques for neural machine translation arXiv. https://arxiv.org/abs/2002.07526.
Yuan G., Ruili W., Feng H.. 2023. How to design translation prompts for ChatGPT: An empirical study arXiv. https://doi.org/10.48550/arXiv.2304.02182.

Article information Continued

Fig. 1.

The Experiences in Using ChatGPT

Fig. 2.

Language Translation Between Korean to English and English to Korean

Fig. 3.

Language Translation Used the Most Commonly

Fig. 4.

Context Information Prompt

Fig. 5.

Cultural References Prompt

Fig. 6.

Cultural References and Nuance Prompt

Fig. 7.

Slang Prompt

Fig. 8.

Idiomatic Expression Prompt

Fig. 9.

Homonym Prompt

Fig. 10.

Subject Omission Prompt

Fig. 11.

Third Person Subject Representation of First-Person Subject Prompt

TABLE 1

Translation Distinction Ratio Between ChatGPT and Human Translation

Method Avg (%) SD
Human translation selection 43.30 7.60
ChatGPT translation selection 56.70

TABLE 2

Distinction Ratio Between ChatGPT and Human Translation

Korean Human translation % ChatGPT translation %
더 좋은 거 보여줄까? Want to see something even better? 58.8 Shall I show you something better? 41.2
아무 집도 안보이는데? I don’t see any houses. 50 I can’t see any houses at all. 50
할머니 사진은 어디다 둬요? Where should Grandma’s picture go? 50 Where should I put my grandmother’s picture? 50
오늘 첫날이니까 다같이 바닥에서 자자. Let’s sleep together on the floor for our first night. 32.1 Since today is the first day, let’s all sleep on the floor together. 67.6
상황이 좋으면 함께 사는 거고 아니면 헤어지는 거야? We can live together when things are good, but when they’re not, we fall apart? 32.4 If the situation is good, we stay together, but if not, we separate? 67.6
약 3주전에 설치된 이 이동식집은 채 고정을 안 시킨 상태였습니다. Well, the mobile home was set, in approximately three weeks ago, and the owner had yet to tie it down. 29.4 This portable house, installed approximately three weeks ago, remained unfixed. 70.6

TABLE 3

The Positive and Negative Viewpoint in ChatGPT

Positive Negative
I think ChatGPT would have translated better. It is translated literally.
I think ChatGPT seems to speak like a real native speaker after learning from the native speakers’ language. It appears to be a direct translation of a pronoun or lacking a context explanation.
I chose it because it’s more colloquial. It’s too long, and there are many formal expressions.
ChatGPT made a natural sentence expression that can be used in real life. I think it will translate words literally rather than capturing the underlying meaning within the sentence.
When translated with ChatGPT, I received a strong sense of colloquial style rather than rigid grammar. The grammar is correct, but it sounds formal and lacks colloquial language. It’s like speaking as if following a grammar book.
I chose expressions that are commonly used because it is based on things that actual people have done. Human translation sounds more natural and makes greater use of idiomatic expressions within the context.
I chose something that’s less awkward. It seems to use words literally and appears to have fewer interjections or expressions of amazement (perhaps a difference in tone?).
The context flows naturally, and it incorporates more idiomatic expressions and colloquialisms. I selected the sentence that gives the feeling of a literal translation from Korean.
Other translation tools feel like they translate everything exactly as it is, while ChatGPT gives a sense of naturalness by blending in a bit of paraphrasing. It felt like a direct word-for-word translation, like what ChatGPT would do, and there were some idiomatic expressions that I didn’t expect, which made me think those were from the translator.