doi: 10.56294/mr202338

REVIEW

Metaverse and translation studies: analysis of machine translation

Análisis de la traducción automática en los estudios sobre metaverso

Rubén González Vallejo¹ *

¹Universidad de Salamanca, Departamento de Filología. Salamanca, España.

Cite as: González Vallejo R. Metaverse and translation studies: analysis of machine translation. Metaverse Basic and Applied Research. 2022; 2:38. https://doi.org/10.56294/mr202338

Submitted: 01-03-2023                           Revised: 09-05-2023                               Accepted: 28-07-2023                         Published: 29-07-2023

Editor: PhD. Dra. Yailen Martínez Jiménez

ABSTRACT

Although translation is essential, in collaboration with artificial intelligence, for the development and success of interaction and communication in the metaverse, little attention has been paid to the direct influence of linguistic communication in three-dimensional space. For this reason, in this paper, we have analyzed different investigations focused on the metaverse, coming from various disciplines using the machine translation service. The purpose is to know the perceptions about its use and quality and thus understand its potential in virtual reality and challenges. Documents from the SciELO, Google Scholar, and Dialnet databases were retrieved. Those studies that have directly or indirectly used the machine translation tool to favor communication and interaction between users were included. The results show a positive general perception by the studies analyzed, although the imprecision of the automatic translation engines and the lack of language combinations are qualified. Lastly, the importance of studying communication in the metaverse is highlighted, opening up the number of available languages and researching the training of translation engines since the income from the machine translation market is destined to increase in the coming years because of the pace of manufacturing and the productivity offered.

Keywords: Metaverse; Neural Machine Translation; Quality.

RESUMEN

Si bien la traducción sea fundamental, en colaboración con la inteligencia artificial, para el desarrollo y el éxito de la interacción y la comunicación en el metaverso, se ha dedicado poca atención a la influencia directa que tiene la comunicación lingüística en el espacio tridimensional. Por ello, en el presente trabajo se han analizado diferentes investigaciones enfocadas en el metaverso, procedentes de varias disciplinas, que han utilizado el servicio de traducción automática. El fin es conocer las percepciones sobre su uso y calidad, y así entender cuáles serían las potencialidades que tendría en la realidad virtual y los retos que plantea. Se recuperaron documentos de las bases de datos SciELO, Google Académico y Dialnet, y se incluyeron aquellos estudios que han usado directa o indirectamente la herramienta de traducción automática para favorecer la comunicación e interacción entre los usuarios. Los resultados arrojan una percepción general positiva por parte de los estudios analizados, si bien se matizan la imprecisión de los motores de traducción automática y la carencia de combinaciones lingüísticas. Por último, se resalta la importancia de estudiar en el metaverso la comunicación, abrir el número de idiomas disponibles e investigar en el entrenamiento de los motores de traducción, pues los ingresos del mercado de la traducción automática están destinados a aumentar en los próximos años a causa del ritmo de fabricación y de la productividad ofrecida.

Palabras clave: Metaverso; Traducción Automática Neuronal; Calidad.

INTRODUCTION

The metaverse is presented as a virtual and three-dimensional ecosystem offering personalized spaces to improve the user experience and a collaborative space focused on improving work productivity. One of the advantages of the metaverse is the individual interaction that users carry out, through which they could improve their language skills and access an immersive learning environment,⁽¹⁾ in addition to the cost savings that their spaces represent compared to trips to the foreigner.⁽²⁾

This note leads us to talk about cosmolocation. The communicative process includes temporal and spatial characteristics that favor interaction and, in the case of virtual environments, multimodality, accessibility, and a high motivational component.⁽³⁾

In addition, users can create content, allowing space for imagination and exploration and fostering effective learning environments.⁽⁴⁾. However, the success of communication in the metaverse is mediated by machine translation engines, which provide the interaction generated between users with the five key components of the platform: "digital, spatial, immersive, shared and in real-time".⁽⁵⁾

In recent years, numerous efforts have been made to improve the productivity of machine translation in favor of reducing language barriers and supporting trade and interculturality. As a result of its high speed of response in communication, it has acquired a leading role in the industrial expansion of recent years and the advent of new technologies. However, its beginnings date back to the 50s. Specifically, in 2020 the information market machine translation was valued at 153,8 million dollars and is expected to reach 230,67 in 2026.⁽⁶⁾

A few months ago, Meta announced the launch of a universal translator for the metaverse through a double plan called: No Language Left Behind, based on a translation system capable of learning any language and its oral translation.⁽⁷⁾ However, the brand recognizes the limitations of machine translation, which cannot cover the wide existing linguistic diversity. It exposes how it improves the need to collect linguistic data from more languages and better treatment of the current ones.⁽⁸⁾

Given the communicative project to be carried out in the virtual world, this paper aims to offer a state of the art of machine translation through different investigations focused on the metaverse, from various disciplines which have used this service. The aim is to know the perceptions about the current use and quality of machine translation and thus understand its potential in virtual reality and the challenges it poses.

METHODS

The research was carried out by collecting data on the available studies on metaverse in the SciELO, Google Scholar, and Dialnet databases. Given the lack of studies that directly study the influence of translation in the metaverse, we have collected, through a bibliographic review, those that have directly or indirectly used the machine translation tool to favor communication and interaction between users. The authors' perception about the use, the (dis)advantages, and the potential this service offers in the metaverse have been qualitatively collected.

RESULTS AND DISCUSSION

Although it is associated with companies' productivity and cost reduction, machine translation is no stranger to the operation of large organizations. For example, in the European Union, machine translation appeared in the 1970s, and, starting in 2010, a hybrid machine translation service called MT@EC (currently eTranslation) was introduced for the translation of Commission work.⁽⁹⁾ In Spain, some state administration bodies, such as the Ministry of Justice, the Ministry of Defense, and the Spanish Agency for Data Protection, among many others, have the PLATA platform. It is an automatic translation system based on statistics through the help of Apertium and Moses engines. Finally, in the US, Obama's cabinet promoted machine translation to promote international communication and trade in the Strategy for American innovation.⁽¹⁰⁾

In the case of the metaverse, neural machine translation is used to promote communication. This communication "groups recurring operations that map a fragment, word or set of words concerning the words with which they are integrated, lemmas which, in turn, are mapped similarly".⁽¹⁰⁾ The combination of artificial neural networks, together with the so-called deep learning, improves self-learning algorithms and leads to better linguistic solutions when dealing with complex issues in translation, such as context, ambiguity, interpretation, or the presence of several records.⁽¹¹⁾

For this reason, and aware that language plays a key role in communication in the metaverse, and must accommodate the linguistic plurality of users, Ifeanyi et al.⁽¹²⁾ investigate translation by comparing the automatic translation tools of Google and DeepL. To do this, they use an analysis based on the accuracy of the translations, the number of languages they offer, their interface, and their integration with other systems. Although they conclude that DeepL provides greater precision than Google and fewer languages, they raise pending issues for improving this linguistic service in the metaverse. Engines must increase the number of languages available and offer more integrations with other applications. Users would need to create their own experiences in the metaverse, beyond access to those available by default, and interop issues would need to be resolved. Although the precision offered by neural machine translation, the result of artificial neural networks that select the most appropriate sequence, increases with the new prediction and statistical models.

Ullah et al.⁽¹³⁾ study the translation offered from English to Urdu in the metaverse. The numerous errors and inaccuracies found at the syntactic and semantic level stand out through the translation analysis of various posts, such as the omission of idioms and expressions, destruction of syntactic structures, and expansion and clarification tendencies, among others. Likewise, Nakahira et al.⁽¹⁴⁾ show errors in the machine translation they use to promote intercultural collaboration in the metaverse. Aware that cultural exchanges are not always affordable for all students, they propose to bring this type of immersive learning to the metaverse. Specifically, they created a learning environment through PBL (Problem-Based Learning), where students could establish exchanges with other cultures, allowing them to integrate new cultural perspectives into their training. However, when using the translation service, they noticed a lack of quality in the conversations, the result of inadequate precision in the translation. They propose establishing a common language and resorting to inverse translation to do this. However, contrary to this last point, the choice of a single language in the metaverse could cause the loss of linguistic diversity, which is why translation would allow for maintaining culture.⁽¹⁵⁾

Other studies have analyzed dialogical interaction, such as the case of Sang et al.⁽¹⁶⁾, who study communication and language. They analyze different approaches the metaverse offers and that it could integrate. Specifically, in terms of multitasking interaction, they propose Google's Translatotron project for dialogue, which would provide an immediate oral translation and reproduce the same prosody of the sender without resorting to a text. They also highlight the lack of multilingual support in the interface, having to resort to English as the standard language for users.

On the other hand, using machine translation raises ethical issues, such as the occasional damage that a translation with errors can cause, the dilemma of the identity of the author of the text in case of liability, and the data breach that can occur.⁽¹⁷⁾ One of these possible damages has to do with the use of language since one of the roles of applied linguistics is to adjust the wording to social needs and avoid racializing its use.⁽¹⁸⁾ For this purpose, this author states that the metaverse must offer an inclusive experience and invites us to think about preserving languages in a virtual world, to address racism and overcome racialization.

From an interdisciplinary point of view, many studies have addressed the functions of the metaverse applied to the field of education.^(19,20,21) However, given these studies' lack of systematic reviews, Tlili et al.⁽²²⁾ propose a content and bibliometric analysis of the existing literature. They highlight the potential of Second Life, not only because of the possibilities it offers to learn a language and establish linguistic debates but also because of the translation service and its role as a communication bridge, which makes possible more specific exchanges. In this regard, we add some studies that have implemented interactions between students in the metaverse. Initially, Farjami et al.⁽²³⁾ created a classroom in Second Life to encourage multilingual classroom discussions within engineering education. Through the questionnaire and the study, they affirmed that the linguistic grid they used brought notable benefits and that the metaverse was declared an effective tool for solving problems in virtual learning. For their part, and also in the engineering field, Kanematsu et al.⁽²⁴⁾ explore an intercontinental experience through the PBL (Project Based Learning) methodology. Two debates were held between students from the United States, Korea, and Japan, one of them with the support of the translation system. They introduced a linguistic grid for the debate with translation, and the interventions were recorded. Although they concluded that the results were remarkable, the Korean students preferred the session without the translation tool in the questionnaire, which is a reason that the authors attribute to the insufficient precision of the English-Korean translation. In this regard, one of the reasons that we propose may be that contrary to the richness of human language, "artificial language [...] is designed with a specific purpose, it is restricted in its syntax and semantics. For this reason, it is more precise, with less room for free interpretation and free from context”.⁽²⁵⁾

CONCLUSIONS

Machine translation is destined to increase its presence in the market due to the productivity it brings to some types of communication and to form an essential pillar in the communication of virtual worlds. In the Metaverse, the user has the translation from a large volume of interlaced data using artificial neural networks and the deep learning technique. However, despite the significant advances in recent years, it is clear that the complexity of human language means that the engines are not able to finalize quality texts about aspects such as interpretation, the categorization of genres for more effective treatment of the text, the homogenization of terminology or the different jargons, among others.

Aware of translation's outstanding role in the metaverse, this work represents the linguistic analysis of the starting point of an ambitious project that foresees a few years of real virtual work meetings in virtual and three-dimensional spaces shared by all to favor the experience of the user. Likewise, oral translation engines are expected to copy the prosody of the voice and provide avatars with greater personalization and reduce automation. Of the papers collected, only two focus on the linguistic aspect. The rest, probably not having linguistics as a direct object of study, are pleased with the possibility offered by machine translation.

Overall, we advocate the need to conduct translation studies in the virtual world since neural machine translation can offer an excellent service in the Metaverse by encompassing other technologies and activities, causing an even greater demand for video game localization, cryptocurrency, or digital commerce, among others.

Finally, we highlight the lack of studies that show the importance of culture since, in the translation process, transcreation is understood in the linguistic field as the process by which the content is adapted to a specific context, being able to change it completely to preserve the same tone and effect. It is essential in intercultural communication because, as the German writer and artist Günter Grass said: “translation is what changes everything so that nobody changes”.

REFERENCES

1.    Junjie GW, Danyang Z, Sangmin ML. Into the Brave New Metaverse: Envisaging Future Language Teaching and Learning. IEEE Transactions on Learning Technologies. 2023; 1-11. https://doi.org/10.1109/TLT.2023.3259470

2.    Sari YE. Authentic Language Learning in Metaverse: A Big Step in the Technological Revolution or a Buzzword? HAYEF: Journal of Education. 2022; 19(3): 255-259. https://doi.org/10.5152/hayef.2022.28

3.    Godwin-Jones R. Emerging spaces for language learning: AI bots, ambient intelligence, and the metaverse. Language Learning & Technology. 2023; 27(2): 6–27.

4.    Xinghong L, Junxiang Z. Foreign Language Learning through Virtual Communities. Energy Procedia. 2012; (17): 737-740. https://doi.org/10.1016/j.egypro.2012.02.165

5.    Hadi R, Melumad S y Park E. The Metaverse: A New Digital Frontier for Consumer Behavior. Journal of Consumer Psychology. 2023.  https://ssrn.com/abstract=4380636

6.    Mordor Intelligence. Mercado de traducción automática: crecimiento, tendencias, impacto de Covid.19 y pronósticos (2023-2028). 2023. https://www.mordorintelligence.com/es/industry-reports/machine-translation-market

7.    Cocking S. How the language landscape will evolve if metaverse succeeds [Irish Tech News]. 2023 Mar 9. https://irishtechnews.ie/will-the-language-evolve-if-metaverse-succeeds/

8.    Meta AI. Teaching AI to translate 100s of spoken and written languages in real time. 2022 Feb 23. https://ai.facebook.com/blog/teaching-ai-to-translate-100s-of-spoken-and-written-languages-in-real-time/

9.    Rodríguez Vázquez S. Traducción automática y posedición en la DGT: de la teoría a la práctica. Puntoycoma. 2020; (167): 13–27.

10.  Pym A, Torres-Simón E. Efectos de la automatización en las competencias básicas del traductor: la traducción automática neuronal. In Vidal Suñé A, Alarcón Alarcón A, editors. Ocupaciones y lenguaje. Indicadores y análisis de competencias lingüísticas en el ámbito laboral. Tarragona: Publicacions URV; 2021, p. 479-509.

11.  Rincón Córcoles A. La traducción científico-técnica en la encrucijada: tecnología y automatización. Informe ACTA. 2020. https://www.acta.es/medios/informes/2020002.pdf

12.  Ifeanyi Nwakan C, Nkechinyere Njoku J, Dong-Seong K. Evaluation of Language Translator Module for Metaverse Virtual Assistant. Conference 2022 Korean Institute of Communications and Information Sciences (KICS). 2022; (78). https://journal-home.s3.ap-northeast-2.amazonaws.com/site/2022s/abs/0087.pdf

13.  Ullah Z, Alam MF, Ihsan W. Juxtaposing Urdu to English Captions of Facebook/ Metaverse: A Neural Machine Translation Study. Global Language Review. 2022; 7(2): 55-66. https://doi.org/10.31703/glr.2022(VII-II).05

14.  Nakahira KT, Nunez Rattia R, Taguchi R, Kanematsuy H, Fukumural Y. (2010). Design of a multilinguistic Problem Based Learning learning environment in the metaverse. 2nd International Symposium on Aware Computing. 2010: 298-303. https://doi.org/10.1109/ISAC.2010.5670497

15.  Malyukov P. Languages in the Metaverse: Why AI is critical for communication in the brave new world. Venture Beat. 2022 Mar 26. https://venturebeat.com/datadecisionmakers/languages-in-the-metaverse-why-ai-is-critical-for-communication-in-the-brave-new-world/

16.  Sang Ming P, Young-Gab K. A Metaverse: Taxonomy, Components, Applications, and Open Challenges. In IEEE Access. 2022;(10):4209-4251. https://doi.org/10.1109/ACCESS.2021.3140175

17.  Canfora C, Ottmann A. Risks in neural machine translation. Translation Spaces. 2020; (9): 58–77. 2020 https://doi.org/10.1075/ts.00021.can

18.  Smith P. Black immigrants in the United States: Transraciolinguistic justice for imagined futures in a global metaverse. Annual Review of Applied Linguistics. 2022;(42):109–118. https://doi.org/10.1017/S0267190522000046

19.  Márquez Díaz JE, Domínguez Saldaña CA, Rodriguez Ávila, CA. Virtual world as a resource for hybrid education. International Journal of Emerging Technologies in Learning. 2020;15(15):94–109. https://doi.org/10.3991/ijet.v15i15.13025

20.  Jovanovic A, Milosavljevic A. VoRtex metaverse platform for gamified collaborative learning. Electronics. 2022;11(3): 317. https://doi.org/10.3390/electronics11030317

21.  Koo H. Training in lung cancer surgery through the metaverse, including extended reality, in the smart operating room of Seoul National University Bundang Hospital, Korea. Journal of Educational Evaluation for Health Professions. 2021;(18):1-4. https://doi.org/10.3352/jeehp.2021.18.33

22.  Tlili A, Huang R, Shehata B et al. Is Metaverse in education a blessing or a curse: a combined content and bibliometric analysis. Smart Learning Environments. 2022; (9): 1-31. https://doi.org/10.1186/s40561-022-00205-x

23.  Farjami S, Taguchi R, Nakahira KT, Nunez Rattia R, Fukumura Y, Kanematsu H. (2011). Multilingual Problem Based Learning in Metaverse. In König A, Dengel A, Hinkelmann K, Kise K, Howlett RJ, Jain LC, editors. Knowledge-Based and Intelligent Information and Engineering Systems. KES 2011. Lecture Notes in Computer Science, vol. 6883. Berlin: Springer; 2011. p. 1-2. https://doi.org/10.1007/978-3-642-23854-3_53

24.  Kanematsu H, Fukumura Y, Barry DM, Sohn SY, Taguchi R, Nunez Rattia A. Conversation Analysis of PBL in Metaverse for students from the USA, Korea and Japan. International Conference on Engineering Education ICEE. 2010; 1-7.

25.  Chaves Torres A, Zuleta Medina A. Procesamiento del lenguaje natural, un reto de la inteligencia artificial. Revista Matices Tecnológicos. 2012;(4):1-5.

FUNDING

This work has been funded by the program Ayudas para la recualificación del sistema universitario español (2021-2023) of the University of Salamanca: Margarita Salas for the training of young doctors.

CONFLICT OF INTEREST

The author declares that there is no conflict of interest.

AUTHORSHIP CONTRIBUTION

Conceptualization: Rubén González Vallejo.

Data curation: Rubén González Vallejo.

Formal analysis: Rubén González Vallejo.

Acquisition of funds: Rubén González Vallejo.

Research: Rubén González Vallejo.

Methodology: Rubén González Vallejo.

Project Administration: Rubén González Vallejo.

Resources: Rubén González Vallejo.

Software: Rubén González Vallejo.

Supervision: Rubén González Vallejo.

Validation: Rubén González Vallejo.

Visualization: Rubén González Vallejo.

Writing - original draft: Rubén González Vallejo.

Writing - revision and editing: Rubén González Vallejo.