Example of Code-Switching between Chinese and English
A mixed-language sentence that blends Chinese and English, such as '周末 我们 打算 去 做 hiking ， 你 想 一起 来 吗 ？' (We plan to go hiking this weekend, would you like to join us?), is a prominent instance of code-switching. When processed by multi-lingual pre-trained models, there is no need to identify whether a token is Chinese or English. Instead, every token is simply treated as an entry in a shared vocabulary, effectively treating the mixed input as a single composite language.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Why Speakers Code-Switch
Benefit of Multilingual Pre-trained Models: Handling Code-Switching
A user is sending text messages that mix two different languages. Which of the following messages best exemplifies the practice of alternating between languages within a single, coherent thought or sentence?
Diagnosing NLP Model Failure
Defining and Illustrating Code-Switching
Language-Independent Token Representations
Grammatical Distinction Between Code-Switching and Lexical Borrowing
A multilingual model is pre-trained on a large corpus of English and Spanish text using a single, unified vocabulary. The model processes the word 'pie', which means 'foot' in Spanish and refers to a baked dish in English. How will this word most likely be represented within the model's vocabulary structure?
Trade-offs of a Unified Vocabulary in Multilingual Models
In a multilingual model pre-trained on English and German, the shared vocabulary is structured into two distinct sections, one for English tokens and one for German tokens, to prevent interference between the languages.
Learn After
A bilingual speaker is sending a text message. Which of the following messages best demonstrates the practice of embedding a word from one language into the grammatical structure of another language within a single, continuous sentence?
Identifying Code-Switching in a Conversation
Constructing a Mixed-Language Sentence