Learn Before
Simultaneous Processing of Input Context Tokens
In the initial stage of text generation, all tokens that constitute the input context, denoted as 'x', are provided to the model simultaneously for parallel processing.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Examples of text generation
Decoding Methods to Generate Continuations in TGM
Stochastic decoding methods in TGM
Building the Encoded Representation of Input
A user gives a language model the input: "Ancient Rome was a civilization known for its". The model then produces the following output: "engineering marvels, such as aqueducts and roads." Based on the two-stage process of text generation, which statement best analyzes this interaction?
Arrange the following stages into the correct sequence that describes how a language model generates text based on an initial input.
Analyzing a Code Generation Scenario
Learn After
When a language model generates a response, it first processes the user's entire input prompt and then generates the output one token at a time. How does the computational approach for these two phases typically differ in terms of how tokens are handled?
When a language model is given an initial text prompt, it processes the tokens of that prompt one by one, in the order they appear, before it starts generating a response.
Processing Asymmetry in Text Generation