Google

The output stage, labeled as step ④, is the final step in the top-k sampling process. In this stage, the single token that was chosen during the selection and sampling phase is finalized as the output. For example, after the token 'on' was sampled from the renormalized distribution, it becomes the definitive next token in the generated sequence.

Output Stage in Top-k Sampling

Based on the provided scenario, describe the two crucial, sequential actions the model must perform on the reduced set of candidates to determine the final output token.

Language Model Token Selection

A language model is generating text and has just identified the 'k' most likely next tokens from a large vocabulary. Arrange the following actions in the correct chronological order to complete the generation of the single next token.

A language model has calculated the probabilities for the next possible tokens in a sequence. The five most likely tokens are: 'the' (0.4), 'a' (0.2), 'on' (0.1), 'in' (0.05), and 'at' (0.05). If the model uses a selection process where only the top 3 candidates are considered (k=3), what will be the new, renormalized probability distribution for the tokens that are ultimately sampled from?

Learn Before

Related