Google

In a greedy search decoding algorithm, the set of candidate sequences at step $$i$$, denoted $$Y_i$$, is reduced to a singleton set containing only the single most optimal sequence generated up to that point. This sequence is constructed by appending the most probable next token to the previous prefix: $$\mathbf{y}^{\mathrm{top}1} = y_1...y_{i-1} y_i^{\mathrm{top}1}$$. The candidate set is formally expressed as $$Y_i = \{\mathbf{y}^{\mathrm{top}1}\}$$.

Formula for the Candidate Set in Greedy Search

In a text generation model that employs a greedy search decoding strategy, the size of the candidate set of sequences, denoted as `|Y_i|`, increases at each step `i` because the model considers a growing number of possible sentence completions.

A language model is generating a sequence using a decoding strategy where, at each step, it only pursues the single most likely option. After step `i-1`, the single best sequence is `y_1...y_{i-1}`. At step `i`, the model computes the probabilities for the next token, finding that `p(token_A | y_1...y_{i-1}) = 0.6`, `p(token_B | y_1...y_{i-1}) = 0.3`, and `p(token_C | y_1...y_{i-1}) = 0.1`. Formally define the candidate set of sequences, `Y_i`, at the end of step `i` and explain the reasoning behind its composition.

Defining the Candidate Set in a Greedy Decoding Strategy

A text generation model employs a decoding strategy where, at each step, it extends the single most probable sequence generated so far. Suppose after three steps, the sequence 'The cat sat' has been generated. In the fourth step, the model calculates probabilities for the next word and finds that 'on' is the most likely, 'under' is the second most likely, and 'by' is the third most likely. Based on this strategy, what is the composition of the candidate set of sequences immediately after this fo

Learn Before

Related