Google

In top-k sampling, the selection pool, denoted as $$V_i$$ for a given step $$i$$, is the set of the top-k most probable tokens from which the next token is chosen. This pool is formally defined as: $$V_i = \{y_i^{\text{top1}}, \dots, y_i^{\text{topk}}\}$$

Definition of the Top-k Selection Pool

A language model is at a specific generation step, which we can call step $i$. It has calculated the probabilities for the next token over a small vocabulary: P('on')=0.05, P('the')=0.45, P('a')=0.25, P('in')=0.10, P('at')=0.15. If the model is using a sampling method where the selection pool is limited to the 3 most probable tokens, what is the set $V_i$ for this step? Please write your answer using set notation, e.g., {'token1', 'token2'}.

Constructing the Top-k Selection Pool

If the parameter 'k' is set to 1, the resulting selection pool will contain only the single most probable token, making the sampling process from this pool effectively equivalent to a greedy search at that step.

A language model is generating text and at a certain step, it predicts the following probabilities for the next token from its vocabulary: {'the': 0.4, 'a': 0.2, 'cat': 0.15, 'dog': 0.1, 'ran': 0.08, 'sat': 0.07}. If the model is configured to use a sampling strategy where the next token is chosen from the 3 most probable candidates, which set represents the selection pool for this step?

Learn Before

Related