Google

The expansion stage is the initial step in the K-best selection process. In this phase, a language model generates a set of potential next tokens or words based on the current context. For example, as depicted in the diagram labeled '① Expansion', given the token 'is', the model might produce a list of candidate words such as '.', 'cute', 'on', 'are', and 'sick'. These candidates are then passed to the subsequent ranking stage.

Expansion Stage in K-Best Selection

When a language model generates the next word in a sentence, it follows a multi-step process to select the most promising options. Arrange the following three actions into the correct logical sequence.

Based on the provided scenario, evaluate the data scientist's belief. Is it correct? Justify your answer by identifying the correct stage this output represents and describing its primary function in the overall selection process.

Identifying the Generation Stage

A language model is generating a sentence and has so far produced the sequence: 'The sun is shining and the birds are...'. In the first phase of determining the next word, the model generates a list of potential candidates from its vocabulary based on the preceding text. Which of the following options best illustrates the direct output of this initial generation phase, before any sorting or filtering occurs?

Learn Before

Related