Google

In the expansion step of beam search, a set of candidate sequences for step `i`, denoted $Y_i$, is formed. This set consists of the K new hypotheses generated by extending the previous step's sequences. It is formally defined as the collection of the top K candidate sequences: $$Y_i = \{\mathbf{y}^{\text{top1}}, \dots, \mathbf{y}^{\text{topK}}\}$$ where each $\mathbf{y}^{\text{top k}}$ is a newly formed sequence.

Formula for the Candidate Set in Beam Search

An algorithm is generating a sequence using a search process that keeps track of the 2 most likely options at each step (K=2). At a certain point, it considers extending the current options, resulting in four potential new sequences with the following scores (higher is better):

- 'The cat sat on' (Score: 0.9)
- 'The cat sat by' (Score: 0.6)
- 'The dog ran to' (Score: 0.8)
- 'The dog ran from' (Score: 0.5)

According to the procedure for forming the candidate set for the next step, which sequences will be included in this set?

Forming the Candidate Set in a Search Algorithm

In a sequence generation algorithm that maintains a fixed number of top hypotheses at each step, the candidate set for the current step is formed by taking the newly generated sequences and adding them to the set of hypotheses from the previous step.

An algorithm is generating a sequence, keeping the 2 most likely partial sequences (the 'beam') at each step. At the current step, the beam contains sequence 'A' (score: -1.0) and sequence 'B' (score: -2.0). The algorithm then identifies the most likely next tokens to extend these sequences:

- For 'A', the next tokens are 'C' (score: -1.5) and 'D' (score: -1.6).
- For 'B', the next tokens are 'E' (score: -0.1) and 'F' (score: -0.2).

Given that scores are combined by addition, which of the foll

Learn Before

Related