Based on the provided scenario, which model (Alpha or Beta) would be better suited for generating a sequence where each new element must be consistent with the overall context of the *entire* preceding sequence? Justify your reasoning by analyzing the information captured by each model's state function.

Google

The formula $$s(\bar{\mathbf{y}}_1 \dots \bar{\mathbf{y}}_{k-1})$$ represents a function, denoted by $$s$$, that computes a state or context based on a sequence of the previous $$k-1$$ outputs. Each $$\bar{\mathbf{y}}_i$$ is a vector representing the $$i$$-th output in the sequence. This is a fundamental concept in autoregressive models where the prediction of the next element depends on the context derived from the preceding elements.

State Function from Previous Outputs

In a model that generates a sequence of items one by one, a function `s(ȳ₁, ..., ȳₖ₋₁)` is used to compute a summary, or 'state', from the `k-1` items that have already been generated. What is the primary purpose of this computed state?

Consider a model generating the sentence 'The cat sat on the mat.' After generating the first three words ('The', 'cat', 'sat'), the model uses a function, `s(ȳ₁, ȳ₂, ȳ₃)`, to compute a summary based on these outputs. In this specific scenario, what does this summary represent, and how is it used by the model to determine the *next* word in the sequence?

Learn Before

Related