Google

In the context of a Large Language Model (LLM), predictive inference involves selecting the optimal output $$\hat{y}$$ by maximizing the model's probability distribution. This is formally expressed by combining the general prediction formula with the specific probability function of the model: $$\hat{y} = \arg\max_{y} \text{LLM Pr}_{\theta}^{s}(y)$$. In this notation, $$\theta$$ represents the model's parameters, and the superscript $$s$$ refers to the specific scoring method used to evaluate the outputs.

Predictive Inference Formula in Large Language Models

Given the following scenario, determine the model's predicted output (ŷ) and explain the reasoning behind the selection process.

Applying Predictive Inference

Consider the formal expression for predictive inference in a large language model:  
`ŷ = argmax_y LLM Pr_θ^s(y)`  

Deconstruct this expression by explaining the specific role and significance of each of the following three components in determining the final output `ŷ`:  
1. `argmax_y`  
2. `LLM Pr(y)`  
3. `θ` and `s`

Deconstructing the Predictive Inference Formula

A large language model, with a fixed set of parameters `θ` and using a specific scoring method `s`, is performing an inference task. Its goal is to select the single most probable output `ŷ` from the entire space of possible outputs `y`. Which mathematical expression correctly represents this process?

Learn Before

Related