Base Case for Sequence Probability

Joint Probability of a Generated Sequence using the Chain Rule

Relationship Between Joint, Conditional, and Marginal Log-Probabilities of Sequences

Derivation of Sequence Log-Probability via Chain Rule

Logarithmic Form of the Chain Rule for Sequence Probability

Formula for an Impossible Initial Event

A language model is tasked with calculating the total probability of the three-token sequence 'the cat sat'. The model provides the following probability estimates:

• The probability of the first token is Pr("the") = 0.1
• The probability of the second token, given the first, is Pr("cat" | "the") = 0.5
• The probability of the third token, given the first two, is Pr("sat" | "the", "cat") = 0.8

Using the principle that the joint probability of a sequence is the product of the condit

Computational Stability of Sequence Probability