Google

This diagram illustrates the architectural flow for a single step of autoregressive generation in a decoder-only Transformer. The process begins with the input sequence, formed by concatenating the prompt tokens (`x0...xm`) and any previously generated tokens (`y1...yi-1`). This sequence is fed into an embedding layer. The resulting embeddings are then processed through a stack of `L` decoder layers, each comprising self-attention and feed-forward network (FFN) modules. The output from the final layer is passed through a linear mapping and a Softmax layer to compute the conditional probability distribution, `Pr(·|x, y_{<i})`, for the next token.

Diagram of the Autoregressive Generation Architectural Flow

A decoder-only language model generates text one token at a time. Arrange the following computational steps in the correct order for generating a single new token, given a prompt and any previously generated tokens.

In the architectural flow for generating a single new token, a decoder-only model processes the input sequence through multiple layers. After the final decoder layer produces its output vector, what is the immediate and primary purpose of applying a final linear mapping and a Softmax function?

In the architectural flow for generating a single new token, a decoder-only model performs several distinct operations. Match each architectural component with its primary function during this single-step process.

Learn Before

Related