What positional adjustment must be applied to the output embeddings to help enforce the autoregressive constraint in the Transformer decoder?
0
1
Tags
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.1 Transformer Architecture Fundamentals - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Decoder Masking and Autoregressive Prediction - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Related
To preserve autoregression in the Transformer decoder, what dependency constraint must be satisfied by the prediction generated for position ?
Explain how the Transformer decoder prevents future position information from influencing predictions, detailing the necessary mechanisms and the operational condition they uphold.