Google

The output of a Transformer layer at depth $$l$$, denoted as $$\mathbf{H}^{l+1}$$, is computed by applying the layer's transformation function to its input, $$\mathbf{H}^{l}$$. This relationship can be expressed by the formula: $$ \mathbf{H}^{l+1} = \mathrm{Layer}(\mathbf{H}^{l}) $$.

Transformer Layer Output Formula

An initial input matrix, denoted as $H^0$, is processed sequentially through three computational layers: $\text{Layer}_1$, $\text{Layer}_2$, and $\text{Layer}_3$. Which expression correctly calculates the output matrix of $\text{Layer}_2$?

In a multi-layer computational model, the output of the fifth layer is a matrix of hidden states denoted as $H^5$. This matrix serves as the input to the sixth layer, which has a transformation function represented as $\text{Layer}_6$. The output of this sixth layer, $H^6$, is calculated by the formula: $H^6 = \text{____}$.

A computational model processes an initial input matrix, $H^0$, through three sequential layers. Arrange the following hidden state matrices in the order they are generated.

Learn Before

Related