1Cademy - Application of RoPE Rotation to a 2D Vector

Learn Before

Application of RoPE to Token Embeddings
Definition of the 2D Rotation Matrix

Formula

Application of RoPE Rotation to a 2D Vector

The Rotary Positional Embedding (RoPE) rotation is applied to a two-dimensional vector $x = \begin{bmatrix} x_1 & x_2 \end{bmatrix}$ by post-multiplying the vector by the corresponding 2D rotation matrix, $R_\theta$ . The resulting rotated vector's components are calculated as follows: $\mathrm{Ro}(x, \theta) = xR_\theta = \begin{bmatrix} x_1 & x_2 \end{bmatrix} \begin{bmatrix} \cos \theta & \sin \theta -\sin \theta & \cos \theta \end{bmatrix} = \begin{bmatrix} \cos \theta \cdot x_1 - \sin \theta \cdot x_2 & \sin \theta \cdot x_1 + \cos \theta \cdot x_2 \end{bmatrix}$

Updated 2026-06-24

Contributors are:

Who are from:

References

Reference of Foundations of Large Language Models Course

Learn After

A two-dimensional vector is represented as x = [3, 4]. Calculate the resulting vector after applying a rotational transformation with an angle θ = 90° (π/2 radians).
Consider the transformation applied to a two-dimensional vector $x = \begin{bmatrix} x_1 & x_2 \end{bmatrix}$ by an angle $\theta$ , resulting in a new vector $y = \begin{bmatrix} \cos \theta \cdot x_1 - \sin \theta \cdot x_2 & \sin \theta \cdot x_1 + \cos \theta \cdot x_2 \end{bmatrix}$ . This transformation will always alter the magnitude (or length) of the original vector $x$ .
Analysis of Rotational Transformation Properties
Derivation of the Dot Product for RoPE-Encoded Vectors

Learn Before

Related

Learn After