Learn Before
Comparison of Complex and Euclidean Representations of RoPE Rotation
While the formula for Rotary Positional Embeddings (RoPE) in complex space, , appears intricate, it has an equivalent and more intuitive formulation within the d-dimensional Euclidean space.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Consider a 4-dimensional real vector . A transformation is applied that groups the vector's components into consecutive pairs, treats each pair as a complex number (a + ib), and rotates them independently. The first pair (components 1 and 2) is rotated by an angle of radians, and the second pair (components 3 and 4) is rotated by an angle of radians. What is the resulting vector after this transformation?
Determining an Unknown Rotation Angle
A transformation is applied to a d-dimensional real vector by grouping its components into d/2 consecutive pairs. Each pair is treated as representing a point in a 2D plane and is independently rotated by a specific, non-zero angle. True or False: This transformation will always change the Euclidean norm (magnitude) of the original d-dimensional vector.
Learn After
Euclidean Representation of RoPE Rotation
Element-wise Formula for RoPE Rotation
A positional encoding method can be described in two ways. In one view, a d-dimensional vector is treated as d/2 complex numbers, and each complex number is rotated by an angle. In another view, the same d-dimensional vector is treated as d/2 pairs of real numbers, and each pair is rotated as a 2D vector by the same corresponding angle. What is the core principle that makes these two views equivalent?
Equivalence of 2D Vector Rotation Representations
Verifying Equivalence of 2D Rotational Methods