braindecode.functional.apply_rotary

braindecode.functional.apply_rotary(x, cos, sin, interleaved=True)

Rotate x by the angles whose cosine and sine are given.

x * cos + rotate(x) * sin, where rotate is rotate_pairs() (interleaved=True) or rotate_half(). cos and sin must broadcast to x; casting is left to the caller.

Parameters:

• x (Tensor) – Shape (..., head_dim).

• cos (Tensor) – From rotary_angles().

• sin (Tensor) – From rotary_angles().

• interleaved (bool) – Feature layout, as in rotary_angles().

Returns:

Same shape as x.

Return type:

Tensor