braindecode.models.NeuroRVQTokenizer#

class braindecode.models.NeuroRVQTokenizer(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, channel_names=None, modality='eeg', patch_size=None, on_non_divisible='pad', on_missing='error', max_patches=None, out_chans=8, encoder_depth=12, decoder_depth=3, num_heads=10, n_code=8192, code_dim=128, num_quantizers=None, drop_prob=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, init_values=0.0, activation=<class 'torch.nn.modules.activation.GELU'>, statistic_code_usage=False)[source]#

NeuroRVQ multi-scale residual-vector-quantized biosignal tokenizer from Barmpas et al. [neurorvq].

Convolution Attention/Transformer Foundation Model Channel

The tokenizer encodes EEG, ECG, EMG or PPG patches through four temporal scales, quantizes each scale with a separate residual vector quantizer, and reconstructs the signal from amplitude and phase components. The Transformer blocks, temporal convolution, channel lists and modality presets are shared with NeuroRVQ.

Inputs must be sampled at the modality’s rate and use channel names from the modality’s pretrained list (see NeuroRVQ). The model does not preprocess signals; the released EEG example applies notch filters at 50, 60 and 100 Hz, a 0.5-44.5 Hz band-pass, clipping at 500 uV and resampling to 200 Hz.

Important

Pre-trained Weights Available

The released tokenizers (ntinosbarmpas/NeuroRVQ, revision d944b87, CC BY-NC 4.0) are hosted as braindecode/neurorvq-tokenizer-eeg-pretrained, braindecode/neurorvq-tokenizer-ecg-pretrained, braindecode/neurorvq-tokenizer-emg-pretrained and braindecode/neurorvq-tokenizer-ppg-pretrained; the foundation models are in braindecode/neurorvq-{eeg,ecg,emg}-pretrained (NeuroRVQ):

from braindecode.models import NeuroRVQTokenizer

model = NeuroRVQTokenizer.from_pretrained(
    "braindecode/neurorvq-tokenizer-eeg-pretrained", chs_info=raw.info["chs"]
)

Raw reconstruction MSE (Table 10 of [neurorvq]), with the data preparation of the EEG-Benchmarking code linked from the NeuroRVQ repository: the port gives 0.0858 on Pavlov 2022 (paper: 0.084) and 0.0748 on High Gamma (paper: 0.090); the authors’ released code gives the same numbers on the same data.

Added in version 2.0.

License

Parameters:
  • n_outputs (int | None) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int | None) – Number of channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int | None) – Number of samples. It gives n_patches patches (ceil(n_times / patch_size) by default, see on_non_divisible), at most max_patches.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency; must be the modality’s rate.

  • channel_names (tuple[str, ...] | list[str] | None) – Ordered channel names. If omitted, names are inferred from chs_info or default to the first channels in the pretrained order.

  • modality (str) – Released configuration (see NeuroRVQ); also sets the defaults of patch_size, max_patches and num_quantizers.

  • patch_size (int | None) – Samples per patch; None uses the modality’s value.

  • on_non_divisible (str) – How PatchTokenizer treats a time axis that is not a multiple of patch_size (right zero-pad, crop, or raise). The reference implementation requires a multiple ("error"); all three give the same model and output when n_times is a multiple of patch_size. With "pad" the zero samples are part of the target (and of its per-window z-scoring).

  • on_missing (str) – Input channels whose name has no pretrained slot: "error" raises, "drop" leaves them out of the input (with a warning), so the tokenizer encodes and reconstructs the channels it knows. Channels are never mapped by guesswork; a montage with no known channel still raises.

  • max_patches (int | None) – Length of the temporal embedding table; None uses the modality’s value.

  • out_chans (int) – Number of channels per temporal-convolution branch; must be divisible by four. The Transformer width is out_chans * patch_size // 8.

  • encoder_depth (int) – Number of shared encoder Transformer blocks.

  • decoder_depth (int) – Number of shared decoder Transformer blocks.

  • num_heads (int) – Number of attention heads in both Transformer stacks.

  • n_code (int) – Number of entries per EMA codebook.

  • code_dim (int) – Dimension of each quantized latent vector.

  • num_quantizers (int | None) – Number of residual codebooks per temporal scale (EMG 16, the others 8); None uses the modality’s value.

  • drop_prob (float) – Dropout probability in the Transformer stacks.

  • attn_drop_rate (float) – Attention-probability dropout.

  • drop_path_rate (float) – Maximum stochastic-depth probability.

  • init_values (float) – LayerScale initialization used by the tokenizer checkpoint.

  • activation (type[Module]) – Activation in the temporal patch embedding.

  • statistic_code_usage (bool) – True lets eval forwards update the cluster_size code-usage EMA, as the authors’ code.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

Output shape: forward returns the target and reconstruction patches, each z-scored per window, shaped (batch, n_chans * n_patches, patch_size); their mean squared difference is the reconstruction error. tokenize returns integer codes shaped (4, num_quantizers, batch, n_chans * n_patches). With on_missing="drop", n_chans counts the kept channels.

Examples

>>> import torch
>>> from braindecode.models import NeuroRVQTokenizer
>>> model = NeuroRVQTokenizer(
...     n_chans=3, n_times=800, sfreq=200, channel_names=["C3", "Cz", "C4"]
... ).eval()
>>> x = torch.randn(2, 3, 800)  # (batch, channels, 4 s at 200 Hz)
>>> target, reconstruction = model(x)
>>> target.shape  # (batch, n_chans * n_patches, patch_size)
torch.Size([2, 12, 200])
>>> mse = (target - reconstruction).square().mean()
>>> model.tokenize(x).shape  # (scale, quantizer, batch, n_chans * n_patches)
torch.Size([4, 8, 2, 12])

References

[neurorvq] (1,2)

Barmpas et al. (2025). NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models. https://arxiv.org/abs/2510.13068

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import NeuroRVQTokenizer

# Train your model
model = NeuroRVQTokenizer(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-neurorvqtokenizer-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import NeuroRVQTokenizer

# Load pretrained model
model = NeuroRVQTokenizer.from_pretrained("username/my-neurorvqtokenizer-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = NeuroRVQTokenizer.from_pretrained("username/my-neurorvqtokenizer-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = NeuroRVQTokenizer.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x)[source]#

Return the per-window z-scored target and reconstruction.

Parameters:

x (Tensor) – The description is missing.

tokenize(x)[source]#

Return four-scale codes without applying EMA updates.

A cold codebook is initialized once from the input vectors by cosine k-means; load pretrained weights first for pretrained codes.

Parameters:

x (Tensor) – The description is missing.

Return type:

Tensor