braindecode.models.NeuroRVQTokenizer#
- class braindecode.models.NeuroRVQTokenizer(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, channel_names=None, modality='eeg', patch_size=None, on_non_divisible='pad', on_missing='error', max_patches=None, out_chans=8, encoder_depth=12, decoder_depth=3, num_heads=10, n_code=8192, code_dim=128, num_quantizers=None, drop_prob=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, init_values=0.0, activation=<class 'torch.nn.modules.activation.GELU'>, statistic_code_usage=False)[source]#
NeuroRVQ multi-scale residual-vector-quantized biosignal tokenizer from Barmpas et al. [neurorvq].
Convolution Attention/Transformer Foundation Model Channel
The tokenizer encodes EEG, ECG, EMG or PPG patches through four temporal scales, quantizes each scale with a separate residual vector quantizer, and reconstructs the signal from amplitude and phase components. The Transformer blocks, temporal convolution, channel lists and
modalitypresets are shared withNeuroRVQ.Inputs must be sampled at the modality’s rate and use channel names from the modality’s pretrained list (see
NeuroRVQ). The model does not preprocess signals; the released EEG example applies notch filters at 50, 60 and 100 Hz, a 0.5-44.5 Hz band-pass, clipping at 500 uV and resampling to 200 Hz.Important
Pre-trained Weights Available
The released tokenizers (ntinosbarmpas/NeuroRVQ, revision
d944b87, CC BY-NC 4.0) are hosted asbraindecode/neurorvq-tokenizer-eeg-pretrained,braindecode/neurorvq-tokenizer-ecg-pretrained,braindecode/neurorvq-tokenizer-emg-pretrainedandbraindecode/neurorvq-tokenizer-ppg-pretrained; the foundation models are inbraindecode/neurorvq-{eeg,ecg,emg}-pretrained(NeuroRVQ):from braindecode.models import NeuroRVQTokenizer model = NeuroRVQTokenizer.from_pretrained( "braindecode/neurorvq-tokenizer-eeg-pretrained", chs_info=raw.info["chs"] )
Raw reconstruction MSE (Table 10 of [neurorvq]), with the data preparation of the EEG-Benchmarking code linked from the NeuroRVQ repository: the port gives 0.0858 on Pavlov 2022 (paper: 0.084) and 0.0748 on High Gamma (paper: 0.090); the authors’ released code gives the same numbers on the same data.
Added in version 2.0.
- Parameters:
n_outputs (
int|None) – Number of outputs of the model. This is the number of classes in the case of classification.chs_info (list of dict) – Information about each individual EEG channel. This should be filled with
info["chs"]. Refer tomne.Infofor more details.n_times (
int|None) – Number of samples. It givesn_patchespatches (ceil(n_times / patch_size)by default, seeon_non_divisible), at mostmax_patches.input_window_seconds (
float|None) – Length of the input window in seconds.sfreq (
float|None) – Sampling frequency; must be the modality’s rate.channel_names (
tuple[str,...] |list[str] |None) – Ordered channel names. If omitted, names are inferred fromchs_infoor default to the first channels in the pretrained order.modality (
str) – Released configuration (seeNeuroRVQ); also sets the defaults ofpatch_size,max_patchesandnum_quantizers.patch_size (
int|None) – Samples per patch;Noneuses the modality’s value.on_non_divisible (
str) – HowPatchTokenizertreats a time axis that is not a multiple ofpatch_size(right zero-pad, crop, or raise). The reference implementation requires a multiple ("error"); all three give the same model and output whenn_timesis a multiple ofpatch_size. With"pad"the zero samples are part of the target (and of its per-window z-scoring).on_missing (
str) – Input channels whose name has no pretrained slot:"error"raises,"drop"leaves them out of the input (with a warning), so the tokenizer encodes and reconstructs the channels it knows. Channels are never mapped by guesswork; a montage with no known channel still raises.max_patches (
int|None) – Length of the temporal embedding table;Noneuses the modality’s value.out_chans (
int) – Number of channels per temporal-convolution branch; must be divisible by four. The Transformer width isout_chans * patch_size // 8.encoder_depth (
int) – Number of shared encoder Transformer blocks.decoder_depth (
int) – Number of shared decoder Transformer blocks.num_heads (
int) – Number of attention heads in both Transformer stacks.n_code (
int) – Number of entries per EMA codebook.code_dim (
int) – Dimension of each quantized latent vector.num_quantizers (
int|None) – Number of residual codebooks per temporal scale (EMG 16, the others 8);Noneuses the modality’s value.drop_prob (
float) – Dropout probability in the Transformer stacks.attn_drop_rate (
float) – Attention-probability dropout.drop_path_rate (
float) – Maximum stochastic-depth probability.init_values (
float) – LayerScale initialization used by the tokenizer checkpoint.activation (
type[Module]) – Activation in the temporal patch embedding.statistic_code_usage (
bool) –Truelets eval forwards update thecluster_sizecode-usage EMA, as the authors’ code.
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
Output shape:
forwardreturns the target and reconstruction patches, each z-scored per window, shaped(batch, n_chans * n_patches, patch_size); their mean squared difference is the reconstruction error.tokenizereturns integer codes shaped(4, num_quantizers, batch, n_chans * n_patches). Withon_missing="drop",n_chanscounts the kept channels.Examples
>>> import torch >>> from braindecode.models import NeuroRVQTokenizer >>> model = NeuroRVQTokenizer( ... n_chans=3, n_times=800, sfreq=200, channel_names=["C3", "Cz", "C4"] ... ).eval() >>> x = torch.randn(2, 3, 800) # (batch, channels, 4 s at 200 Hz) >>> target, reconstruction = model(x) >>> target.shape # (batch, n_chans * n_patches, patch_size) torch.Size([2, 12, 200]) >>> mse = (target - reconstruction).square().mean() >>> model.tokenize(x).shape # (scale, quantizer, batch, n_chans * n_patches) torch.Size([4, 8, 2, 12])
References
[neurorvq] (1,2)Barmpas et al. (2025). NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models. https://arxiv.org/abs/2510.13068
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import NeuroRVQTokenizer # Train your model model = NeuroRVQTokenizer(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-neurorvqtokenizer-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import NeuroRVQTokenizer # Load pretrained model model = NeuroRVQTokenizer.from_pretrained("username/my-neurorvqtokenizer-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = NeuroRVQTokenizer.from_pretrained("username/my-neurorvqtokenizer-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = NeuroRVQTokenizer.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods