braindecode.models.NeuroRVQ#

class braindecode.models.NeuroRVQ(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, channel_names=None, modality='eeg', patch_size=None, on_non_divisible='pad', on_missing='error', max_patches=None, depth=12, num_heads=10, mlp_ratio=4.0, out_chans=8, qkv_bias=True, qk_norm=True, init_values=1e-05, drop_prob=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#

NeuroRVQ foundation model for EEG, ECG and EMG from Barmpas et al. [neurorvq].

Convolution Attention/Transformer Foundation Model Channel

The model combines four temporal convolution scales with a shared Transformer encoder and learned channel/time embeddings; this class adapts the pretrained encoder to Braindecode’s window classifier API.

Inputs are (batch, channels, time), sampled at the modality’s rate (EEG 200 Hz, ECG 200 Hz, EMG 1000 Hz, PPG 100 Hz), made of whole patches (EEG 200, ECG 40, EMG 200, PPG 80 samples), with channel names from the modality’s pretrained list. This model does not preprocess signals; the authors’ EEG example applies a 0.5–45 Hz band-pass, resamples to 200 Hz and clips amplitudes to +/-500.

The upstream implementation [neurorvqcode] and checkpoints are licensed CC BY-NC 4.0. This is a non-commercial research license.

License

Important

Pre-trained Weights Available

The released foundation models (ntinosbarmpas/NeuroRVQ, revision d944b87, CC BY-NC 4.0) are hosted as braindecode/neurorvq-eeg-pretrained, braindecode/neurorvq-ecg-pretrained and braindecode/neurorvq-emg-pretrained; the tokenizers are in braindecode/neurorvq-tokenizer-{eeg,ecg,emg,ppg}-pretrained (NeuroRVQTokenizer). The classification head is not pretrained:

from braindecode.models import NeuroRVQ

model = NeuroRVQ.from_pretrained(
    "braindecode/neurorvq-eeg-pretrained", chs_info=raw.info["chs"], n_outputs=4
)
Parameters:

  • n_outputs (int | None) – Number of task-specific output classes.

  • n_chans (int | None) – Number of channels in each input window.

  • chs_info (list of dict or None) – MNE channel metadata. Channel names are used to select the pretrained spatial embedding slots when channel_names is not provided.

  • n_times (int | None) – Number of input time samples. It gives n_patches patches (ceil(n_times / patch_size) by default, see on_non_divisible), at most max_patches.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency. An integer rate other than the modality’s (EEG 200 Hz) is resampled to it inside the model with torchaudio.transforms.Resample, with a warning; the patch count follows the resampled length.

  • channel_names (tuple[str, ...] | list[str] | None) – Ordered channel names. Names are case-insensitive and must occur in the modality’s pretrained list (EEG: 104 electrodes; ECG: the 12 standard leads and vx, vy, vz; EMG: c1-c16; PPG: ppg_c1). If omitted, names are read from chs_info; without either, the first entries of that list are used, which only suits training from scratch.

  • modality (str) – Released configuration: sampling rate, temporal kernel sizes, channel list, head pooling (EEG flattens the tokens, the others average them) and the defaults of patch_size and max_patches. No PPG foundation model was released.

  • patch_size (int | None) – Samples per temporal patch; None uses the modality’s value.

  • on_non_divisible (str) – How PatchTokenizer treats a time axis that is not a multiple of patch_size (right zero-pad, crop, or raise). The reference implementation requires a multiple ("error"); all three give the same model and output when n_times is a multiple of patch_size.

  • on_missing (str) – Input channels whose name has no pretrained slot: "error" raises, "drop" leaves them out of the input (with a warning), so the model reads the channels it knows. Channels are never mapped by guesswork.

  • max_patches (int | None) – Length of the temporal embedding table (EEG 256, ECG 600, EMG 256, PPG 12); None uses the modality’s value.

  • depth (int) – Number of shared Transformer blocks applied to each temporal branch.

  • num_heads (int) – Number of attention heads.

  • mlp_ratio (float) – Transformer feed-forward expansion ratio.

  • out_chans (int) – Channels per temporal convolution branch. Must be divisible by four.

  • qkv_bias (bool) – Whether to use the released query/value attention biases.

  • qk_norm (bool) – Whether to normalize query and key vectors with LayerNorm.

  • init_values (float) – LayerScale initialization value used by the released EEG checkpoint.

  • drop_prob (float) – Dropout probability in the patch embedding, feed-forward blocks, and positional embedding path.

  • attn_drop_rate (float) – Attention-probability dropout.

  • drop_path_rate (float) – Maximum stochastic-depth probability across Transformer depth.

  • activation (type[Module]) – Activation in the multi-scale temporal patch embedding. GELU reproduces the released model.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

Input shape: (batch, n_chans, n_times).

Output shape: by default, (batch, n_outputs) logits. With return_features=True, returns a dictionary whose "features" entry has shape (batch, 4 * embed_dim * n_chans * n_patches) for EEG and (batch, 4 * embed_dim) for the other modalities, where embed_dim = out_chans * patch_size // 8.

Added in version 2.0.

References

[neurorvq]

Barmpas et al. (2025). NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models. https://arxiv.org/abs/2510.13068

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import NeuroRVQ

# Train your model
model = NeuroRVQ(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-neurorvq-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import NeuroRVQ

# Load pretrained model
model = NeuroRVQ.from_pretrained("username/my-neurorvq-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = NeuroRVQ.from_pretrained("username/my-neurorvq-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = NeuroRVQ.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x, return_features=False)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

  • x (Tensor) – The description is missing.

  • return_features (bool) – The description is missing.

Return type:

Union[Tensor, Dict[str, Optional[Tensor]]]

reset_head(n_outputs)[source]#

Replace the classification head for a new number of outputs.

This is called automatically by from_pretrained() when the user passes an n_outputs that differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync with self._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also use self._set_n_outputs to validate and record the new value.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.