braindecode.models.NeuroRVQ#

class braindecode.models.NeuroRVQ(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, channel_names=None, modality='eeg', patch_size=None, max_patches=None, depth=12, num_heads=10, mlp_ratio=4.0, out_chans=8, qkv_bias=True, qk_norm=True, init_values=1e-05, drop_prob=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#

NeuroRVQ foundation model for EEG, ECG and EMG from Barmpas et al. [neurorvq].

Convolution Attention/Transformer Foundation Model Channel

The model combines four temporal convolution scales with a shared Transformer encoder and learned channel/time embeddings; this class adapts the pretrained encoder to Braindecode’s window classifier API.

Inputs are (batch, channels, time), sampled at the modality’s rate (EEG 200 Hz, ECG 200 Hz, EMG 1000 Hz, PPG 100 Hz), made of whole patches (EEG 200, ECG 40, EMG 200, PPG 80 samples), with channel names from the modality’s pretrained list. This model does not preprocess signals; the authors’ EEG example applies a 0.5–45 Hz band-pass, resamples to 200 Hz and clips amplitudes to +/-500.

The upstream implementation and checkpoints are licensed CC BY-NC 4.0. This is a non-commercial research license.

License

Important

Pre-trained Weights Available

The released foundation models (ntinosbarmpas/NeuroRVQ, revision d944b87, CC BY-NC 4.0) are hosted as braindecode/neurorvq-eeg-pretrained, braindecode/neurorvq-ecg-pretrained and braindecode/neurorvq-emg-pretrained; the tokenizers are in braindecode/neurorvq-tokenizer-{eeg,ecg,emg,ppg}-pretrained (NeuroRVQTokenizer). The classification head is not pretrained:

from braindecode.models import NeuroRVQ

model = NeuroRVQ.from_pretrained(
    "braindecode/neurorvq-eeg-pretrained", chs_info=raw.info["chs"], n_outputs=4
)
Parameters:

  • n_outputs (int | None) – Number of task-specific output classes.

  • n_chans (int | None) – Number of channels in each input window.

  • chs_info (list of dict or None) – MNE channel metadata. Channel names are used to select the pretrained spatial embedding slots when channel_names is not provided.

  • n_times (int | None) – Number of input time samples. Must be divisible by patch_size and no greater than patch_size * max_patches.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency. If provided or inferable, it must be the modality’s rate.

  • channel_names (tuple[str, ...] | list[str] | None) – Ordered channel names. Names are case-insensitive and must occur in the modality’s pretrained list (EEG: 104 electrodes; ECG: the 12 standard leads and vx, vy, vz; EMG: c1-c16; PPG: ppg_c1). If omitted, names are read from chs_info; without either, the first entries of that list are used, which only suits training from scratch.

  • modality (str) – Released configuration: sampling rate, temporal kernel sizes, channel list, head pooling (EEG flattens the tokens, the others average them) and the defaults of patch_size and max_patches. No PPG foundation model was released.

  • patch_size (int | None) – Samples per temporal patch; None uses the modality’s value.

  • max_patches (int | None) – Length of the temporal embedding table (EEG 256, ECG 600, EMG 256, PPG 12); None uses the modality’s value.

  • depth (int) – Number of shared Transformer blocks applied to each temporal branch.

  • num_heads (int) – Number of attention heads.

  • mlp_ratio (float) – Transformer feed-forward expansion ratio.

  • out_chans (int) – Channels per temporal convolution branch. Must be divisible by four.

  • qkv_bias (bool) – Whether to use the released query/value attention biases.

  • qk_norm (bool) – Whether to normalize query and key vectors with LayerNorm.

  • init_values (float) – LayerScale initialization value used by the released EEG checkpoint.

  • drop_prob (float) – Dropout probability in the patch embedding, feed-forward blocks, and positional embedding path.

  • attn_drop_rate (float) – Attention-probability dropout.

  • drop_path_rate (float) – Maximum stochastic-depth probability across Transformer depth.

  • activation (type[Module]) – Activation in the multi-scale temporal patch embedding. GELU reproduces the released model.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.

Methods

forward(x, return_features=False)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

  • x (Tensor) – The description is missing.

  • return_features (bool) – The description is missing.

reset_head(n_outputs)[source]#

Replace the classification head for a new number of outputs.

This is called automatically by from_pretrained() when the user passes an n_outputs that differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync with self._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also use self._set_n_outputs to validate and record the new value.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.