braindecode.models.NeuroRVQTokenizer#

class braindecode.models.NeuroRVQTokenizer(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, channel_names=None, modality='eeg', patch_size=None, max_patches=None, out_chans=8, encoder_depth=12, decoder_depth=3, num_heads=10, n_code=8192, code_dim=128, num_quantizers=None, drop_prob=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, init_values=0.0, activation=<class 'torch.nn.modules.activation.GELU'>, statistic_code_usage=False)[source]#

NeuroRVQ multi-scale residual-vector-quantized biosignal tokenizer from Barmpas et al. [neurorvq].

Convolution Attention/Transformer Foundation Model Channel

The tokenizer encodes EEG, ECG, EMG or PPG patches through four temporal scales, quantizes each scale with a separate residual vector quantizer, and reconstructs the signal from amplitude and phase components. The Transformer blocks, temporal convolution, channel lists and modality presets are shared with NeuroRVQ.

Inputs must be sampled at the modality’s rate, contain complete patches and use channel names from the modality’s pretrained list (see NeuroRVQ). The model does not preprocess signals; the released EEG example applies notch filters at 50, 60 and 100 Hz, a 0.5-44.5 Hz band-pass, clipping at 500 uV and resampling to 200 Hz.

Important

Pre-trained Weights Available

The released tokenizers (ntinosbarmpas/NeuroRVQ, revision d944b87, CC BY-NC 4.0) are hosted as braindecode/neurorvq-tokenizer-eeg-pretrained, braindecode/neurorvq-tokenizer-ecg-pretrained, braindecode/neurorvq-tokenizer-emg-pretrained and braindecode/neurorvq-tokenizer-ppg-pretrained; the foundation models are in braindecode/neurorvq-{eeg,ecg,emg}-pretrained (NeuroRVQ):

from braindecode.models import NeuroRVQTokenizer

model = NeuroRVQTokenizer.from_pretrained(
    "braindecode/neurorvq-tokenizer-eeg-pretrained", chs_info=raw.info["chs"]
)

Raw reconstruction MSE (Table 10 of [neurorvq]), with the data preparation of the EEG-Benchmarking code linked from the NeuroRVQ repository: the port gives 0.0858 on Pavlov 2022 (paper: 0.084) and 0.0748 on High Gamma (paper: 0.090); the authors’ released code gives the same numbers on the same data.

Added in version 1.9.

License

Parameters:

  • n_outputs (int | None) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int | None) – Number of channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int | None) – Number of samples; must be divisible by patch_size and no longer than patch_size * max_patches.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency; must be the modality’s rate.

  • channel_names (tuple[str, ...] | list[str] | None) – Ordered channel names. If omitted, names are inferred from chs_info or default to the first channels in the pretrained order.

  • modality (str) – Released configuration (see NeuroRVQ); also sets the defaults of patch_size, max_patches and num_quantizers.

  • patch_size (int | None) – Samples per patch; None uses the modality’s value.

  • max_patches (int | None) – Length of the temporal embedding table; None uses the modality’s value.

  • out_chans (int) – Number of channels per temporal-convolution branch; must be divisible by four. The Transformer width is out_chans * patch_size // 8.

  • encoder_depth (int) – Number of shared encoder Transformer blocks.

  • decoder_depth (int) – Number of shared decoder Transformer blocks.

  • num_heads (int) – Number of attention heads in both Transformer stacks.

  • n_code (int) – Number of entries per EMA codebook.

  • code_dim (int) – Dimension of each quantized latent vector.

  • num_quantizers (int | None) – Number of residual codebooks per temporal scale (EMG 16, the others 8); None uses the modality’s value.

  • drop_prob (float) – Dropout probability in the Transformer stacks.

  • attn_drop_rate (float) – Attention-probability dropout.

  • drop_path_rate (float) – Maximum stochastic-depth probability.

  • init_values (float) – LayerScale initialization used by the tokenizer checkpoint.

  • activation (type[Module]) – Activation in the temporal patch embedding.

  • statistic_code_usage (bool) – True lets eval forwards update the cluster_size code-usage EMA, as the authors’ code.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.

Methods

forward(x)[source]#

Return the per-window z-scored target and reconstruction.

Parameters:

x (Tensor) – The description is missing.

tokenize(x)[source]#

Return four-scale codes without applying EMA updates.

A cold codebook is initialized once from the input vectors by cosine k-means; load pretrained weights first for pretrained codes.

Parameters:

x (Tensor) – The description is missing.

Return type:

Tensor