braindecode.models.CORTEG#

class braindecode.models.CORTEG(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, patch_size=16, hi_patch_size=25, embed_dim=512, depth=8, num_heads=8, mlp_ratio=4, drop_prob=0.0, drop_path=0.1, activation=<class 'torch.nn.modules.activation.GELU'>, n_chans_pos=145, merge_strategy='average', n_merge_blocks=4, knn_k=8, adapter_hidden=128, n_fourier_freqs=32, fourier_max_freq=50.0, gate_bottleneck=16, on_non_divisible='pad')[source]#

CORTEG from Yang et al. (2026) [Yang2026b].

Attention/Transformer Foundation Model Channel

https://raw.githubusercontent.com/LiuyinYang1101/CORTEG/main/docs/assets/figure1_overview_scheme.png

CORTEG overview, reproduced from the official repository (LiuyinYang1101/CORTEG).#

Added in version 2.1.

Architecture Overview

A scalp-EEG foundation model, STEEGFormer, adapted to intracranial (ECoG) recordings. The transformer stays the pretrained one; around it, a spatial adapter places each electrode by its 3D position, and a second token stream carries the high-gamma activity the scalp model was never trained on. The paper trains only the spatial adapter, the high-gamma patch embedding, LoRA on the last n_merge_blocks blocks with their LayerNorms, and the head (about 297 K parameters for the small backbone); this module builds the network, and LoRA and freezing are left to the training code.

Macro Components

  • Dual-stream tokenizer (CORTEG.patch_embed, CORTEG.patch_embed_hi). Operations: the low-frequency stream x is cut into patches of patch_size samples, the high-gamma stream x_hi into patches of hi_patch_size, each linearly embedded and the high-gamma tokens average-pooled to the low-frequency count. The high-gamma embedding starts from the pretrained low-frequency one, linearly interpolated to its patch size.

  • KNNSoftFourier spatial adapter (CORTEG.channel_adapter). Operations: a soft branch maps each electrode position to softmax weights over the 142 positioned slots of the pretrained channel embedding (CORTEG.channel_pos, kept frozen) and returns their weighted sum; the last bias starts from a Gaussian knn_k-nearest- neighbour kernel over the slots’ scalp positions, so the soft branch starts as a nearest-electrode interpolation. A Fourier residual (n_fourier_freqs log-spaced frequencies per axis, up to fourier_max_freq, zero-initialized output) adds a position-specific correction. Role: replace the scalp channel embedding, which knows no intracranial electrode.

  • Transformer encoder (CORTEG.encoder). Operations: the depth pretrained pre-norm ViT blocks of ST-EEGFormer, over a CLS token and the (time-patch, channel) tokens of the low-frequency stream.

  • Late fusion (merge_strategy). Operations: "average" replaces the low-frequency tokens by their mean with the high-gamma tokens before the last n_merge_blocks blocks; "layerwise_gate" adds the high-gamma tokens after every block, scaled per block and per sample by a small gate network that starts at zero.

  • Read-out (CORTEG.norm, CORTEG.final_layer). Operations: layer norm, mean over all tokens (CLS included), linear head.

Inputs

The paper decomposes the raw signal into two streams, both z-scored per channel with training-set statistics:

  • x: the low-frequency stream, resampled to 128 Hz (the 16-sample patches of the pretrained model then span 125 ms);

  • x_hi: the 70-200 Hz Hilbert envelope at 200 Hz, so that its 25-sample patches line up with those of x.

With MNE, raw.copy().filter(70, 200).apply_hilbert(envelope=True) .resample(200) gives x_hi. Without x_hi the model reads x alone, as the paper’s low-frequency-only ablation.

Electrode positions are 3D coordinates in metres (the paper’s MNI coordinates), passed as pos or read from chs_info.

Pre-trained weights

The ST-EEGFormer small backbone, with the high-gamma embedding warm-started from its low-frequency one, loads from the re-hosted checkpoint; the adapter, gate and head keep their initialization:

model = CORTEG.from_pretrained(
    "braindecode/corteg-small-pretrained", n_outputs=5, n_times=128,
    chs_info=chs_info,
)

The authors’ Stanford model (finger flexion, nine patients pooled) is re-hosted with its LoRA factors folded in:

model = CORTEG.from_pretrained(
    "braindecode/corteg-stanford-pooled", chs_info=chs_info
)

Note

The outputs match the reference implementation for the same weights and inputs, for both merge strategies. ch_mask is an addition: the reference batches one patient at a time.

Parameters:

  • n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int) – Number of EEG channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int) – Number of time samples of the input window.

  • input_window_seconds (float) – Length of the input window in seconds.

  • sfreq (float) – Sampling frequency of the EEG recordings.

  • patch_size (int) – Patch size of the low-frequency stream (16 for the released backbones).

  • hi_patch_size (int) – Patch size of the high-gamma stream.

  • embed_dim (int) – Token embedding dimension (512 for the small backbone).

  • depth (int) – Number of Transformer encoder blocks.

  • num_heads (int) – Number of attention heads.

  • mlp_ratio (int) – Hidden-to-embedding ratio of the MLP blocks.

  • drop_prob (float) – Dropout rate.

  • drop_path (float) – Stochastic-depth rate (max of a linear schedule over depth).

  • activation (type[Module]) – Activation layer class of the feed-forward blocks and of the MLPs of the spatial adapter and the gate.

  • n_chans_pos (int) – Size of the pretrained channel vocabulary (145 for small/base/large).

  • merge_strategy (str) – Fusion of the high-gamma stream: the paper’s default mean before the last n_merge_blocks blocks (the released Stanford adapter), or the layer-wise gate (its BrainTreebank results).

  • n_merge_blocks (int) – Number of blocks after the "average" fusion.

  • knn_k (int) – Number of nearest scalp slots of the soft branch’s initial kernel.

  • adapter_hidden (int) – Hidden width of the spatial adapter’s MLPs.

  • n_fourier_freqs (int) – Number of Fourier frequencies per axis of the residual branch.

  • fourier_max_freq (float) – Highest of those frequencies, log-spaced from 1, in radians per metre.

  • gate_bottleneck (int) – Hidden width of the layer-wise gate.

  • on_non_divisible (str) – How a stream whose length is not a multiple of its patch size is handled (right zero-pad, crop, or raise); the reference crops.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.

References

[Yang2026b]

Yang, L., Sun, Q., Van Dyck, B., Calvo Merino, E. & Van Hulle, M. M. (2026). CORTEG: Foundation models enable cross-modality representation transfer from scalp to intracranial brain recordings. Advances in Neural Information Processing Systems (NeurIPS 2026). https://arxiv.org/abs/2605.10337

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import CORTEG

# Train your model
model = CORTEG(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-corteg-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import CORTEG

# Load pretrained model
model = CORTEG.from_pretrained("username/my-corteg-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = CORTEG.from_pretrained("username/my-corteg-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = CORTEG.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x, x_hi=None, pos=None, ch_mask=None, return_features=False)[source]#

Encode a batch of intracranial recordings.

Parameters:

  • x (Tensor) – Low-frequency stream, (batch, n_chans, n_times).

  • x_hi (Optional[Tensor]) – High-gamma stream of the same channels, (batch, n_chans, n_times_hi). Without it the model reads x alone.

  • pos (Optional[Tensor]) – Electrode positions in metres, (batch, n_chans, 3) or (n_chans, 3); by default those of chs_info.

  • ch_mask (Optional[Tensor]) – (batch, n_chans) mask of real channels, see check_ch_mask(). Padded channels take part in no attention and no pooling.

  • return_features (bool) – If True, return {"features", "cls_token", "tokens"}: the layer-normalized patch tokens (batch, n_patches * n_chans, embed_dim), the CLS token, and the patch tokens as (batch, n_chans, n_patches, embed_dim); padded channels’ tokens are zero.

Returns:

Outputs (batch, n_outputs), or the feature dict.

Return type:

Union[Tensor, Dict[str, Tensor]]

reset_head(n_outputs)[source]#

Replace the linear head for a new n_outputs.

Parameters:

n_outputs (int) – New number of output classes.

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.