braindecode.models.EEGCLIP#

class braindecode.models.EEGCLIP(n_outputs=None, n_chans=None, n_times=None, text_encoder=None, text_embedding_dim=768, eeg_encoder=None, eeg_embedding_dim=128, text_pooling='cls', projection_layers=3, activation=<class 'torch.nn.modules.activation.ReLU'>, drop_prob=0.1, initial_temperature=0.07, chs_info=None, input_window_seconds=None, sfreq=None)[source]#

Dual encoder for contrastive alignment of EEG and text [eegclip].

Foundation Model Convolution

EEG-CLIP overview (N'dir et al., 2025, Fig. 1).

EEG-CLIP learns a shared embedding space for paired EEG recordings and clinical text reports with a symmetric contrastive objective. The default EEG encoder is a dense-prediction Deep4Net whose 128 log-softmax outputs per time step are projected and averaged over time, as in the authors’ code. The text encoder is any torch.nn.Module (e.g. a Hugging Face ClinicalBERT, kept as an optional user dependency); with text_encoder=None the text inputs are precomputed features.

forward(X) returns the projected EEG embeddings only, so the model keeps braindecode’s Tensor-valued contract. Train with forward_paired() and contrastive_loss(); score zero-shot with compute_logits() against embedded candidate descriptions.

Parameters:

  • n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int) – Number of EEG channels.

  • n_times (int) – Number of time samples of the input window.

  • text_encoder (torch.nn.Module | None) – Text encoder called as text_encoder(text_inputs, **kwargs). It returns a tensor, or a tuple / Hugging Face output whose first element is the tensor. None treats text_inputs as precomputed features.

  • text_embedding_dim (int) – Dimension of the pooled text features.

  • eeg_encoder (torch.nn.Module | None) – EEG encoder returning (batch, features) or (batch, features, time). None builds the default Deep4Net.

  • eeg_embedding_dim (int) – Feature dimension of the EEG encoder output.

  • text_pooling ({"cls", "mean"}) – Pooling of token-sequence text outputs: first token, or the mean over tokens (masked when attention_mask is given).

  • projection_layers (int) – Linear layers in each projection head; 3 as in the paper.

  • activation (type[Module]) – Activation of the non-final projection blocks.

  • drop_prob (float) – Dropout probability of the non-final projection blocks.

  • initial_temperature (float) – The logit scale starts at log(1 / initial_temperature) and, as in the authors’ code, multiplies the similarities without exp.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • input_window_seconds (float) – Length of the input window in seconds.

  • sfreq (float) – Sampling frequency of the EEG recordings.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

n_outputs is the shared embedding dimension, not a number of classes. The released modelsexample.ckpt predates the authors’ current projection head and is not loadable. Models with a custom eeg_encoder or text_encoder cannot be rebuilt from a config, so get_config() and save_pretrained raise for them; save their state_dict.

The authors’ repository has no license file; this implementation follows braindecode’s BSD-3-Clause license, and its projection head mirrors the authors’ five-line ProjectionHead.

Examples

Train on paired EEG windows and precomputed text features:

import torch
from braindecode.models import EEGCLIP

model = EEGCLIP(
    n_chans=21, n_times=1000, n_outputs=64, text_embedding_dim=768
)
eeg_windows = torch.randn(2, 21, 1000)
text_features = torch.randn(2, 768)
paired = model.forward_paired(eeg_windows, text_features)
loss = model.contrastive_loss(
    paired["eeg_embeds"], paired["text_embeds"]
)

References

[eegclip]

N’dir, T. C., Schirrmeister, R. T., & Ball, T. (2025). EEG-CLIP: Learning EEG representations from natural language descriptions. Frontiers in Robotics and AI, 12, 1625731. https://doi.org/10.3389/frobt.2025.1625731

Added in version 1.9.

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import EEGCLIP

# Train your model
model = EEGCLIP(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-eegclip-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import EEGCLIP

# Load pretrained model
model = EEGCLIP.from_pretrained("username/my-eegclip-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = EEGCLIP.from_pretrained("username/my-eegclip-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = EEGCLIP.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

compute_logits(eeg_embeds, text_embeds)[source]#

Return EEG-to-text and text-to-EEG similarity logits.

As in the authors’ ClipLoss, embeddings are not L2-normalized and the raw logit_scale multiplies the dot products.

Parameters:

  • eeg_embeds – The description is missing.

  • text_embeds – The description is missing.

contrastive_loss(eeg_embeds, text_embeds, *, distributed=False, group=None)[source]#

Symmetric cross-entropy over a batch of paired EEG/text embeddings.

With distributed=True in an initialized process group, the pairs of every rank of group serve as negatives (ranks may hold different batch sizes), and all its ranks must call this method. Each rank’s loss is scaled so that DDP’s gradient mean gives the gradient of the loss on the concatenated batch, which equals the mean of the ranks’ losses.

Parameters:

  • eeg_embeds – The description is missing.

  • text_embeds – The description is missing.

  • distributed – The description is missing.

  • group – The description is missing.

encode_text(text_inputs, attention_mask=None, **text_kwargs)[source]#

Encode text tokens or features as shared-space projection vectors.

Parameters:

  • text_inputs – The description is missing.

  • attention_mask – The description is missing.

  • **text_kwargs – The description is missing.

forward(X)[source]#

Return projected EEG embeddings, shape (batch, n_outputs).

Parameters:

X – The description is missing.

forward_paired(X, text_inputs, attention_mask=None, **text_kwargs)[source]#

Return paired EEG/text embeddings and bidirectional similarity logits.

Parameters:

  • X – The description is missing.

  • text_inputs – The description is missing.

  • attention_mask – The description is missing.

  • **text_kwargs – The description is missing.

get_config()[source]#

Return the config; custom encoders cannot be rebuilt from one.

Returns:

All __init__ parameters, JSON-serializable. type[nn.Module] parameters (e.g. activation) are encoded as importable dotted-path strings.

Return type:

dict

Examples

>>> import json
>>> from braindecode.models import EEGNet
>>> model = EEGNet(n_chans=22, n_times=1000, n_outputs=4, F1=16)
>>> config = model.get_config()
>>> config["F1"]
16
>>> # Save to disk
>>> with open("config.json", "w") as f:
...     json.dump(config, f)

Added in version 1.4.

reset_head(n_outputs)[source]#

Rebuild both projection heads for a new shared embedding dimension.

Parameters:

n_outputs (int) – New number of output classes.

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.