braindecode.models.EEGCLIP#

class braindecode.models.EEGCLIP(n_outputs=None, n_chans=None, n_times=None, text_encoder=None, text_embedding_dim=768, eeg_encoder=None, eeg_embedding_dim=128, text_pooling='cls', projection_layers=3, activation=<class 'torch.nn.modules.activation.ReLU'>, drop_prob=0.1, initial_temperature=0.07, chs_info=None, input_window_seconds=None, sfreq=None)[source]#

Dual encoder for contrastive alignment of EEG and text [eegclip].

Foundation Model Convolution

EEG-CLIP overview (N'dir et al., 2025, Fig. 1).

EEG-CLIP learns a shared embedding space for paired EEG recordings and clinical text reports with a symmetric contrastive objective. The default EEG encoder is a dense-prediction Deep4Net whose 128 log-softmax outputs per time step are projected and averaged over time, as in the authors’ code. The text encoder is any torch.nn.Module (e.g. a Hugging Face ClinicalBERT, kept as an optional user dependency); with text_encoder=None the text inputs are precomputed features.

forward(X) returns the projected EEG embeddings only, so the model keeps braindecode’s Tensor-valued contract. Train with forward_paired() and contrastive_loss(); score zero-shot with compute_logits() against embedded candidate descriptions.

Parameters:

  • n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int) – Number of EEG channels.

  • n_times (int) – Number of time samples of the input window.

  • text_encoder (torch.nn.Module | None) – Text encoder called as text_encoder(text_inputs, **kwargs). It returns a tensor, or a tuple / Hugging Face output whose first element is the tensor. None treats text_inputs as precomputed features.

  • text_embedding_dim (int) – Dimension of the pooled text features.

  • eeg_encoder (torch.nn.Module | None) – EEG encoder returning (batch, features) or (batch, features, time). None builds the default Deep4Net.

  • eeg_embedding_dim (int) – Feature dimension of the EEG encoder output.

  • text_pooling ({"cls", "mean"}) – Pooling of token-sequence text outputs: first token, or the mean over tokens (masked when attention_mask is given).

  • projection_layers (int) – Linear layers in each projection head; 3 as in the paper.

  • activation (type[Module]) – Activation of the non-final projection blocks.

  • drop_prob (float) – Dropout probability of the non-final projection blocks.

  • initial_temperature (float) – The logit scale starts at log(1 / initial_temperature) and, as in the authors’ code, multiplies the similarities without exp.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • input_window_seconds (float) – Length of the input window in seconds.

  • sfreq (float) – Sampling frequency of the EEG recordings.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

n_outputs is the shared embedding dimension, not a number of classes. The released modelsexample.ckpt predates the authors’ current projection head and is not loadable. Models with a custom eeg_encoder or text_encoder cannot be rebuilt from a config, so get_config() and save_pretrained raise for them; save their state_dict.

The authors’ repository has no license file; this implementation follows braindecode’s BSD-3-Clause license, and its projection head mirrors the authors’ five-line ProjectionHead.

Examples

Train on paired EEG windows and precomputed text features:

import torch
from braindecode.models import EEGCLIP

model = EEGCLIP(
    n_chans=21, n_times=1000, n_outputs=64, text_embedding_dim=768
)
eeg_windows = torch.randn(2, 21, 1000)
text_features = torch.randn(2, 768)
paired = model.forward_paired(eeg_windows, text_features)
loss = model.contrastive_loss(
    paired["eeg_embeds"], paired["text_embeds"]
)

References

[eegclip]

N’dir, T. C., Schirrmeister, R. T., & Ball, T. (2025). EEG-CLIP: Learning EEG representations from natural language descriptions. Frontiers in Robotics and AI, 12, 1625731. https://doi.org/10.3389/frobt.2025.1625731

Added in version 1.9.

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import EEGCLIP

# Train your model
model = EEGCLIP(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub
model.push_to_hub(
    repo_id="username/my-eegclip-model",
    commit_message="Initial model upload",
)

Loading a model from the Hub:

from braindecode.models import EEGCLIP

# Load pretrained model
model = EEGCLIP.from_pretrained("username/my-eegclip-model")

# Load with a different number of outputs (head is rebuilt automatically)
model = EEGCLIP.from_pretrained("username/my-eegclip-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = EEGCLIP.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

compute_logits(eeg_embeds, text_embeds)[source]#

Return EEG-to-text and text-to-EEG similarity logits.

As in the authors’ ClipLoss, embeddings are not L2-normalized and the raw logit_scale multiplies the dot products.

Parameters:

  • eeg_embeds – The description is missing.

  • text_embeds – The description is missing.

contrastive_loss(eeg_embeds, text_embeds)[source]#

Symmetric cross-entropy over a batch of paired EEG/text embeddings.

Parameters:

  • eeg_embeds – The description is missing.

  • text_embeds – The description is missing.

encode_text(text_inputs, attention_mask=None, **text_kwargs)[source]#

Encode text tokens or features as shared-space projection vectors.

Parameters:

  • text_inputs – The description is missing.

  • attention_mask – The description is missing.

  • **text_kwargs – The description is missing.

forward(X)[source]#

Return projected EEG embeddings, shape (batch, n_outputs).

Parameters:

X – The description is missing.

forward_paired(X, text_inputs, attention_mask=None, **text_kwargs)[source]#

Return paired EEG/text embeddings and bidirectional similarity logits.

Parameters:

  • X – The description is missing.

  • text_inputs – The description is missing.

  • attention_mask – The description is missing.

  • **text_kwargs – The description is missing.

get_config()[source]#

Return the config; custom encoders cannot be rebuilt from one.

Returns:

All __init__ parameters, JSON-serializable. type[nn.Module] parameters (e.g. activation) are encoded as importable dotted-path strings.

Return type:

dict

Examples

>>> import json
>>> from braindecode.models import EEGNet
>>> model = EEGNet(n_chans=22, n_times=1000, n_outputs=4, F1=16)
>>> config = model.get_config()
>>> config["F1"]
16
>>> # Save to disk
>>> with open("config.json", "w") as f:
...     json.dump(config, f)

Added in version 1.4.

reset_head(n_outputs)[source]#

Rebuild both projection heads for a new shared embedding dimension.

Parameters:

n_outputs (int) – New number of output classes.

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.