braindecode.models.GeoNIRS#

class braindecode.models.GeoNIRS(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=10, max_patches=15, on_non_divisible='error', embed_dim=192, num_heads=8, depth=4, mlp_ratio=4.0, geometry_hidden_dim=None, geometry_heads=None, n_latent_channels=16, compressor_heads=None, feature_dim=64, use_channel_geometry=True, pool='flatten', channel_order=None, drop_prob=0.1, head_drop_prob=0.1, minimum_patch_valid_fraction=0.8, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#

GeoNIRS from Li et al. (2026).

Attention/Transformer Channel Foundation Model

GeoNIRS is an fNIRS foundation model pretrained on unlabelled recordings with different probe layouts [1]. Two things make such pretraining hard (§1): “channel semantics vary across probe layouts: the same channel index may correspond to different cortical locations”, and “Slowly varying hemodynamic signals make within-signal masked reconstruction susceptible to temporal interpolation.”

GeoNIRS answers the first with geometry: it “uses the physical locations of source-detector pairs as a shared spatial reference across heterogeneous probe layouts” and “compresses variable-size channel sets into a fixed-size latent representation”. It answers the second by rebuilding one chromophore from the other (“bidirectional HbO/HbR cross-reconstruction with contiguous temporal masking”). This class is the pretrained encoder of Figure 1 of the paper (arXiv:2610.17845) with the paper’s linear head; the pretraining decoders are not part of it.

Architecture Overview

For \(C\) HbO/HbR pairs at 5 Hz, cut into \(P\) patches of \(S\) samples:

x                                         (batch, 2C, n_times)
- temporal encoders, one per chromophore  -> 2 x (batch, C, P, D)
- geometry encoder                        -> (batch, C, D)
- channel compressor, M latent queries    -> 2 x (batch, M, P, D)
- chromophore projections, D -> F         -> (batch, 2, M, P, F)
- read-out: pool="flatten"                -> (batch, 2 * M * P * F)
            pool="mean_std"               -> (batch, 4 * M * F)
- head_dropout, final_layer               -> (batch, n_outputs)

GeoNIRS-Base on a 12 s window (n_times=60) gives tokens of shape (batch, 2, 16, 6, 64), read out as 12288 or 4096 features.

Macro Components

  • GeoNIRS.hbo_encoder, GeoNIRS.hbr_encoder Temporal encoders (§2.2)

    “Every patch is projected into a \(D\)-dimensional token and augmented with a learnable temporal positional embedding. Separate HbO and HbR temporal encoders then model the token sequence within each physical channel”. Inside: a linear projection, LayerNorm, an embedding of the patch index and a pre-norm Transformer over the patches of one channel. All channels share the weights, so “the model does not require a fixed \(C_d\)”.

  • GeoNIRS.geometry_encoder Geometry encoder (Appendix B.2)

    “Each physical channel \(c\) is represented by a six-dimensional geometry vector” of its source \(s_c\) and detector \(d_c\):

    \[g_c = [\mathrm{mid}_c, \Delta_c], \qquad \mathrm{mid}_c = \frac{s_c + d_c}{2}, \qquad \Delta_c = d_c - s_c.\]

    The midpoint gives “the spatial location of the channel” and \(\Delta_c\) “the source-detector separation and orientation”. Coordinates are centred and divided by “a fixed reference scale of 25.46 mm”; an MLP and one Transformer layer turn them into \(G\).

  • GeoNIRS.channel_compressor Geometry-aware channel compressor (§2.3)

    “To transform variable-size physical channel sets into fixed-length representations, we introduce \(M\) learnable latent queries”, \(L\). For each patch \(p\) they attend over the channels, with keys made of the two chromophores fused and the geometry:

    \[\bar{H}_p = \tfrac{1}{2}\left(H^O_{:,p} + H^R_{:,p}\right), \quad Q = W_Q\,\mathrm{LN}(L), \quad K_p = W_K\,\mathrm{LN}\left(\bar{H}_p + G\right), \quad A_p = \mathrm{softmax}\left(\frac{Q K_p^\top}{\sqrt{d_h}}\right), \quad Z^s_p = A_p\,W_V\,\mathrm{LN}\left(H^s_{:,p}\right), \quad s \in \{O, R\}.\]

    “HbO and HbR share the same attention weights but retain separate value representations”, and “Invalid channels are masked before the softmax operation.” An output projection and a feed-forward block follow.

  • GeoNIRS.hbo_projection, GeoNIRS.hbr_projection Chromophore projections (§2.4)

    “a chromophore-specific residual projection that maps the feature dimension from \(D\) to \(F\) without mixing latent channels or temporal patches”: three residual blocks, of widths [192, 128, 96, 64] for GeoNIRS-Base (Table 6).

  • GeoNIRS.final_layer Head (§2.5, Appendix B.5.2)

    The tokens of the two chromophores are “flattened, standardized using training-set statistics, followed by dropout (\(p = 0.1\)) and a single linear classification layer”. pool="flatten" is that head without the standardization; pool="mean_std" reads out the mean and standard deviation of \([Z^O \,\|\, Z^R]\) over the valid patches.

Temporal, Spatial, and Spectral Encoding

  • Temporal. A patch is \(S = 10\) samples, 2 s at 5 Hz, and the positional embedding has 15 rows, the 30 s pretraining window. The compressor keeps the patch axis: \(M\) tokens per 2 s patch.

  • Spatial. A channel has no index embedding. It is known only through its geometry \(g_c\), in the keys of the compressor, so any number of channels in any layout maps to the same \(M\) latent channels. Zeroing or shuffling the geometry at inference lowers the task-averaged balanced accuracy by 2.34 and 3.24 points (Table 3).

  • Chromophores. The rows are HbO and HbR concentration changes (no HbT), each with its own temporal encoder and projection. The encoder has no frequency decomposition.

  • Validity masks. “Samples and channels rejected during preprocessing quality control, together with patches containing insufficient valid samples, are excluded using validity masks” (§2.2). Here a non-finite sample is missing, a patch needs minimum_patch_valid_fraction of its samples, and ch_mask marks padded pairs.

Pretrained Weights

The authors release both backbones in the checkpoint/ folder (Git LFS) of GeoNIRS/GeoNIRS@81f8995, geonirs_base.pt and geonirs_large.pt. Both use \(M = 16\) and \(S = 10\) (Table 6); the defaults build GeoNIRS-Base.

Component

GeoNIRS-Base

GeoNIRS-Large

Pretraining corpus

8 datasets, 383.30 h

22 datasets, 827.04 h

embed_dim (\(D\))

192

384

depth / num_heads

4 / 8

6 / 12

feature_dim (\(F\))

64

128

Parameters, without the head

4,443,462

24,394,310

load_state_dict() takes the file as released: it drops the pretraining decoders and keeps the head’s initialisation. The weights expect 5 Hz signals standardized per recording and at most 150 samples.

import torch
from braindecode.models import GeoNIRS

payload = torch.load("geonirs_base.pt", map_location="cpu", weights_only=True)
model = GeoNIRS(chs_info=raw.info["chs"], n_outputs=2, n_times=60, sfreq=5.0)
model.load_state_dict(payload)

Differences from the reference implementation

  • Input. One (batch, 2C, n_times) tensor, stacked or interleaved (the MNE layout), instead of a batch of hbo, hbr, validity masks and geometry. A NaN marks a sample the reference zero-fills and flags.

  • Geometry. Read from chs_info (loc[3:9]), centred and scaled as the reference’s scripts/build_fair_input.py; a pair without positions gets a zero geometry. forward(geometry=...) replaces it.

  • Head. The training-set standardization is left to the caller. pool="mean_std" gives the features of the reference’s mean_std_linear head.

  • Added: on_non_divisible="pad" / "crop" and ch_mask. Left out: the decoders, the temporal masking and LoRA.

Added in version 2.1.

Parameters:
  • n_outputs (int | None) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int | None) – Number of EEG channels.

  • chs_info (list[dict] | None) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int | None) – Number of time samples of the input window.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency of the EEG recordings.

  • patch_size (int) – Samples per patch, \(S\) in the paper: 2 s at 5 Hz.

  • max_patches (int) – Rows of the temporal positional embedding, the 15 patches of a 30 s pretraining window; raised to the patch count of n_times.

  • on_non_divisible (Literal['error', 'pad', 'crop']) – n_times not a multiple of patch_size: refuse (as the reference); pad with missing samples, to which the validity rule applies; or crop.

  • embed_dim (int) – Token width, \(D\) in the paper (“Embedding width”, Table 6).

  • num_heads (int) – Heads of the temporal Transformers (Table 6).

  • depth (int) – Layers of each temporal Transformer (Table 6).

  • mlp_ratio (float) – Feed-forward width of the temporal Transformers, over embed_dim.

  • geometry_hidden_dim (int | None) – Hidden width of the geometry MLP; None is embed_dim (Table 6).

  • geometry_heads (int | None) – Heads of the geometry Transformer layer; None is num_heads.

  • n_latent_channels (int) – Learnable latent queries, \(M\) in the paper.

  • compressor_heads (int | None) – Heads of the compressor’s attention; None is num_heads.

  • feature_dim (int) – Token width after the projection, \(F\) in the paper (Table 6).

  • use_channel_geometry (bool) – Add the geometry \(G\) to the keys of the compressor. False builds no geometry encoder: the “w/o Geometry” variant of Table 4.

  • pool (Literal['flatten', 'mean_std']) – Read-out of the (batch, 2, M, P, F) tokens: all of them flattened (the head of the paper), or their mean and standard deviation over the valid patches.

  • channel_order (Optional[Literal['stacked', 'interleaved']]) – [HbO_1..HbO_C, HbR_1..HbR_C] or [HbO_1, HbR_1, ...]; None reads it from chs_info (stacked without it).

  • drop_prob (float) – Dropout in the backbone, 0.1 in both checkpoints.

  • head_drop_prob (float) – Dropout before final_layer, \(p = 0.1\) in Appendix B.5.2.

  • minimum_patch_valid_fraction (float) – Share of measured samples a patch needs to be valid: “at least 80% of its time points” (Appendix B.1.2).

  • activation (type[Module]) – Activation of the backbone; GELU in the reference.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

Protocol. For pretraining (Appendix B.1), raw intensity is converted to HbO and HbR concentration changes with the modified Beer-Lambert law, resampled to 5 Hz and standardized per recording, HbO and HbR separately, then cut into 30 s windows; B.1.1 lists no band-pass filter. Downstream (§3.1, Appendix B.5-B.6), eight held-out binary tasks in windows of 12 s or 10 s are scored by subject-level five-fold cross-validation with seeds 810 and 811. With linear probing, LoRA and full fine-tuning GeoNIRS reaches 63.06, 63.42 and 63.54 task-averaged balanced accuracy (Table 1).

Replication. Linear probe of the frozen GeoNIRS-Base on DS42 (action mode, ds005935) and DS19 (pain intensity, ds005777), with the paper’s folds and seeds. “Authors’ pipeline” is their repository at 81f8995 run end to end; “Port” is this class with a re-implementation of their probe.

Task

Metric

Paper

Authors’ pipeline

Port

DS42

macro-F1

0.7018

0.7000

0.7039

DS42

AUROC

0.7840

0.7840

0.7840

DS19

macro-F1

0.5598

0.5599

0.5304

DS19

AUROC

0.5683

0.5687

0.5549

Both pipelines build the same windows (1628 and 1190), on which this class gives the authors’ encoder features (max-abs difference 0.0, Base and Large, eval mode).

References

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import GeoNIRS

# Train your model
model = GeoNIRS(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-geonirs-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import GeoNIRS

# Load pretrained model
model = GeoNIRS.from_pretrained("username/my-geonirs-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = GeoNIRS.from_pretrained("username/my-geonirs-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = GeoNIRS.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

encode(x, geometry=None, ch_mask=None)[source]#

Encode a window into the latent tokens of the two chromophores.

The backbone of the paper: temporal encoders, geometry encoder, channel compressor and chromophore projections.

Parameters:
  • x (Tensor) – HbO/HbR signals of shape (batch, n_chans, n_times). A NaN marks a missing sample.

  • geometry (Optional[Tensor]) – Geometry vector \(g_c\) of each pair, (n_pairs, 6) or (batch, n_pairs, 6). None takes the one of chs_info. Not read with use_channel_geometry=False.

  • ch_mask (Optional[Tensor]) – (batch, n_chans) mask of the real channels of a padded batch; it must pad whole HbO/HbR pairs.

Returns:

"foundation": \(Z^O\) and \(Z^R\), (batch, 2, n_latent_channels, n_patches, feature_dim). "valid": boolean (batch, n_latent_channels, n_patches), False for the patches that have no valid channel (their tokens are zero).

Return type:

dict[str, Tensor]

forward(x, return_features=False, geometry=None, ch_mask=None)[source]#

Forward pass.

Parameters:
  • x (Tensor) – HbO/HbR signals of shape (batch, n_chans, n_times). A NaN marks a missing sample.

  • return_features (bool) – Return the features the head reads instead of the outputs.

  • geometry (Optional[Tensor]) – Geometry vector of each pair, (n_pairs, 6) or (batch, n_pairs, 6); see encode().

  • ch_mask (Optional[Tensor]) – (batch, n_chans) mask of the real channels of a padded batch; see encode().

Returns:

Outputs of shape (batch, n_outputs). With return_features=True, {"features": Tensor, "cls_token": None} with the (batch, n_features) input of final_layer.

Return type:

Tensor | dict[str, Tensor | None]

load_state_dict(state_dict, strict=True, assign=False)[source]#

Load a state dict, or a checkpoint as the authors release it.

Parameters:
Returns:

The missing and unexpected keys.

Return type:

NamedTuple

reset_head(n_outputs)[source]#

Replace final_layer by a new one with n_outputs outputs.

Parameters:

n_outputs (int) – New number of outputs.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.