braindecode.models.GeoNIRS#
- class braindecode.models.GeoNIRS(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=10, max_patches=15, on_non_divisible='error', embed_dim=192, num_heads=8, depth=4, mlp_ratio=4.0, geometry_hidden_dim=None, geometry_heads=None, n_latent_channels=16, compressor_heads=None, feature_dim=64, use_channel_geometry=True, pool='flatten', channel_order=None, drop_prob=0.1, head_drop_prob=0.1, minimum_patch_valid_fraction=0.8, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#
GeoNIRS from Li et al. (2026).
Attention/Transformer Channel Foundation Model
GeoNIRS is an fNIRS foundation model pretrained on unlabelled recordings with different probe layouts [1]. Two things make such pretraining hard (§1): “channel semantics vary across probe layouts: the same channel index may correspond to different cortical locations”, and “Slowly varying hemodynamic signals make within-signal masked reconstruction susceptible to temporal interpolation.”
GeoNIRS answers the first with geometry: it “uses the physical locations of source-detector pairs as a shared spatial reference across heterogeneous probe layouts” and “compresses variable-size channel sets into a fixed-size latent representation”. It answers the second by rebuilding one chromophore from the other (“bidirectional HbO/HbR cross-reconstruction with contiguous temporal masking”). This class is the pretrained encoder of Figure 1 of the paper (arXiv:2610.17845) with the paper’s linear head; the pretraining decoders are not part of it.
Architecture Overview
For \(C\) HbO/HbR pairs at 5 Hz, cut into \(P\) patches of \(S\) samples:
x (batch, 2C, n_times) - temporal encoders, one per chromophore -> 2 x (batch, C, P, D) - geometry encoder -> (batch, C, D) - channel compressor, M latent queries -> 2 x (batch, M, P, D) - chromophore projections, D -> F -> (batch, 2, M, P, F) - read-out: pool="flatten" -> (batch, 2 * M * P * F) pool="mean_std" -> (batch, 4 * M * F) - head_dropout, final_layer -> (batch, n_outputs)
GeoNIRS-Base on a 12 s window (
n_times=60) gives tokens of shape(batch, 2, 16, 6, 64), read out as 12288 or 4096 features.Macro Components
GeoNIRS.hbo_encoder,GeoNIRS.hbr_encoderTemporal encoders (§2.2)“Every patch is projected into a \(D\)-dimensional token and augmented with a learnable temporal positional embedding. Separate HbO and HbR temporal encoders then model the token sequence within each physical channel”. Inside: a linear projection, LayerNorm, an embedding of the patch index and a pre-norm Transformer over the patches of one channel. All channels share the weights, so “the model does not require a fixed \(C_d\)”.
GeoNIRS.geometry_encoderGeometry encoder (Appendix B.2)“Each physical channel \(c\) is represented by a six-dimensional geometry vector” of its source \(s_c\) and detector \(d_c\):
\[g_c = [\mathrm{mid}_c, \Delta_c], \qquad \mathrm{mid}_c = \frac{s_c + d_c}{2}, \qquad \Delta_c = d_c - s_c.\]The midpoint gives “the spatial location of the channel” and \(\Delta_c\) “the source-detector separation and orientation”. Coordinates are centred and divided by “a fixed reference scale of 25.46 mm”; an MLP and one Transformer layer turn them into \(G\).
GeoNIRS.channel_compressorGeometry-aware channel compressor (§2.3)“To transform variable-size physical channel sets into fixed-length representations, we introduce \(M\) learnable latent queries”, \(L\). For each patch \(p\) they attend over the channels, with keys made of the two chromophores fused and the geometry:
\[\bar{H}_p = \tfrac{1}{2}\left(H^O_{:,p} + H^R_{:,p}\right), \quad Q = W_Q\,\mathrm{LN}(L), \quad K_p = W_K\,\mathrm{LN}\left(\bar{H}_p + G\right), \quad A_p = \mathrm{softmax}\left(\frac{Q K_p^\top}{\sqrt{d_h}}\right), \quad Z^s_p = A_p\,W_V\,\mathrm{LN}\left(H^s_{:,p}\right), \quad s \in \{O, R\}.\]“HbO and HbR share the same attention weights but retain separate value representations”, and “Invalid channels are masked before the softmax operation.” An output projection and a feed-forward block follow.
GeoNIRS.hbo_projection,GeoNIRS.hbr_projectionChromophore projections (§2.4)“a chromophore-specific residual projection that maps the feature dimension from \(D\) to \(F\) without mixing latent channels or temporal patches”: three residual blocks, of widths [192, 128, 96, 64] for GeoNIRS-Base (Table 6).
GeoNIRS.final_layerHead (§2.5, Appendix B.5.2)The tokens of the two chromophores are “flattened, standardized using training-set statistics, followed by dropout (\(p = 0.1\)) and a single linear classification layer”.
pool="flatten"is that head without the standardization;pool="mean_std"reads out the mean and standard deviation of \([Z^O \,\|\, Z^R]\) over the valid patches.
Temporal, Spatial, and Spectral Encoding
Temporal. A patch is \(S = 10\) samples, 2 s at 5 Hz, and the positional embedding has 15 rows, the 30 s pretraining window. The compressor keeps the patch axis: \(M\) tokens per 2 s patch.
Spatial. A channel has no index embedding. It is known only through its geometry \(g_c\), in the keys of the compressor, so any number of channels in any layout maps to the same \(M\) latent channels. Zeroing or shuffling the geometry at inference lowers the task-averaged balanced accuracy by 2.34 and 3.24 points (Table 3).
Chromophores. The rows are HbO and HbR concentration changes (no HbT), each with its own temporal encoder and projection. The encoder has no frequency decomposition.
Validity masks. “Samples and channels rejected during preprocessing quality control, together with patches containing insufficient valid samples, are excluded using validity masks” (§2.2). Here a non-finite sample is missing, a patch needs
minimum_patch_valid_fractionof its samples, andch_maskmarks padded pairs.
Pretrained Weights
The authors release both backbones in the
checkpoint/folder (Git LFS) of GeoNIRS/GeoNIRS@81f8995,geonirs_base.ptandgeonirs_large.pt. Both use \(M = 16\) and \(S = 10\) (Table 6); the defaults build GeoNIRS-Base.Component
GeoNIRS-Base
GeoNIRS-Large
Pretraining corpus
8 datasets, 383.30 h
22 datasets, 827.04 h
embed_dim(\(D\))192
384
depth/num_heads4 / 8
6 / 12
feature_dim(\(F\))64
128
Parameters, without the head
4,443,462
24,394,310
load_state_dict()takes the file as released: it drops the pretraining decoders and keeps the head’s initialisation. The weights expect 5 Hz signals standardized per recording and at most 150 samples.import torch from braindecode.models import GeoNIRS payload = torch.load("geonirs_base.pt", map_location="cpu", weights_only=True) model = GeoNIRS(chs_info=raw.info["chs"], n_outputs=2, n_times=60, sfreq=5.0) model.load_state_dict(payload)
Differences from the reference implementation
Input. One
(batch, 2C, n_times)tensor, stacked or interleaved (the MNE layout), instead of a batch ofhbo,hbr, validity masks andgeometry. A NaN marks a sample the reference zero-fills and flags.Geometry. Read from
chs_info(loc[3:9]), centred and scaled as the reference’sscripts/build_fair_input.py; a pair without positions gets a zero geometry.forward(geometry=...)replaces it.Head. The training-set standardization is left to the caller.
pool="mean_std"gives the features of the reference’smean_std_linearhead.Added:
on_non_divisible="pad"/"crop"andch_mask. Left out: the decoders, the temporal masking and LoRA.
Added in version 2.1.
- Parameters:
n_outputs (
int|None) – Number of outputs of the model. This is the number of classes in the case of classification.chs_info (
list[dict] |None) – Information about each individual EEG channel. This should be filled withinfo["chs"]. Refer tomne.Infofor more details.n_times (
int|None) – Number of time samples of the input window.input_window_seconds (
float|None) – Length of the input window in seconds.sfreq (
float|None) – Sampling frequency of the EEG recordings.patch_size (
int) – Samples per patch, \(S\) in the paper: 2 s at 5 Hz.max_patches (
int) – Rows of the temporal positional embedding, the 15 patches of a 30 s pretraining window; raised to the patch count ofn_times.on_non_divisible (
Literal['error','pad','crop']) –n_timesnot a multiple ofpatch_size: refuse (as the reference); pad with missing samples, to which the validity rule applies; or crop.embed_dim (
int) – Token width, \(D\) in the paper (“Embedding width”, Table 6).num_heads (
int) – Heads of the temporal Transformers (Table 6).depth (
int) – Layers of each temporal Transformer (Table 6).mlp_ratio (
float) – Feed-forward width of the temporal Transformers, overembed_dim.geometry_hidden_dim (
int|None) – Hidden width of the geometry MLP;Noneisembed_dim(Table 6).geometry_heads (
int|None) – Heads of the geometry Transformer layer;Noneisnum_heads.n_latent_channels (
int) – Learnable latent queries, \(M\) in the paper.compressor_heads (
int|None) – Heads of the compressor’s attention;Noneisnum_heads.feature_dim (
int) – Token width after the projection, \(F\) in the paper (Table 6).use_channel_geometry (
bool) – Add the geometry \(G\) to the keys of the compressor.Falsebuilds no geometry encoder: the “w/o Geometry” variant of Table 4.pool (
Literal['flatten','mean_std']) – Read-out of the(batch, 2, M, P, F)tokens: all of them flattened (the head of the paper), or their mean and standard deviation over the valid patches.channel_order (
Optional[Literal['stacked','interleaved']]) –[HbO_1..HbO_C, HbR_1..HbR_C]or[HbO_1, HbR_1, ...];Nonereads it fromchs_info(stacked without it).drop_prob (
float) – Dropout in the backbone, 0.1 in both checkpoints.head_drop_prob (
float) – Dropout beforefinal_layer, \(p = 0.1\) in Appendix B.5.2.minimum_patch_valid_fraction (
float) – Share of measured samples a patch needs to be valid: “at least 80% of its time points” (Appendix B.1.2).activation (
type[Module]) – Activation of the backbone; GELU in the reference.
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
Protocol. For pretraining (Appendix B.1), raw intensity is converted to HbO and HbR concentration changes with the modified Beer-Lambert law, resampled to 5 Hz and standardized per recording, HbO and HbR separately, then cut into 30 s windows; B.1.1 lists no band-pass filter. Downstream (§3.1, Appendix B.5-B.6), eight held-out binary tasks in windows of 12 s or 10 s are scored by subject-level five-fold cross-validation with seeds 810 and 811. With linear probing, LoRA and full fine-tuning GeoNIRS reaches 63.06, 63.42 and 63.54 task-averaged balanced accuracy (Table 1).
Replication. Linear probe of the frozen GeoNIRS-Base on DS42 (action mode, ds005935) and DS19 (pain intensity, ds005777), with the paper’s folds and seeds. “Authors’ pipeline” is their repository at 81f8995 run end to end; “Port” is this class with a re-implementation of their probe.
Task
Metric
Paper
Authors’ pipeline
Port
DS42
macro-F1
0.7018
0.7000
0.7039
DS42
AUROC
0.7840
0.7840
0.7840
DS19
macro-F1
0.5598
0.5599
0.5304
DS19
AUROC
0.5683
0.5687
0.5549
Both pipelines build the same windows (1628 and 1190), on which this class gives the authors’ encoder features (max-abs difference 0.0, Base and Large, eval mode).
References
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import GeoNIRS # Train your model model = GeoNIRS(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-geonirs-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import GeoNIRS # Load pretrained model model = GeoNIRS.from_pretrained("username/my-geonirs-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = GeoNIRS.from_pretrained("username/my-geonirs-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = GeoNIRS.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods
- encode(x, geometry=None, ch_mask=None)[source]#
Encode a window into the latent tokens of the two chromophores.
The backbone of the paper: temporal encoders, geometry encoder, channel compressor and chromophore projections.
- Parameters:
x (
Tensor) – HbO/HbR signals of shape(batch, n_chans, n_times). A NaN marks a missing sample.geometry (
Optional[Tensor]) – Geometry vector \(g_c\) of each pair,(n_pairs, 6)or(batch, n_pairs, 6).Nonetakes the one ofchs_info. Not read withuse_channel_geometry=False.ch_mask (
Optional[Tensor]) –(batch, n_chans)mask of the real channels of a padded batch; it must pad whole HbO/HbR pairs.
- Returns:
"foundation": \(Z^O\) and \(Z^R\),(batch, 2, n_latent_channels, n_patches, feature_dim)."valid": boolean(batch, n_latent_channels, n_patches),Falsefor the patches that have no valid channel (their tokens are zero).- Return type:
- forward(x, return_features=False, geometry=None, ch_mask=None)[source]#
Forward pass.
- Parameters:
x (
Tensor) – HbO/HbR signals of shape(batch, n_chans, n_times). A NaN marks a missing sample.return_features (
bool) – Return the features the head reads instead of the outputs.geometry (
Optional[Tensor]) – Geometry vector of each pair,(n_pairs, 6)or(batch, n_pairs, 6); seeencode().ch_mask (
Optional[Tensor]) –(batch, n_chans)mask of the real channels of a padded batch; seeencode().
- Returns:
Outputs of shape
(batch, n_outputs). Withreturn_features=True,{"features": Tensor, "cls_token": None}with the(batch, n_features)input offinal_layer.- Return type:
- load_state_dict(state_dict, strict=True, assign=False)[source]#
Load a state dict, or a checkpoint as the authors release it.
- Parameters:
state_dict (dict) – A state dict of this class, or the released file’s payload (
{"model": ..., "config": ...}).strict (
bool) – As intorch.nn.Module.load_state_dict().assign (
bool) – As intorch.nn.Module.load_state_dict().
- Returns:
The missing and unexpected keys.
- Return type:
NamedTuple