braindecode.models.JungMAE#

class braindecode.models.JungMAE(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=10, embed_dim=192, depth=4, num_heads=8, mlp_ratio=4.0, drop_prob=0.1, head_drop_prob=0.1, minimum_patch_valid_fraction=0.8, activation=<class 'torch.nn.modules.activation.GELU'>, channel_order=None, on_non_divisible='error')[source]#

JungMAE from Jung et al. (2025), as implemented by the GeoNIRS authors.

Attention/Transformer Foundation Model

Added in version 2.1.

Labelled fNIRS data are slow and costly to collect, and supervised decoders need a lot of them. Jung, Lee and An [1] “propose a novel foundation model for fNIRS data based on a self-supervised masked autoencoder framework”, pretrained on unlabelled recordings. They report that it matches supervised training “while requiring only one-third of the labeled training data”, and that “a larger masking size aligns with the low-frequency nature of fNIRS signals”. Their paper (IEEE BCI 2025) is not open access and came without code, so what is said about it here comes from its abstract.

This class ports the implementation and the jung_mae.pt weights of the GeoNIRS authors (GeoNIRS/GeoNIRS at commit 81f8995), who use the model as a baseline and call it Jung-MAE (Tables 1, 14 and 15 of [2]). Their paper describes it as “a paper-guided implementation of the multi-dataset fNIRS pretraining framework of Jung et al. (2025)” which, to “accommodate heterogeneous channel counts”, “uses channel-independent encoding with shared parameters” (Section 3.1). The details below are read from the GeoNIRS code.

Architectural Overview

Every HbO/HbR pair (one source-detector channel) is a sequence of its own. Its window is cut into 2 s patches, each patch becomes one token that holds both chromophores, and a small Transformer encodes the tokens of that pair. The same weights serve every pair, which is how one checkpoint handles montages of any size. The classifier then flattens the tokens of all the pairs. With \(B\) the batch size, \(C\) = n_chans // 2 pairs, \(Q\) = patch_size, \(P\) patches and \(D\) = embed_dim, the model has the following sequence of steps:

x (B, 2C, n_times): HbO and HbR rows, NaN = missing sample
1. HbO/HbR split, invalid samples to zero  -> (B, C, P*Q) each
2. to_patches: [HbO_t, HbR_t, ...]         -> (B*C, P, 2Q)
3. patch_embed + sinusoidal position       -> (B*C, P, D)
4. encoder, invalid patches not attended   -> (B, C, P, D)   encode()
5. flatten_layer, head_dropout             -> (B, C*P*D)
6. final_layer                             -> (B, n_outputs)

Macro Components

  • JungMAE.patch_embed Patch tokenization

    A patch interleaves the \(Q\) HbO samples \(o\) and HbR samples \(r\) of pair \(c\). One Linear maps it to a \(D\)-wide token, and the fixed sinusoidal encoding of the patch index \(p\) is added:

    \[\mathbf{z}_{c,p} = \mathbf{W}\,[o_{c,pQ},\, r_{c,pQ},\, \dots,\, o_{c,pQ+Q-1},\, r_{c,pQ+Q-1}] + \mathbf{b} + \mathrm{PE}(p).\]
  • Patch validity (minimum_patch_valid_fraction)

    A sample is valid when both chromophores are finite and it is not in mask. Invalid samples are set to zero, and a patch counts only when at least \(\lceil 0.8\,Q \rceil\) of its samples are valid. The GeoNIRS paper applies the same 80 % rule to its own model (Appendix B.1.2).

  • JungMAE.encoder Transformer encoder

    Four pre-norm TransformerEncoderLayer blocks (8 heads, feed-forward width 768, GELU) and a final LayerNorm, over the \(P\) tokens of one pair. Invalid patches are not attended and their output is zero, so a pair without any valid patch gives zero features.

  • JungMAE.final_layer Classification head

    Flatten, Dropout and one Linear, the flatten_linear head of the GeoNIRS benchmark. It is not pretrained, and its input size fixes n_chans and n_times.

Temporal, Spatial, and Chromophore Encoding

  • Temporal. Non-overlapping 2 s patches (10 samples at 5 Hz) with a fixed sinusoidal position. The GeoNIRS paper pretrains on 30 s windows (15 patches) and keeps each downstream task’s 10 s or 12 s window (Appendix B.1.2).

  • Spatial. None. Pairs never attend to each other and the model sees no optode position. A pair is identified only by where its features sit in the head input, so the pairs must keep the order used in training.

  • Chromophores. HbO and HbR share each token, sample by sample; HbT is not used. The rows must be the HbO and the HbR rows of the same pairs (see channel_order).

Pretraining

In the reference, each pair keeps half of its valid patches visible, drawn at random, and hides the others. The encoder reads the visible tokens only. A lighter decoder (two layers of width 96, with a learned mask token) predicts every patch, and the loss is the mean squared error over the valid samples of the hidden patches. The decoder is stored in jung_mae.pt and is not part of this class.

Known Limitations

  • The model “does not model physical channel geometry” (GeoNIRS paper, Section 3.1): different channel counts “are supported without modeling physical channel coordinates or cross-layout correspondence” (Table 13).

  • It is “a paper-guided implementation rather than an exact reproduction of the original method” (GeoNIRS paper, Appendix G).

Pretrained Weights

checkpoint/jung_mae.pt of GeoNIRS/GeoNIRS at 81f8995 was trained by the GeoNIRS authors and is not on Hugging Face. It holds the encoder (1,783,872 parameters), the decoder (244,436) and the model configuration, which equals the defaults of this class.

Important

load_state_dict() takes the file as released. It drops the decoder and keeps the new final_layer:

import torch
from braindecode.models import JungMAE

payload = torch.load("jung_mae.pt", map_location="cpu", weights_only=True)
model = JungMAE(n_chans=40, n_outputs=2, n_times=60, sfreq=5.0)
model.load_state_dict(payload)

The weights expect the inputs of the GeoNIRS paper (Appendix B.1): HbO and HbR in µmol/L resampled to 5 Hz, each chromophore z-scored per recording over all its valid channels and time points.

Differences from the Reference Implementation

  • Input. One tensor (batch, n_chans, n_times), with the HbO rows then the HbR rows, or MNE’s interleaved order read from chs_info. The reference takes hbo and hbr tensors with validity masks. Here NaN marks a missing sample (an all-NaN row is a missing channel), and the opt-in mask marks more.

  • Parity. The reference packs the valid tokens of each pair; this class masks the invalid ones as attention keys. With jung_mae.pt the features equal those of the authors’ module (max-abs 0.0) in eval mode under torch.no_grad(). With gradients on they differ by about 2e-6 at most (float32 rounding), and in training mode the dropout draws differ.

  • Head. The benchmark head standardizes the flattened features with training-set statistics before its Dropout; this head does not, so standardize the eval-mode return_features yourself to reproduce the benchmark. final_layer starts Xavier-uniform with a zero bias, as the other Linear layers of the reference.

  • Additions. on_non_divisible="pad" or "crop" (the reference refuses a partial patch) and mask. ch_mask is refused, because final_layer spans every pair.

Parameters:
  • n_outputs (int | None) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int | None) – Number of EEG channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int | None) – Number of time samples of the input window.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency of the EEG recordings.

  • patch_size (int) – Samples per patch (patch_samples in the reference), 2 s at 5 Hz.

  • embed_dim (int) – Width of the tokens (encoder_dim in the reference).

  • depth (int) – Number of Transformer layers (encoder_layers in the reference).

  • num_heads (int) – Attention heads per layer (encoder_heads in the reference).

  • mlp_ratio (float) – Feed-forward width over embed_dim.

  • drop_prob (float) – Dropout of the encoder layers (dropout in the reference).

  • head_drop_prob (float) – Dropout before final_layer (head_dropout in the benchmark).

  • minimum_patch_valid_fraction (float) – Fraction of valid samples a patch needs, in (0, 1].

  • activation (type[Module]) – Activation of the feed-forward layers.

  • channel_order (Optional[Literal['stacked', 'interleaved']]) – [HbO_1..HbO_C, HbR_1..HbR_C] or MNE’s [HbO_1, HbR_1, ...]. None reads it from chs_info ("stacked" without it). n_chans must be even.

  • on_non_divisible (Literal['error', 'pad', 'crop']) – What to do when n_times is not a multiple of patch_size: raise, as the reference does; pad with missing samples, to which the validity rule applies; or crop.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

The GeoNIRS benchmark (Section 3.1 and Appendices B.5-B.6 of [2]) evaluates eight held-out binary tasks with subject-level five-fold cross-validation and the seeds 810 and 811. Every method gets the same windows and folds and the same training: 30 epochs, AdamW, class-weighted cross-entropy and model selection by validation balanced accuracy. Jung-MAE-LP trains the head only, with a learning rate of 1e-4.

This class with jung_mae.pt and the benchmark’s linear-probe loop reproduces Jung-MAE-LP (mean over the five test folds and the two seeds):

Task

Metric

GeoNIRS paper (%)

This class (%)

DS42 (action mode)

macro-F1

68.36

68.35

DS42 (action mode)

AUROC

76.78

76.80

DS19 (pain intensity)

macro-F1

54.23

54.95

DS19 (pain intensity)

AUROC

58.39

58.63

References

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import JungMAE

# Train your model
model = JungMAE(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-jungmae-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import JungMAE

# Load pretrained model
model = JungMAE.from_pretrained("username/my-jungmae-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = JungMAE.from_pretrained("username/my-jungmae-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = JungMAE.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

encode(x, mask=None)[source]#

Encode the patches of every HbO/HbR pair.

Parameters:
  • x (Tensor) – Input tensor with shape (batch_size, n_chans, n_times). NaN marks a missing sample.

  • mask (Optional[Tensor]) – Same shape as x. True samples count as missing.

Returns:

"foundation", the tokens with shape (batch_size, n_pairs, n_patches, embed_dim), zero at the invalid patches, and "valid", the boolean patch validity with shape (batch_size, n_pairs, n_patches). These are the names of the reference.

Return type:

dict[str, Tensor]

forward(x, return_features=False, ch_mask=None, mask=None)[source]#

Forward pass of the JungMAE model.

Parameters:
  • x (Tensor) – Input tensor with shape (batch_size, n_chans, n_times). NaN marks a missing sample.

  • return_features (bool) – If True, return the input of final_layer instead of the logits.

  • ch_mask (Optional[Tensor]) – Not supported, because final_layer spans every pair. Mark a missing channel with NaN instead.

  • mask (Optional[Tensor]) – Same shape as x. True samples count as missing.

Returns:

Output tensor with shape (batch_size, n_outputs), or {"features": (batch_size, n_pairs * n_patches * embed_dim), "cls_token": None} with return_features=True.

Return type:

Tensor | dict[str, Tensor | None]

load_state_dict(state_dict, strict=True, assign=False)[source]#

Load a state dict, or the jung_mae.pt payload as released.

The payload keeps the weights under "model" and has the pretraining decoder, which is dropped. A state dict without final_layer keeps the head of this model.

Parameters:
  • state_dict – The description is missing.

  • strict (bool) – The description is missing.

  • assign (bool) – The description is missing.

reset_head(n_outputs)[source]#

Replace the classification head for a new number of outputs.

This is called automatically by from_pretrained() when the user passes an n_outputs that differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync with self._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also use self._set_n_outputs to validate and record the new value.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.