braindecode.models.JungMAE#
- class braindecode.models.JungMAE(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=10, embed_dim=192, depth=4, num_heads=8, mlp_ratio=4.0, drop_prob=0.1, head_drop_prob=0.1, minimum_patch_valid_fraction=0.8, activation=<class 'torch.nn.modules.activation.GELU'>, channel_order=None, on_non_divisible='error')[source]#
JungMAE from Jung et al. (2025), as implemented by the GeoNIRS authors.
Attention/Transformer Foundation Model
Added in version 2.1.
Labelled fNIRS data are slow and costly to collect, and supervised decoders need a lot of them. Jung, Lee and An [1] “propose a novel foundation model for fNIRS data based on a self-supervised masked autoencoder framework”, pretrained on unlabelled recordings. They report that it matches supervised training “while requiring only one-third of the labeled training data”, and that “a larger masking size aligns with the low-frequency nature of fNIRS signals”. Their paper (IEEE BCI 2025) is not open access and came without code, so what is said about it here comes from its abstract.
This class ports the implementation and the
jung_mae.ptweights of the GeoNIRS authors (GeoNIRS/GeoNIRSat commit81f8995), who use the model as a baseline and call it Jung-MAE (Tables 1, 14 and 15 of [2]). Their paper describes it as “a paper-guided implementation of the multi-dataset fNIRS pretraining framework of Jung et al. (2025)” which, to “accommodate heterogeneous channel counts”, “uses channel-independent encoding with shared parameters” (Section 3.1). The details below are read from the GeoNIRS code.Architectural Overview
Every HbO/HbR pair (one source-detector channel) is a sequence of its own. Its window is cut into 2 s patches, each patch becomes one token that holds both chromophores, and a small Transformer encodes the tokens of that pair. The same weights serve every pair, which is how one checkpoint handles montages of any size. The classifier then flattens the tokens of all the pairs. With \(B\) the batch size, \(C\) =
n_chans // 2pairs, \(Q\) =patch_size, \(P\) patches and \(D\) =embed_dim, the model has the following sequence of steps:x (B, 2C, n_times): HbO and HbR rows, NaN = missing sample 1. HbO/HbR split, invalid samples to zero -> (B, C, P*Q) each 2. to_patches: [HbO_t, HbR_t, ...] -> (B*C, P, 2Q) 3. patch_embed + sinusoidal position -> (B*C, P, D) 4. encoder, invalid patches not attended -> (B, C, P, D) encode() 5. flatten_layer, head_dropout -> (B, C*P*D) 6. final_layer -> (B, n_outputs)
Macro Components
JungMAE.patch_embedPatch tokenizationA patch interleaves the \(Q\) HbO samples \(o\) and HbR samples \(r\) of pair \(c\). One
Linearmaps it to a \(D\)-wide token, and the fixed sinusoidal encoding of the patch index \(p\) is added:\[\mathbf{z}_{c,p} = \mathbf{W}\,[o_{c,pQ},\, r_{c,pQ},\, \dots,\, o_{c,pQ+Q-1},\, r_{c,pQ+Q-1}] + \mathbf{b} + \mathrm{PE}(p).\]Patch validity (
minimum_patch_valid_fraction)A sample is valid when both chromophores are finite and it is not in
mask. Invalid samples are set to zero, and a patch counts only when at least \(\lceil 0.8\,Q \rceil\) of its samples are valid. The GeoNIRS paper applies the same 80 % rule to its own model (Appendix B.1.2).JungMAE.encoderTransformer encoderFour pre-norm
TransformerEncoderLayerblocks (8 heads, feed-forward width 768, GELU) and a finalLayerNorm, over the \(P\) tokens of one pair. Invalid patches are not attended and their output is zero, so a pair without any valid patch gives zero features.JungMAE.final_layerClassification headFlatten,
Dropoutand oneLinear, theflatten_linearhead of the GeoNIRS benchmark. It is not pretrained, and its input size fixesn_chansandn_times.
Temporal, Spatial, and Chromophore Encoding
Temporal. Non-overlapping 2 s patches (10 samples at 5 Hz) with a fixed sinusoidal position. The GeoNIRS paper pretrains on 30 s windows (15 patches) and keeps each downstream task’s 10 s or 12 s window (Appendix B.1.2).
Spatial. None. Pairs never attend to each other and the model sees no optode position. A pair is identified only by where its features sit in the head input, so the pairs must keep the order used in training.
Chromophores. HbO and HbR share each token, sample by sample; HbT is not used. The rows must be the HbO and the HbR rows of the same pairs (see
channel_order).
Pretraining
In the reference, each pair keeps half of its valid patches visible, drawn at random, and hides the others. The encoder reads the visible tokens only. A lighter decoder (two layers of width 96, with a learned mask token) predicts every patch, and the loss is the mean squared error over the valid samples of the hidden patches. The decoder is stored in
jung_mae.ptand is not part of this class.Known Limitations
The model “does not model physical channel geometry” (GeoNIRS paper, Section 3.1): different channel counts “are supported without modeling physical channel coordinates or cross-layout correspondence” (Table 13).
It is “a paper-guided implementation rather than an exact reproduction of the original method” (GeoNIRS paper, Appendix G).
Pretrained Weights
checkpoint/jung_mae.ptofGeoNIRS/GeoNIRSat81f8995was trained by the GeoNIRS authors and is not on Hugging Face. It holds the encoder (1,783,872 parameters), the decoder (244,436) and the model configuration, which equals the defaults of this class.Important
load_state_dict()takes the file as released. It drops the decoder and keeps the newfinal_layer:import torch from braindecode.models import JungMAE payload = torch.load("jung_mae.pt", map_location="cpu", weights_only=True) model = JungMAE(n_chans=40, n_outputs=2, n_times=60, sfreq=5.0) model.load_state_dict(payload)
The weights expect the inputs of the GeoNIRS paper (Appendix B.1): HbO and HbR in µmol/L resampled to 5 Hz, each chromophore z-scored per recording over all its valid channels and time points.
Differences from the Reference Implementation
Input. One tensor
(batch, n_chans, n_times), with the HbO rows then the HbR rows, or MNE’s interleaved order read fromchs_info. The reference takeshboandhbrtensors with validity masks. Here NaN marks a missing sample (an all-NaN row is a missing channel), and the opt-inmaskmarks more.Parity. The reference packs the valid tokens of each pair; this class masks the invalid ones as attention keys. With
jung_mae.ptthe features equal those of the authors’ module (max-abs 0.0) in eval mode undertorch.no_grad(). With gradients on they differ by about 2e-6 at most (float32 rounding), and in training mode the dropout draws differ.Head. The benchmark head standardizes the flattened features with training-set statistics before its Dropout; this head does not, so standardize the eval-mode
return_featuresyourself to reproduce the benchmark.final_layerstarts Xavier-uniform with a zero bias, as the other Linear layers of the reference.Additions.
on_non_divisible="pad"or"crop"(the reference refuses a partial patch) andmask.ch_maskis refused, becausefinal_layerspans every pair.
- Parameters:
n_outputs (
int|None) – Number of outputs of the model. This is the number of classes in the case of classification.chs_info (list of dict) – Information about each individual EEG channel. This should be filled with
info["chs"]. Refer tomne.Infofor more details.n_times (
int|None) – Number of time samples of the input window.input_window_seconds (
float|None) – Length of the input window in seconds.sfreq (
float|None) – Sampling frequency of the EEG recordings.patch_size (
int) – Samples per patch (patch_samplesin the reference), 2 s at 5 Hz.embed_dim (
int) – Width of the tokens (encoder_dimin the reference).depth (
int) – Number of Transformer layers (encoder_layersin the reference).num_heads (
int) – Attention heads per layer (encoder_headsin the reference).mlp_ratio (
float) – Feed-forward width overembed_dim.drop_prob (
float) – Dropout of the encoder layers (dropoutin the reference).head_drop_prob (
float) – Dropout beforefinal_layer(head_dropoutin the benchmark).minimum_patch_valid_fraction (
float) – Fraction of valid samples a patch needs, in (0, 1].activation (
type[Module]) – Activation of the feed-forward layers.channel_order (
Optional[Literal['stacked','interleaved']]) –[HbO_1..HbO_C, HbR_1..HbR_C]or MNE’s[HbO_1, HbR_1, ...].Nonereads it fromchs_info("stacked"without it).n_chansmust be even.on_non_divisible (
Literal['error','pad','crop']) – What to do whenn_timesis not a multiple ofpatch_size: raise, as the reference does; pad with missing samples, to which the validity rule applies; or crop.
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
The GeoNIRS benchmark (Section 3.1 and Appendices B.5-B.6 of [2]) evaluates eight held-out binary tasks with subject-level five-fold cross-validation and the seeds 810 and 811. Every method gets the same windows and folds and the same training: 30 epochs, AdamW, class-weighted cross-entropy and model selection by validation balanced accuracy. Jung-MAE-LP trains the head only, with a learning rate of 1e-4.
This class with
jung_mae.ptand the benchmark’s linear-probe loop reproduces Jung-MAE-LP (mean over the five test folds and the two seeds):Task
Metric
GeoNIRS paper (%)
This class (%)
DS42 (action mode)
macro-F1
68.36
68.35
DS42 (action mode)
AUROC
76.78
76.80
DS19 (pain intensity)
macro-F1
54.23
54.95
DS19 (pain intensity)
AUROC
58.39
58.63
References
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import JungMAE # Train your model model = JungMAE(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-jungmae-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import JungMAE # Load pretrained model model = JungMAE.from_pretrained("username/my-jungmae-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = JungMAE.from_pretrained("username/my-jungmae-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = JungMAE.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods
- encode(x, mask=None)[source]#
Encode the patches of every HbO/HbR pair.
- Parameters:
- Returns:
"foundation", the tokens with shape (batch_size, n_pairs, n_patches, embed_dim), zero at the invalid patches, and"valid", the boolean patch validity with shape (batch_size, n_pairs, n_patches). These are the names of the reference.- Return type:
- forward(x, return_features=False, ch_mask=None, mask=None)[source]#
Forward pass of the JungMAE model.
- Parameters:
x (
Tensor) – Input tensor with shape (batch_size, n_chans, n_times). NaN marks a missing sample.return_features (
bool) – If True, return the input offinal_layerinstead of the logits.ch_mask (
Optional[Tensor]) – Not supported, becausefinal_layerspans every pair. Mark a missing channel with NaN instead.mask (
Optional[Tensor]) – Same shape asx. True samples count as missing.
- Returns:
Output tensor with shape (batch_size, n_outputs), or
{"features": (batch_size, n_pairs * n_patches * embed_dim), "cls_token": None}withreturn_features=True.- Return type:
- load_state_dict(state_dict, strict=True, assign=False)[source]#
Load a state dict, or the
jung_mae.ptpayload as released.The payload keeps the weights under
"model"and has the pretraining decoder, which is dropped. A state dict withoutfinal_layerkeeps the head of this model.
- reset_head(n_outputs)[source]#
Replace the classification head for a new number of outputs.
This is called automatically by
from_pretrained()when the user passes ann_outputsthat differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync withself._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also useself._set_n_outputsto validate and record the new value.Examples
>>> from braindecode.models import BENDR >>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4) >>> model.reset_head(10) >>> model.n_outputs 10
Added in version 1.4.