braindecode.models.FNIRSPreT#

class braindecode.models.FNIRSPreT(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, embed_dim=64, depth=6, num_heads=8, mlp_dim=64, pool='cls', head_dim=64, n_conv_filters=8, spatial_kernel_size=5, temporal_kernel_size=30, temporal_stride=4, channel_order=None, drop_prob=0.0, embed_drop_prob=0.0, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#

fNIRS-PreT from Wang et al. (2022).

Convolution Attention/Transformer

fNIRS-T and fNIRS-PreT architecture

The figure of the authors’ repository (fig/model.png of wzhlearning/fNIRS-Transformer at 6ebe987f, MIT). fNIRS-PreT is the right-hand input path: “Modified Beer-Lambert law”, “Segmentation”, then the “Preprocessing Module”, then the two branches of fNIRS-T. The paper (IEEE JBHI 2022) is not open access; its caption is not reproduced.#

fNIRS-T expects signals that were band-pass filtered and baseline-corrected by hand. From the abstract [1]: “a preprocessing module, which consists of one-dimensional average pooling and layer normalization, is designed to replace filtering and baseline correction of data preprocessing. It makes fNIRS-T an end-to-end network, called fNIRS-PreT.” The abstract adds that “fNIRS-PreT using raw data also achieves competitive performance to fNIRS-T”, without a number.

The authors’ PreBlock (model.py) is that module: “Preprocessing module. It is designed to replace filtering and baseline correction.” Only the abstract, the README, the figure and the code are public, and the description below comes from them.

Architecture Overview

This is FNIRST with the preprocessing module (pre) first, as the authors’ fNIRS_PreT (PreBlock, then fNIRS_T). With \(H\) = n_chans // 2 channels and \(W\) = n_times samples:

input: H HbO rows and H HbR rows             (batch, 2 H, W)
to_chromophore_planes                        (batch, 2, H, W)
pre: per plane, average_pooling, norms[i]    (batch, 2, H, W)
fNIRS-T levels, Concat, head                 (batch, n_outputs)

Macro Components

  • pre.average_pooling Average pooling. Three moving averages along time, AvgPool1d with kernels \(k\) = 5, 13 and 7, stride 1 and “same” zero padding (the authors’ pool1, pool2, pool3), shared by HbO and HbR. With \(k = 2r + 1\), each is

    \[m_k(x)[t] = \frac{1}{k} \sum_{j=-r}^{r} x[t + j] .\]
  • pre.norms Layer normalization. One LayerNorm over the \(W\) samples per chromophore (the authors’ ln_0, ln_1): each channel row is centred and scaled over the window, then given a learned scale and shift per sample, so the weights are tied to n_times.

  • Every other module is that of FNIRST.

Temporal, Spatial, and Spectral Encoding

The module keeps the \(W\) samples and treats every channel row alike; HbO and HbR share the moving averages and have their own LayerNorm. It has no explicit frequency decomposition: it stands in for the filtering and baseline correction, so the inputs are raw segmented concentration changes. The rest is as in FNIRST.

Pretrained Weights

None: the authors release code, not weights.

Differences from the reference implementation

pre.ln_i is pre.norms.i and the authors’ fNIRS_T. prefix is removed: an authors’ fNIRS_PreT state_dict loads after FNIRST’s conversion with these two renames, and the port and the authors’ model then agree to float32 rounding (max-abs below 1e-6 on the dataset A, B and C configurations). The port indexes each chromophore plane, where the authors’ PreBlock calls squeeze(), which also drops the batch axis for a batch of one. The input layout, the opt-in mask, the logits and the parameters are those of FNIRST.

Parameters:
  • n_outputs (int | None) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int | None) – Number of EEG channels.

  • chs_info (list[dict] | None) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int | None) – Number of time samples of the input window.

  • input_window_seconds (float | None) – Length of the input window in seconds.

  • sfreq (float | None) – Sampling frequency of the EEG recordings.

  • embed_dim (int) – \(D\), the authors’ dim, “last dimension of output tensor after linear transformation”; 64 for their datasets A and B, 128 for C.

  • depth (int) – Authors’ depth, “number of Transformer blocks” per level.

  • num_heads (int) – Authors’ heads, “number of the multi-head self-attention”.

  • mlp_dim (int) – Authors’ mlp_dim, “dimension of the MLP layer”.

  • pool (Literal['cls', 'mean']) – Read-out of each level (authors’ pool): "cls", its class token; "mean", the mean over its tokens (class token included).

  • head_dim (int) – \(d_h\), the authors’ dim_head: width of each attention head.

  • n_conv_filters (int) – \(C_S = C_C\), feature maps of both convolutions (authors: 8).

  • spatial_kernel_size (int) – \(k_s\), height of \(\mathrm{Conv}_S\) in HbO/HbR pairs (authors: 5); at most the number of pairs, n_chans // 2.

  • temporal_kernel_size (int) – \(k_t\), temporal kernel of both convolutions, in samples (authors: 30); n_times must be at least this.

  • temporal_stride (int) – \(s_t\), temporal stride of both convolutions, in samples (authors: 4).

  • channel_order (Optional[Literal['stacked', 'interleaved']]) – [HbO_1..HbO_H, HbR_1..HbR_H] or MNE’s [HbO_1, HbR_1, ...]; None reads it from chs_info.

  • drop_prob (float) – Authors’ dropout in the attention and feed-forward sub-layers.

  • embed_drop_prob (float) – Authors’ emb_dropout, “dropout for patch embeddings”.

  • activation (type[Module]) – Activation of the feed-forward sub-layers (authors: GELU).

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

The protocol of the authors’ released code is that of FNIRST, on raw data: their README asks to disable the filtering and baseline correction of B_mat2xls.m and C_mat2xls.m by hand for fNIRS-PreT, and Load_Dataset_A skips the dataset A filters for model='fNIRS-PreT'. The flooding levels are those of fNIRS-T except on dataset B (0.40, 0.38 and 0.35 from the zero-based epochs 0, 30 and 50).

The dataset C replication is the fNIRS-PreT row of the table of FNIRST. Both sides received the band-pass filtered, baseline-corrected windows of fNIRS-T there, as the released Load_Dataset_C has no raw-data path.

Added in version 2.1.

License

References

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import FNIRSPreT

# Train your model
model = FNIRSPreT(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-fnirspret-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import FNIRSPreT

# Load pretrained model
model = FNIRSPreT.from_pretrained("username/my-fnirspret-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = FNIRSPreT.from_pretrained("username/my-fnirspret-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = FNIRSPreT.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.