braindecode.models.FNIRSPreT#
- class braindecode.models.FNIRSPreT(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, embed_dim=64, depth=6, num_heads=8, mlp_dim=64, pool='cls', head_dim=64, n_conv_filters=8, spatial_kernel_size=5, temporal_kernel_size=30, temporal_stride=4, channel_order=None, drop_prob=0.0, embed_drop_prob=0.0, activation=<class 'torch.nn.modules.activation.GELU'>)[source]#
fNIRS-PreT from Wang et al. (2022).
Convolution Attention/Transformer
The figure of the authors’ repository (
fig/model.pngof wzhlearning/fNIRS-Transformer at6ebe987f, MIT). fNIRS-PreT is the right-hand input path: “Modified Beer-Lambert law”, “Segmentation”, then the “Preprocessing Module”, then the two branches of fNIRS-T. The paper (IEEE JBHI 2022) is not open access; its caption is not reproduced.#fNIRS-T expects signals that were band-pass filtered and baseline-corrected by hand. From the abstract [1]: “a preprocessing module, which consists of one-dimensional average pooling and layer normalization, is designed to replace filtering and baseline correction of data preprocessing. It makes fNIRS-T an end-to-end network, called fNIRS-PreT.” The abstract adds that “fNIRS-PreT using raw data also achieves competitive performance to fNIRS-T”, without a number.
The authors’
PreBlock(model.py) is that module: “Preprocessing module. It is designed to replace filtering and baseline correction.” Only the abstract, the README, the figure and the code are public, and the description below comes from them.Architecture Overview
This is
FNIRSTwith the preprocessing module (pre) first, as the authors’fNIRS_PreT(PreBlock, thenfNIRS_T). With \(H\) =n_chans // 2channels and \(W\) =n_timessamples:input: H HbO rows and H HbR rows (batch, 2 H, W) to_chromophore_planes (batch, 2, H, W) pre: per plane, average_pooling, norms[i] (batch, 2, H, W) fNIRS-T levels, Concat, head (batch, n_outputs)
Macro Components
pre.average_poolingAverage pooling. Three moving averages along time,AvgPool1dwith kernels \(k\) = 5, 13 and 7, stride 1 and “same” zero padding (the authors’pool1,pool2,pool3), shared by HbO and HbR. With \(k = 2r + 1\), each is\[m_k(x)[t] = \frac{1}{k} \sum_{j=-r}^{r} x[t + j] .\]pre.normsLayer normalization. OneLayerNormover the \(W\) samples per chromophore (the authors’ln_0,ln_1): each channel row is centred and scaled over the window, then given a learned scale and shift per sample, so the weights are tied ton_times.Every other module is that of
FNIRST.
Temporal, Spatial, and Spectral Encoding
The module keeps the \(W\) samples and treats every channel row alike; HbO and HbR share the moving averages and have their own LayerNorm. It has no explicit frequency decomposition: it stands in for the filtering and baseline correction, so the inputs are raw segmented concentration changes. The rest is as in
FNIRST.Pretrained Weights
None: the authors release code, not weights.
Differences from the reference implementation
pre.ln_iispre.norms.iand the authors’fNIRS_T.prefix is removed: an authors’fNIRS_PreTstate_dictloads after FNIRST’s conversion with these two renames, and the port and the authors’ model then agree to float32 rounding (max-abs below 1e-6 on the dataset A, B and C configurations). The port indexes each chromophore plane, where the authors’PreBlockcallssqueeze(), which also drops the batch axis for a batch of one. The input layout, the opt-inmask, the logits and the parameters are those ofFNIRST.- Parameters:
n_outputs (
int|None) – Number of outputs of the model. This is the number of classes in the case of classification.chs_info (
list[dict] |None) – Information about each individual EEG channel. This should be filled withinfo["chs"]. Refer tomne.Infofor more details.n_times (
int|None) – Number of time samples of the input window.input_window_seconds (
float|None) – Length of the input window in seconds.sfreq (
float|None) – Sampling frequency of the EEG recordings.embed_dim (
int) – \(D\), the authors’dim, “last dimension of output tensor after linear transformation”; 64 for their datasets A and B, 128 for C.depth (
int) – Authors’depth, “number of Transformer blocks” per level.num_heads (
int) – Authors’heads, “number of the multi-head self-attention”.mlp_dim (
int) – Authors’mlp_dim, “dimension of the MLP layer”.pool (
Literal['cls','mean']) – Read-out of each level (authors’pool):"cls", its class token;"mean", the mean over its tokens (class token included).head_dim (
int) – \(d_h\), the authors’dim_head: width of each attention head.n_conv_filters (
int) – \(C_S = C_C\), feature maps of both convolutions (authors: 8).spatial_kernel_size (
int) – \(k_s\), height of \(\mathrm{Conv}_S\) in HbO/HbR pairs (authors: 5); at most the number of pairs,n_chans // 2.temporal_kernel_size (
int) – \(k_t\), temporal kernel of both convolutions, in samples (authors: 30);n_timesmust be at least this.temporal_stride (
int) – \(s_t\), temporal stride of both convolutions, in samples (authors: 4).channel_order (
Optional[Literal['stacked','interleaved']]) –[HbO_1..HbO_H, HbR_1..HbR_H]or MNE’s[HbO_1, HbR_1, ...];Nonereads it fromchs_info.drop_prob (
float) – Authors’dropoutin the attention and feed-forward sub-layers.embed_drop_prob (
float) – Authors’emb_dropout, “dropout for patch embeddings”.activation (
type[Module]) – Activation of the feed-forward sub-layers (authors: GELU).
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
The protocol of the authors’ released code is that of
FNIRST, on raw data: their README asks to disable the filtering and baseline correction ofB_mat2xls.mandC_mat2xls.mby hand for fNIRS-PreT, andLoad_Dataset_Askips the dataset A filters formodel='fNIRS-PreT'. The flooding levels are those of fNIRS-T except on dataset B (0.40, 0.38 and 0.35 from the zero-based epochs 0, 30 and 50).The dataset C replication is the fNIRS-PreT row of the table of
FNIRST. Both sides received the band-pass filtered, baseline-corrected windows of fNIRS-T there, as the releasedLoad_Dataset_Chas no raw-data path.Added in version 2.1.
References
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import FNIRSPreT # Train your model model = FNIRSPreT(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-fnirspret-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import FNIRSPreT # Load pretrained model model = FNIRSPreT.from_pretrained("username/my-fnirspret-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = FNIRSPreT.from_pretrained("username/my-fnirspret-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = FNIRSPreT.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.