braindecode.models.TCCNet#

class braindecode.models.TCCNet(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, filters=((32, 32), (32, 32), (64, 64), (128, )), kernel_size=3, drop_prob=0.05, activation=<class 'torch.nn.modules.activation.ReLU'>, classification_head='att', mask_value=-10.0)[source]#

TCCNet from Tsinganos et al. (2019) [tccnet2019].

Convolution Recurrent Attention/Transformer

TCCNet architecture

“Proposed model” panel of the authors’ ICASSP 2019 poster (MIT, from [tccnetcode]): dilated causal convolutions, read out by an average over time (AoT) or an attention (Att) head. Braindecode returns the logits, without the softmax.#

A temporal convolutional network that classifies a whole sEMG gesture. Residual blocks of causal convolutions, whose dilation doubles at every layer and keeps doubling across blocks, map the gesture to a sequence of features. An average over time ("aot") or a context-vector attention ("att") pools that sequence into one vector for a linear classifier.

The paper compares four models that differ in receptive field (RF) and head (poster Table 1; parameters counted for 10 channels and 53 classes):

Model

filters

Convs

Parameters

AoT, 300 ms

((32, 64), (64, 128))

4

60,117

AoT, 2500 ms

((32, 32), (32, 32), (64, 64), (128,))

7

72,245

Att, 300 ms

((32, 64), (64, 128))

4

76,629

Att, 2500 ms

((32, 32), (32, 32), (64, 64), (128,))

7

88,757

The defaults build Att, 2500 ms, the best model of the paper (89.76 % top-1). Pass classification_head="aot" and/or filters=((32, 64), (64, 128)) for the other three. With kernel_size=3, the RF is 31 samples (310 ms at 100 Hz) for the 4-convolution models and 255 samples (2550 ms) for the 7-convolution ones.

Inputs are (batch, n_chans, n_times) and may have any length. The softmax of the reference is left out, because braindecode models return logits.

Parameters:
  • n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int) – Number of EEG channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int) – Number of time samples of the input window.

  • input_window_seconds (float) – Length of the input window in seconds.

  • sfreq (float) – Sampling frequency of the EEG recordings.

  • filters (Sequence[Sequence[int]]) – Output filters of each causal convolution, one inner sequence per residual block. It sets the number of blocks and of convolutions per block.

  • kernel_size (int) – Temporal kernel size of every causal convolution.

  • drop_prob (float) – Channel-wise dropout after each activated convolution, as Keras SpatialDropout1D.

  • activation (type[Module]) – Activation after each causal convolution.

  • classification_head (str) – "att": context-vector attention; "aot": average over the valid time steps.

  • mask_value (float | None) – A time step where every channel equals this value is padding: it is zeroed before every convolution and left out of the pooling, as the reference’s Keras Masking layer does. None turns masking off.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

Protocol of the reference code [tccnetcode], for all four models: one model per subject of NinaPro DB1 (27 subjects, 10 sEMG channels at 100 Hz, 53 classes including rest). Each channel is rectified and low-pass filtered by a 1 Hz first-order Butterworth filter applied forward-backward (filtfilt). Each whole gesture repetition is one sample: the variable-length gestures of a batch are pre-padded with -10 to the longest one and masked. Repetitions 1, 3, 4, 6, 8, 9 and 10 train; 2, 5 and 7 test. Training uses Adam (learning rate 0.01, epsilon 0.001) for 30 epochs, batch size 128, L2 0.0005 on convolution and classifier kernels (use optimizer weight decay in PyTorch), class- balanced sample weights, and 10 augmented copies of each training gesture (time and magnitude warping 0.2, noise at 25 dB SNR). The generator’s min-max normalisation is off in the released configurations.

The masked average over time equals the reference only for pre-padding, the reference pipeline: the reference sums every time step, padded ones included, and divides by the number of valid steps, while this port always leaves padded steps out; on a fully padded input it gives NaN, as the reference does. The bias of each convolution is added only where the convolution output is non-zero, as in the reference CausalConv1D; this only matters for bit-exact agreement with it. The attention subtracts the row maximum before the exponential: the same values as the reference, which overflows to NaN once the context vector grows large. Initialisation follows Keras 2.2.4: truncated Glorot normal kernels, zero biases, Glorot uniform attention weights.

License

References

[tccnet2019]

Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., & Skodras, A. (2019). Improved Gesture Recognition Based on sEMG Signals and TCN. ICASSP 2019, 1169-1173. https://doi.org/10.1109/ICASSP.2019.8683239

[tccnetcode] (1,2)

Tsinganos, P. et al. ICASSP2019_TCN source code. DSIP-UPatras/ICASSP2019_TCN

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import TCCNet

# Train your model
model = TCCNet(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-tccnet-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import TCCNet

# Load pretrained model
model = TCCNet.from_pretrained("username/my-tccnet-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = TCCNet.from_pretrained("username/my-tccnet-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = TCCNet.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x)[source]#

Return logits for a batch of sEMG gestures.

Parameters:

x (Tensor) – The description is missing.

Return type:

Tensor

reset_head(n_outputs)[source]#

Swap the classifier for a new number of outputs.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.