braindecode.models.TCCNet#
- class braindecode.models.TCCNet(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, filters=((32, 32), (32, 32), (64, 64), (128, )), kernel_size=3, drop_prob=0.05, activation=<class 'torch.nn.modules.activation.ReLU'>, classification_head='att', mask_value=-10.0)[source]#
TCCNet from Tsinganos et al. (2019) [tccnet2019].
Convolution Recurrent Attention/Transformer
“Proposed model” panel of the authors’ ICASSP 2019 poster (MIT, from [tccnetcode]): dilated causal convolutions, read out by an average over time (AoT) or an attention (Att) head. Braindecode returns the logits, without the softmax.#
A temporal convolutional network that classifies a whole sEMG gesture. Residual blocks of causal convolutions, whose dilation doubles at every layer and keeps doubling across blocks, map the gesture to a sequence of features. An average over time (
"aot") or a context-vector attention ("att") pools that sequence into one vector for a linear classifier.The paper compares four models that differ in receptive field (RF) and head (poster Table 1; parameters counted for 10 channels and 53 classes):
Model
filtersConvs
Parameters
AoT, 300 ms
((32, 64), (64, 128))4
60,117
AoT, 2500 ms
((32, 32), (32, 32), (64, 64), (128,))7
72,245
Att, 300 ms
((32, 64), (64, 128))4
76,629
Att, 2500 ms
((32, 32), (32, 32), (64, 64), (128,))7
88,757
The defaults build Att, 2500 ms, the best model of the paper (89.76 % top-1). Pass
classification_head="aot"and/orfilters=((32, 64), (64, 128))for the other three. Withkernel_size=3, the RF is 31 samples (310 ms at 100 Hz) for the 4-convolution models and 255 samples (2550 ms) for the 7-convolution ones.Inputs are
(batch, n_chans, n_times)and may have any length. The softmax of the reference is left out, because braindecode models return logits.- Parameters:
n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.
n_chans (int) – Number of EEG channels.
chs_info (list of dict) – Information about each individual EEG channel. This should be filled with
info["chs"]. Refer tomne.Infofor more details.n_times (int) – Number of time samples of the input window.
input_window_seconds (float) – Length of the input window in seconds.
sfreq (float) – Sampling frequency of the EEG recordings.
filters (
Sequence[Sequence[int]]) – Output filters of each causal convolution, one inner sequence per residual block. It sets the number of blocks and of convolutions per block.kernel_size (
int) – Temporal kernel size of every causal convolution.drop_prob (
float) – Channel-wise dropout after each activated convolution, as KerasSpatialDropout1D.activation (
type[Module]) – Activation after each causal convolution.classification_head (
str) –"att": context-vector attention;"aot": average over the valid time steps.mask_value (
float|None) – A time step where every channel equals this value is padding: it is zeroed before every convolution and left out of the pooling, as the reference’s KerasMaskinglayer does.Noneturns masking off.
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
Protocol of the reference code [tccnetcode], for all four models: one model per subject of NinaPro DB1 (27 subjects, 10 sEMG channels at 100 Hz, 53 classes including rest). Each channel is rectified and low-pass filtered by a 1 Hz first-order Butterworth filter applied forward-backward (
filtfilt). Each whole gesture repetition is one sample: the variable-length gestures of a batch are pre-padded with-10to the longest one and masked. Repetitions 1, 3, 4, 6, 8, 9 and 10 train; 2, 5 and 7 test. Training uses Adam (learning rate 0.01, epsilon 0.001) for 30 epochs, batch size 128, L2 0.0005 on convolution and classifier kernels (use optimizer weight decay in PyTorch), class- balanced sample weights, and 10 augmented copies of each training gesture (time and magnitude warping 0.2, noise at 25 dB SNR). The generator’s min-max normalisation is off in the released configurations.The masked average over time equals the reference only for pre-padding, the reference pipeline: the reference sums every time step, padded ones included, and divides by the number of valid steps, while this port always leaves padded steps out; on a fully padded input it gives NaN, as the reference does. The bias of each convolution is added only where the convolution output is non-zero, as in the reference
CausalConv1D; this only matters for bit-exact agreement with it. The attention subtracts the row maximum before the exponential: the same values as the reference, which overflows to NaN once the context vector grows large. Initialisation follows Keras 2.2.4: truncated Glorot normal kernels, zero biases, Glorot uniform attention weights.References
[tccnet2019]Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., & Skodras, A. (2019). Improved Gesture Recognition Based on sEMG Signals and TCN. ICASSP 2019, 1169-1173. https://doi.org/10.1109/ICASSP.2019.8683239
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import TCCNet # Train your model model = TCCNet(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-tccnet-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import TCCNet # Load pretrained model model = TCCNet.from_pretrained("username/my-tccnet-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = TCCNet.from_pretrained("username/my-tccnet-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = TCCNet.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods