braindecode.models.TinyMyo#
- class braindecode.models.TinyMyo(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=20, embed_dim=192, depth=8, num_heads=3, mlp_ratio=4.0, qkv_bias=True, drop_prob=0.1, attn_drop=0.1, drop_path=0.1, activation=<class 'torch.nn.modules.activation.GELU'>, norm_layer=<class 'torch.nn.modules.normalization.LayerNorm'>, on_non_divisible='pad', n_channel_slots=16, init_std=0.02)[source]#
TinyMyo from Fasulo et al. (2026) [1].
Attention/Transformer Foundation Model
Added in version 2.1.
TinyMyo is a 3.6M-parameter Transformer encoder for surface EMG (sEMG). The authors run it on the GAP9 microcontroller. They pretrained it with masked patch reconstruction on NinaPro DB6, NinaPro DB7 and emg2pose (2000 Hz, up to 16 channels, 0.5 s windows), and fine-tune it for gesture classification, hand-kinematics regression and silent-speech decoding.
Architecture Overview
The model cuts each channel into patches of
patch_sizesamples, and each patch becomes one token. One Transformer encoder reads the tokens of all channels, so attention mixes time and channels. With \(B\) the batch size, \(C\) =n_chans, \(P\) patches per channel and \(D\) =embed_dim:x (B, C, n_times) 1. patch_embed: Linear on each patch -> (B, C, P, D) 2. + channel_embed (one per channel slot) 3. channel-major tokens -> (B, C * P, D) 4. blocks: RoPE self-attention + MLP -> (B, C * P, D) 5. norm -> (B, C, P, D) return_features 6. mean over P, flatten C and D -> (B, C * D) 7. final_layer -> (B, n_outputs)
Macro Components
TinyMyo.patch_embedOperations. One
Linearshared by all channels maps each patch ofpatch_sizesamples to aD-wide token (a stridedConv2din the reference, the same operation). Role. Channel-independent patching.TinyMyo.channel_embedOperations. A learned vector per channel index, added to every token of that channel. Role. Channel slots.
TinyMyo.blocksOperations. Pre-norm blocks with rotary position embeddings (RoPE) on queries and keys. Role. Transformer encoder.
TinyMyo.final_layerOperations. Averages the tokens of each channel over time, concatenates the channels and applies one
Linear(EMGClassificationHeadin the reference). Role. Classification head.
Temporal, Spatial, and Spectral Encoding
Time. The rotary positions restart at zero for each channel, so the first patch of every channel has the same position.
Channels. Channel slots follow the electrode order used in fine-tuning. The checkpoints have 16 slots, and a model with fewer channels uses the first
n_chans.Frequency. The patch projection takes time-domain EMG samples.
Additional Mechanisms
During training, each Transformer block applies stochastic depth to its attention and MLP residual branches.
Pretrained Weights
The authors’ weights (PulpBio/TinyMyo, CC BY-ND 4.0), converted to these parameter names, are on the Hugging Face Hub:
Repository
n_chansOutputs
Task
braindecode/tinymyo-pretrainedup to 16
(encoder)
masked reconstruction
braindecode/tinymyo-db516
53
NinaPro DB5 gestures
braindecode/tinymyo-epn6128
6
EPN-612 gestures
braindecode/tinymyo-uci-emg8
6
UCI EMG gestures
Important
Fine-tune the pretrained encoder with a new head, or load a released classifier as it is:
from braindecode.models import TinyMyo # Pretrained encoder, new 4-class head model = TinyMyo.from_pretrained( "braindecode/tinymyo-pretrained", n_chans=8, n_outputs=4, n_times=1000 ) # Released EPN-612 classifier model = TinyMyo.from_pretrained("braindecode/tinymyo-epn612")
The authors pretrain on 20-450 Hz band-passed signals with a 50 Hz notch, min-max normalized per channel. In fine-tuning they band-pass at 20-90 Hz and z-score each channel.
With the released weights (float32, CPU), the outputs match the reference (
models/TinyMyo.pyof BioFoundation atd4484fe) within 1e-5, and so do the gradients of the released classifiers (#1649); the re-hosted files reproduce the originals (PulpBio/TinyMyoatbc14906).Differences from the Reference Implementation
This class ports the classification head only. The reference also has the pretraining decoder (with
maskandpad_mask_chinputs) and a convolutional regression head for NinaPro DB8, whose checkpoint is not re-hosted.The reference requires
n_timesto equal itsimg_size. This class computes the RoPE tables for the input length, so any length works, andon_non_divisiblehandles a partial last patch. The authors trained the weights on 1000-sample windows.
- Parameters:
n_outputs (
Optional[int]) – Number of outputs of the model. This is the number of classes in the case of classification.chs_info (list of dict) – Information about each individual EEG channel. This should be filled with
info["chs"]. Refer tomne.Infofor more details.n_times (
Optional[int]) – Number of time samples of the input window.input_window_seconds (
Optional[float]) – Length of the input window in seconds.sfreq (
Optional[float]) – Sampling frequency of the EEG recordings.patch_size (
int) – Samples per patch.embed_dim (
int) – Width of the tokens.depth (
int) – Number of Transformer blocks (n_layerin the reference). The smaller TinyssimoMyo checkpoint has 4.num_heads (
int) – Attention heads per block. They must divideembed_dim, into heads of even width for RoPE.mlp_ratio (
float) – Hidden width of the MLP overembed_dim.qkv_bias (
bool) – Bias of the query, key and value projections.drop_prob (
float) – Dropout after the attention projection and in the MLP (proj_dropin the reference).attn_drop (
float) – Dropout on the attention weights.drop_path (
float) – Stochastic depth rate of each block.norm_layer (
type[Module]) – Normalization of the blocks and of the encoder output.on_non_divisible (
Literal['pad','crop','error']) – HowPatchTokenizertreats a time axis that is not a multiple ofpatch_size.n_channel_slots (
int) – Number of channel slots (rows ofchannel_embed), which is the largestn_chansthe model accepts. The released checkpoints have 16.init_std (
float) – Std of the truncated normal that initializeschannel_embed, truncated at+-init_std.
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.
References
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import TinyMyo # Train your model model = TinyMyo(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-tinymyo-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import TinyMyo # Load pretrained model model = TinyMyo.from_pretrained("username/my-tinymyo-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = TinyMyo.from_pretrained("username/my-tinymyo-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = TinyMyo.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods
- forward(x, return_features=False)[source]#
Forward pass.
- Parameters:
- Returns:
Logits of shape
(batch, n_outputs), or the features.- Return type:
- reset_head(n_outputs)[source]#
Replace the classification head for a new number of outputs.
This is called automatically by
from_pretrained()when the user passes ann_outputsthat differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync withself._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also useself._set_n_outputsto validate and record the new value.Examples
>>> from braindecode.models import BENDR >>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4) >>> model.reset_head(10) >>> model.n_outputs 10
Added in version 1.4.