braindecode.models.TinyMyo#

class braindecode.models.TinyMyo(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, patch_size=20, embed_dim=192, depth=8, num_heads=3, mlp_ratio=4.0, qkv_bias=True, drop_prob=0.1, attn_drop=0.1, drop_path=0.1, activation=<class 'torch.nn.modules.activation.GELU'>, norm_layer=<class 'torch.nn.modules.normalization.LayerNorm'>, on_non_divisible='pad', n_channel_slots=16, init_std=0.02)[source]#

TinyMyo from Fasulo et al. (2026) [1].

Attention/Transformer Foundation Model

Added in version 2.1.

TinyMyo is a 3.6M-parameter Transformer encoder for surface EMG (sEMG). The authors run it on the GAP9 microcontroller. They pretrained it with masked patch reconstruction on NinaPro DB6, NinaPro DB7 and emg2pose (2000 Hz, up to 16 channels, 0.5 s windows), and fine-tune it for gesture classification, hand-kinematics regression and silent-speech decoding.

Architecture Overview

The model cuts each channel into patches of patch_size samples, and each patch becomes one token. One Transformer encoder reads the tokens of all channels, so attention mixes time and channels. With \(B\) the batch size, \(C\) = n_chans, \(P\) patches per channel and \(D\) = embed_dim:

x (B, C, n_times)
1. patch_embed: Linear on each patch  -> (B, C, P, D)
2. + channel_embed (one per channel slot)
3. channel-major tokens               -> (B, C * P, D)
4. blocks: RoPE self-attention + MLP  -> (B, C * P, D)
5. norm                               -> (B, C, P, D)   return_features
6. mean over P, flatten C and D       -> (B, C * D)
7. final_layer                        -> (B, n_outputs)

Macro Components

TinyMyo.patch_embed

Operations. One Linear shared by all channels maps each patch of patch_size samples to a D-wide token (a strided Conv2d in the reference, the same operation). Role. Channel-independent patching.

TinyMyo.channel_embed

Operations. A learned vector per channel index, added to every token of that channel. Role. Channel slots.

TinyMyo.blocks

Operations. Pre-norm blocks with rotary position embeddings (RoPE) on queries and keys. Role. Transformer encoder.

TinyMyo.final_layer

Operations. Averages the tokens of each channel over time, concatenates the channels and applies one Linear (EMGClassificationHead in the reference). Role. Classification head.

Temporal, Spatial, and Spectral Encoding

  • Time. The rotary positions restart at zero for each channel, so the first patch of every channel has the same position.

  • Channels. Channel slots follow the electrode order used in fine-tuning. The checkpoints have 16 slots, and a model with fewer channels uses the first n_chans.

  • Frequency. The patch projection takes time-domain EMG samples.

Additional Mechanisms

During training, each Transformer block applies stochastic depth to its attention and MLP residual branches.

Pretrained Weights

The authors’ weights (PulpBio/TinyMyo, CC BY-ND 4.0), converted to these parameter names, are on the Hugging Face Hub:

Repository

n_chans

Outputs

Task

braindecode/tinymyo-pretrained

up to 16

(encoder)

masked reconstruction

braindecode/tinymyo-db5

16

53

NinaPro DB5 gestures

braindecode/tinymyo-epn612

8

6

EPN-612 gestures

braindecode/tinymyo-uci-emg

8

6

UCI EMG gestures

Important

Fine-tune the pretrained encoder with a new head, or load a released classifier as it is:

from braindecode.models import TinyMyo

# Pretrained encoder, new 4-class head
model = TinyMyo.from_pretrained(
    "braindecode/tinymyo-pretrained", n_chans=8, n_outputs=4, n_times=1000
)

# Released EPN-612 classifier
model = TinyMyo.from_pretrained("braindecode/tinymyo-epn612")

The authors pretrain on 20-450 Hz band-passed signals with a 50 Hz notch, min-max normalized per channel. In fine-tuning they band-pass at 20-90 Hz and z-score each channel.

With the released weights (float32, CPU), the outputs match the reference (models/TinyMyo.py of BioFoundation at d4484fe) within 1e-5, and so do the gradients of the released classifiers (#1649); the re-hosted files reproduce the originals (PulpBio/TinyMyo at bc14906).

Differences from the Reference Implementation

  • This class ports the classification head only. The reference also has the pretraining decoder (with mask and pad_mask_ch inputs) and a convolutional regression head for NinaPro DB8, whose checkpoint is not re-hosted.

  • The reference requires n_times to equal its img_size. This class computes the RoPE tables for the input length, so any length works, and on_non_divisible handles a partial last patch. The authors trained the weights on 1000-sample windows.

Parameters:
  • n_outputs (Optional[int]) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (Optional[int]) – Number of EEG channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (Optional[int]) – Number of time samples of the input window.

  • input_window_seconds (Optional[float]) – Length of the input window in seconds.

  • sfreq (Optional[float]) – Sampling frequency of the EEG recordings.

  • patch_size (int) – Samples per patch.

  • embed_dim (int) – Width of the tokens.

  • depth (int) – Number of Transformer blocks (n_layer in the reference). The smaller TinyssimoMyo checkpoint has 4.

  • num_heads (int) – Attention heads per block. They must divide embed_dim, into heads of even width for RoPE.

  • mlp_ratio (float) – Hidden width of the MLP over embed_dim.

  • qkv_bias (bool) – Bias of the query, key and value projections.

  • drop_prob (float) – Dropout after the attention projection and in the MLP (proj_drop in the reference).

  • attn_drop (float) – Dropout on the attention weights.

  • drop_path (float) – Stochastic depth rate of each block.

  • activation (type[Module]) – Activation of the MLP.

  • norm_layer (type[Module]) – Normalization of the blocks and of the encoder output.

  • on_non_divisible (Literal['pad', 'crop', 'error']) – How PatchTokenizer treats a time axis that is not a multiple of patch_size.

  • n_channel_slots (int) – Number of channel slots (rows of channel_embed), which is the largest n_chans the model accepts. The released checkpoints have 16.

  • init_std (float) – Std of the truncated normal that initializes channel_embed, truncated at +-init_std.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.

References

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import TinyMyo

# Train your model
model = TinyMyo(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-tinymyo-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import TinyMyo

# Load pretrained model
model = TinyMyo.from_pretrained("username/my-tinymyo-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = TinyMyo.from_pretrained("username/my-tinymyo-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = TinyMyo.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x, return_features=False)[source]#

Forward pass.

Parameters:
  • x (Tensor) – Shape (batch, n_chans, n_times).

  • return_features (bool) – Return {"features": (batch, n_chans, n_patches, embed_dim), "cls_token": None}, the normed encoder tokens, instead of the logits.

Returns:

Logits of shape (batch, n_outputs), or the features.

Return type:

Tensor | dict[str, Optional[Tensor]]

reset_head(n_outputs)[source]#

Replace the classification head for a new number of outputs.

This is called automatically by from_pretrained() when the user passes an n_outputs that differs from the saved config. Override in subclasses that need a model-specific head structure. Implementations keep changed constructor arguments in sync with self._update_init_kwargs, so that a saved model can be loaded back. Implementations requiring positive outputs can also use self._set_n_outputs to validate and record the new value.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.