braindecode.models.SensingDynamics#

class braindecode.models.SensingDynamics(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, temporal_channels=256, mid_channels=32, spatial_channels=64, mlp_hidden=512, drop_prob=0.25, mlp_drop_prob=0.4, lowpass_hz=20.0, lowpass_order=4, temporal_kernel=(1, 31), temporal_stride=(1, 8), mid_kernel=(8, 18), mid_dilation=(2, 1), spatial_kernel=(3, 1), electrode_wrap=4, activation=<class 'braindecode.modules.activation.SmoothMaximumUnit'>)[source]#

SensingDynamics from Sîmpetru et al. (2022) [simpetru2022].

Convolution Channel

SensingDynamics architecture with circular electrode padding

Figure 7 of Sîmpetru et al. (2022), CC BY-ND. Raw and low-pass filtered sEMG enter stacked, the electrode axis is circularly padded, and a 512/512 MLP reads out the joint angles.#

Architecture Overview

The emg2pose adaptation replaces the original five-grid 3D convolutions with 2D convolutions over the 16 wristband electrodes and time. Dividing the original spatial geometry by four preserves its relative receptive field. The network directly regresses joint angles.

Macro Components

SensingDynamics.lowpass (spectral input copy)

Operations. Periodic fourth-order, zero-phase 20 Hz Butterworth response. Role. Exposes the low-frequency neural drive alongside broadband monopolar EMG.

SensingDynamics.conv1 (action-potential detector)

Operations. Conv2d (1, 31), stride (1, 8), BatchNorm, SMU, and spatial dropout. Role. Detects short motor-unit waveform patterns independently at each electrode.

SensingDynamics.conv2 / SensingDynamics.conv3 (ring mixer)

Operations. Circular padding by four electrodes; dilated Conv2d (8, 18) with dilation (2, 1); Conv2d (3, 1); BatchNorm and SMU after both convolutions. Role. Mixes synchronous activity around the wristband with a 17-electrode spatial and 167-sample temporal receptive field.

SensingDynamics.mlp / SensingDynamics.final_layer

Operations. Flatten the 64 by 8 spatial features with an einops layer, then apply a 512, 512, n_outputs MLP. Role. Regresses joint angles at the feature rate before linear interpolation to the input grid.

Temporal, Spatial, and Spectral Encoding

  • Time: valid temporal kernels cover 167 input samples and stride by 8.

  • Channels/space: circular padding respects the wristband topology.

  • Frequency: broadband EMG is depth-stacked with a 20 Hz low-pass copy.

Additional Mechanisms

SMU keeps its published fixed alpha=0.01 and independently learnable mu=2.5 initialization. The source paper’s 150 ms output moving average is postprocessing and is not part of this neural-network module. Pass an externally filtered tensor as x_lowpass to reproduce a particular historical filter’s boundary handling exactly. The emg2pose benchmark reports a 10,167-sample training window for this baseline; it does not publish how the adapted network assigns its 167-sample receptive field between target alignment and window context.

Note

The emg2pose paper specifies that kernels, strides, and dilations were changed to preserve the original receptive fields, but its public code and checkpoint omit SensingDynamics. The 16-channel spatial dimensions in this implementation are consequently derived from that published rule; exact checkpoint equivalence cannot be asserted without the unreleased implementation.

Added in version 1.8.

Parameters:

  • n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.

  • n_chans (int) – Number of EEG channels.

  • chs_info (list of dict) – Information about each individual EEG channel. This should be filled with info["chs"]. Refer to mne.Info for more details.

  • n_times (int) – Number of time samples of the input window.

  • input_window_seconds (float) – Length of the input window in seconds.

  • sfreq (float) – Sampling frequency of the EEG recordings.

  • temporal_channels (int) – Output width of the action-potential detector. Default is 256.

  • mid_channels (int) – Output width of the dilated electrode mixer. Default is 32.

  • spatial_channels (int) – Output width of the final convolution. Default is 64.

  • mlp_hidden (int) – Width of both hidden MLP layers. Default is 512.

  • drop_prob (float) – Spatial dropout after the first convolution. Default is 0.25.

  • mlp_drop_prob (float) – Dropout before the MLP. Default is 0.4.

  • lowpass_hz (float) – Cutoff of the Butterworth input copy. Default is 20.

  • lowpass_order (int) – Order of that Butterworth filter. Default is 4.

  • temporal_kernel (tuple[int, int]) – (electrodes, time) kernel of the action-potential detector. Default is (1, 31).

  • temporal_stride (tuple[int, int]) – Stride of that convolution. Default is (1, 8).

  • mid_kernel (tuple[int, int]) – (electrodes, time) kernel of the electrode mixer. Default is (8, 18).

  • mid_dilation (tuple[int, int]) – Dilation of the mixer. Default is (2, 1).

  • spatial_kernel (tuple[int, int]) – (electrodes, time) kernel of the final convolution. Default is (3, 1).

  • electrode_wrap (int) – Circular padding applied to each end of the electrode axis, which is a ring on the wristband. Default is 4.

  • activation (type[Module]) – Activation class. The default is the paper’s learnable SMU.

Raises:

ValueError – If some input signal-related parameters are not specified: and can not be inferred.

Notes

If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.

References

[simpetru2022]

Sîmpetru, Arkudas, Braun, Osswald, Oliveira, Eskofier, Kinfe, Del Vecchio (2022). Sensing the Full Dynamics of the Human Hand with a Neural Interface and Deep Learning. bioRxiv 2022.07.29.502064. doi:10.1101/2022.07.29.502064

Hugging Face Hub integration

When the optional huggingface_hub package is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:

pip install braindecode[hub]

Pushing a model to the Hub:

from braindecode.models import SensingDynamics

# Train your model
model = SensingDynamics(n_chans=22, n_outputs=4, n_times=1000)
# ... training code ...

# Push to the Hub; ``datasets`` and ``base_model`` go to the card
model.push_to_hub(
    repo_id="username/my-sensingdynamics-model",
    commit_message="Initial model upload",
    model_card_kwargs={"datasets": ["BNCI2014_001"]},
)

Loading a model from the Hub:

from braindecode.models import SensingDynamics

# Load pretrained model
model = SensingDynamics.from_pretrained("username/my-sensingdynamics-model")
# Data the weights were trained on, from the Hub model card
print(model.card_data.datasets if model.card_data else None)

# Load with a different number of outputs (head is rebuilt automatically)
model = SensingDynamics.from_pretrained("username/my-sensingdynamics-model", n_outputs=4)

Extracting features and replacing the head:

import torch

x = torch.randn(1, model.n_chans, model.n_times)
# Extract encoder features (consistent dict across all models)
out = model(x, return_features=True)
features = out["features"]

# Replace the classification head
model.reset_head(n_outputs=10)

Saving and restoring full configuration:

import json

config = model.get_config()            # all __init__ params
with open("config.json", "w") as f:
    json.dump(config, f)

model2 = SensingDynamics.from_config(config)    # reconstruct (no weights)

All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.

See Loading and Adapting Pretrained Foundation Models for a complete tutorial.

Methods

forward(x, x_lowpass=None)[source]#

Predict angles, optionally using an externally filtered input copy.

Parameters:

  • x (Tensor) – The description is missing.

  • x_lowpass (Tensor | None) – The description is missing.

Return type:

Tensor

reset_head(n_outputs)[source]#

Swap the output layer for a new output dimensionality.

Parameters:

n_outputs (int) – New number of output classes.

Return type:

None

Examples

>>> from braindecode.models import BENDR
>>> model = BENDR(n_chans=22, n_times=1000, n_outputs=4)
>>> model.reset_head(10)
>>> model.n_outputs
10

Added in version 1.4.