braindecode.models.NeuroPose#
- class braindecode.models.NeuroPose(n_outputs=None, n_chans=None, chs_info=None, n_times=None, input_window_seconds=None, sfreq=None, *, encoder_channels=(32, 128, 256), encoder_pool_sizes=((10, 2), (8, 2), (4, 4)), decoder_channels=(128, 32, 1), decoder_upsample_sizes=((10, 4), (8, 4), (4, 2)), n_res_blocks=5, n_convs_per_block=3, kernel_size=(3, 2), activation=<class 'torch.nn.modules.activation.ReLU'>, drop_prob=0.05)[source]#
NeuroPose from Liu et al (2021) [liu2021neuropose].
Convolution
Figure 6 of Liu et al. (2021), CC BY. The three stages map onto
NeuroPose’sencoder,resnetanddecoder.#Convolutional encoder-decoder with a residual bottleneck mapping wearable sEMG windows to continuous finger joint angles. The defaults reproduce the configuration emg2pose [salter2024] used for its published NeuroPose baseline, which widens Liu et al.’s original pooling schedule for a device with 10x the sampling rate and 2x the spatial resolution.
Architecture Overview
(B, n_chans, T)→ add a feature axis, giving(B, 1, T, n_chans)→ three Conv2d-BN-ReLU-Dropout-MaxPool encoder stages downsampling (time x electrodes) by (10, 2), (8, 2), (4, 4) →n_res_blocksresidual blocks → three mirrored Conv-BN-ReLU-Dropout-Upsample decoder stages restoring the temporal axis → flatten the feature and electrode axes → linear head →(B, T, n_outputs).Macro Components
NeuroPose.encoderOperations. Three Conv2d-BN-ReLU-Dropout-MaxPool blocks with widths
encoder_channelsandpadding="same"3x2 kernels, pooling (time x electrodes) byencoder_pool_sizes. Role. Compresses the window along time while mixing adjacent electrodes, until the electrode axis is a single position.NeuroPose.resnet(residual bottleneck)Operations.
n_res_blocksblocks computingx + f(x), wherefisn_convs_per_blockConv-BN-ReLU-Dropout groups at constant width. No activation follows the sum. Role. The paper’s key accuracy lever: deeper feature extraction without convergence loss ({3, 5, 7} swept upstream).NeuroPose.decoder/NeuroPose.final_layerOperations. Three Conv-BN-ReLU-Dropout-Upsample(nearest) stages invert the encoder’s pooling;
final_layermaps the flattened feature/electrode axes ton_outputsper frame. Role. Dense per-frame joint-angle regression.
Temporal, Spatial, and Spectral Encoding
Time: The pooling/upsampling pyramid captures temporal structure.
Channels/space: Small 2-D kernels mix neighboring electrodes.
Frequency: Spectral content is learned from the raw waveform; unlike a decimating front end, no band is discarded up front.
Additional Mechanisms
The paper’s anatomical constraints (bounded activations over joint ranges), group-wise MSE losses and the temporal-smoothness penalty are training-side concerns outside this module; its BN-freezing transfer-learning recipe applies unchanged (freeze all but BatchNorm layers, fine-tune on ~90 s of target-user data).
Relationship to emg2pose’s NeuroPose baseline
With default arguments this class is operation-equivalent to emg2pose’s
network/neuropose.yaml: the same block widths, kernels, pooling and upsampling schedule, the same residual depth, and the same 6,354,903 parameters atn_chans=16, n_outputs=20.One deviation remains, and it is a superset rather than a difference: upstream requires the window length to be divisible by the total temporal pooling factor (320 by default), having no way to recover the samples that flooring discards. This class interpolates the decoded sequence back to
T, which is exactly the identity whenTis divisible by that factor and well-defined when it is not.Pre-trained weights
Meta’s released
regression_neuropose.ckptis rehosted with these parameter names, so no conversion step is needed:NeuroPose.from_pretrained("braindecode/NeuroPose-emg2pose")
The rehosted weights reproduce the reference implementation exactly. They stay under emg2pose’s CC BY-NC-SA 4.0 license; only the code here is BSD-3.
Liu et al.’s original 200 Hz Myo configuration remains reachable by passing the original schedule:
NeuroPose( n_chans=8, n_outputs=16, n_times=1_000, sfreq=200.0, encoder_pool_sizes=((5, 2), (4, 2), (2, 2)), decoder_upsample_sizes=((5, 4), (4, 2), (2, 2)), n_res_blocks=3, n_convs_per_block=2, )
Added in version 1.8.
- Parameters:
n_outputs (int) – Number of outputs of the model. This is the number of classes in the case of classification.
n_chans (int) – Number of EEG channels.
chs_info (list of dict) – Information about each individual EEG channel. This should be filled with
info["chs"]. Refer tomne.Infofor more details.n_times (int) – Number of time samples of the input window.
input_window_seconds (float) – Length of the input window in seconds.
sfreq (float) – Sampling frequency of the EEG recordings.
encoder_channels (
tuple[int,...]) – Output widths of the encoder stages. The default is(32, 128, 256)(emg2pose).encoder_pool_sizes (
tuple[tuple[int,int],...]) – Per-stage max-pool factors as(time, electrodes). The default is((10, 2), (8, 2), (4, 4)); Liu et al. use((5, 2), (4, 2), (2, 2))at 200 Hz.decoder_channels (
tuple[int,...]) – Output widths of the decoder stages. The default is(128, 32, 1).decoder_upsample_sizes (
tuple[tuple[int,int],...]) – Per-stage nearest-neighbour upsampling factors as(time, electrodes). The default is((10, 4), (8, 4), (4, 2)).n_res_blocks (
int) – Residual blocks between encoder and decoder. The default is5.n_convs_per_block (
int) – Convolutions inside each residual block. The default is3.kernel_size (
tuple[int,int]) – Convolution kernel as(time, electrodes), applied withpadding="same"throughout. The default is(3, 2).activation (
type[Module]) – Activation class used throughout. The default isnn.ReLU.drop_prob (
float) – Dropout probability inside every block. The default is0.05(paper value).
- Raises:
ValueError – If some input signal-related parameters are not specified: and can not be inferred.
Notes
If some input signal-related parameters are not specified, there will be an attempt to infer them from the other parameters.
References
[liu2021neuropose]Liu, Zhang, Gowda (2021). NeuroPose: 3D Hand Pose Tracking using EMG Wearables. The Web Conference 2021 / IEEE Internet of Things Journal 46(1), 2022. doi:10.1145/3442381.3449890
[salter2024]Salter, Warren, Schlager, Spurr, Han, Bhasin, Cai, Walkington, Bolarinwa, Wang, Wang, Danielson, Merel, Pnevmatikakis, Marshall (2024). emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation. NeurIPS Datasets and Benchmarks. arXiv:2412.02725
Hugging Face Hub integration
When the optional
huggingface_hubpackage is installed, all models automatically gain the ability to be pushed to and loaded from the Hugging Face Hub. Install with:pip install braindecode[hub]
Pushing a model to the Hub:
from braindecode.models import NeuroPose # Train your model model = NeuroPose(n_chans=22, n_outputs=4, n_times=1000) # ... training code ... # Push to the Hub; ``datasets`` and ``base_model`` go to the card model.push_to_hub( repo_id="username/my-neuropose-model", commit_message="Initial model upload", model_card_kwargs={"datasets": ["BNCI2014_001"]}, )
Loading a model from the Hub:
from braindecode.models import NeuroPose # Load pretrained model model = NeuroPose.from_pretrained("username/my-neuropose-model") # Data the weights were trained on, from the Hub model card print(model.card_data.datasets if model.card_data else None) # Load with a different number of outputs (head is rebuilt automatically) model = NeuroPose.from_pretrained("username/my-neuropose-model", n_outputs=4)
Extracting features and replacing the head:
import torch x = torch.randn(1, model.n_chans, model.n_times) # Extract encoder features (consistent dict across all models) out = model(x, return_features=True) features = out["features"] # Replace the classification head model.reset_head(n_outputs=10)
Saving and restoring full configuration:
import json config = model.get_config() # all __init__ params with open("config.json", "w") as f: json.dump(config, f) model2 = NeuroPose.from_config(config) # reconstruct (no weights)
All model parameters (both EEG-specific and model-specific such as dropout rates, activation functions, number of filters) are automatically saved to the Hub and restored when loading.
See Loading and Adapting Pretrained Foundation Models for a complete tutorial.
Methods
- forward(x)[source]#
Define the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
