Load and save dataset example

In this example, we show how to load and save braindecode datasets.

# Authors: Lukas Gemein <l.gemein@gmail.com>
#
# License: BSD (3-clause)

from braindecode.datasets.moabb import MOABBDataset
from braindecode.datautil.preprocess import preprocess, MNEPreproc
from braindecode.datautil.serialization import load_concat_dataset
from braindecode.datautil.windowers import create_windows_from_events

First, we load some dataset using MOABB.

ds = MOABBDataset(
    dataset_name='BNCI2014001',
    subject_ids=[1],
)

Out:

48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]

We can apply preprocessing steps to the dataset. It is also possible to skip this step and not apply any preprocessing.

preprocess(
    concat_ds=ds,
    preprocessors=[MNEPreproc(fn='resample', sfreq=10)]
)

Out:

/home/runner/work/braindecode/braindecode/braindecode/datautil/preprocess.py:87: UserWarning: MNEPreproc is deprecated. Use Preprocessor with `apply_on_array=False` instead.
  warn('MNEPreproc is deprecated. Use Preprocessor with '
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]
48 events found
Event IDs: [1 2 3 4]

We save the dataset to a an existing directory. It will create a ‘.fif’ file for every dataset in the concat dataset. Additionally it will create two JSON files, the first holding the description of the dataset, the second holding the name of the target. If you want to store to the same directory several times, for example due to trying different preprocessing, you can choose to overwrite the existing files.

ds.save(
    path='./',
    overwrite=False,
)

Out:

Writing /home/runner/work/braindecode/braindecode/examples/0-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/0-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/1-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/1-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/2-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/2-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/3-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/3-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/4-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/4-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/5-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/5-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/6-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/6-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/7-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/7-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/8-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/8-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/9-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/9-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/10-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/10-raw.fif
[done]
Writing /home/runner/work/braindecode/braindecode/examples/11-raw.fif
Closing /home/runner/work/braindecode/braindecode/examples/11-raw.fif
[done]

We load the saved dataset from a directory. Signals can be preloaded in compliance with mne. Optionally, only specific ‘.fif’ files can be loaded by specifying their ids. The target name can be changed, if the dataset supports it (TUHAbnormal for example supports ‘pathological’, ‘age’, and ‘gender’. If you stored a preprocessed version with target ‘pathological’ it is possible to change the target upon loading).

ds_loaded = load_concat_dataset(
    path='./',
    preload=True,
    ids_to_load=[1,3],
    target_name=None,
)

Out:

Opening raw data file ./1-raw.fif...
    Range : 0 ... 3868 =      0.000 ...   386.800 secs
Ready.
Reading 0 ... 3868  =      0.000 ...   386.800 secs...
Opening raw data file ./3-raw.fif...
    Range : 0 ... 3868 =      0.000 ...   386.800 secs
Ready.
Reading 0 ... 3868  =      0.000 ...   386.800 secs...

The serialization utility also supports WindowsDatasets, so we create compute windows next.

windows_ds = create_windows_from_events(
    concat_ds=ds_loaded,
    trial_start_offset_samples=0,
    trial_stop_offset_samples=0,
)

Out:

Used Annotations descriptions: ['feet', 'left_hand', 'right_hand', 'tongue']
Adding metadata with 4 columns
Replacing existing metadata with 4 columns
48 matching events found
No baseline correction applied
0 projection items activated
Loading data for 48 events and 40 original time points ...
0 bad epochs dropped
Used Annotations descriptions: ['feet', 'left_hand', 'right_hand', 'tongue']
Adding metadata with 4 columns
Replacing existing metadata with 4 columns
48 matching events found
No baseline correction applied
0 projection items activated
Loading data for 48 events and 40 original time points ...
0 bad epochs dropped

Again, we save the dataset to an existing directory. It will create a ‘-epo.fif’ file for every dataset in the concat dataset. Additionally it will create a JSON file holding the description of the dataset. If you want to store to the same directory several times, for example due to trying different windowing parameters, you can choose to overwrite the existing files.

windows_ds.save(
    path='./',
    overwrite=True,
)

Out:

Loading data for 1 events and 40 original time points ...
Loading data for 48 events and 40 original time points ...
Loading data for 1 events and 40 original time points ...
Loading data for 48 events and 40 original time points ...
Load the saved dataset from a directory. Signals can be preloaded in

compliance with mne. Optionally, only specific ‘-epo.fif’ files can be

loaded by specifying their ids.

windows_ds_loaded = load_concat_dataset(
    path='./',
    preload=False,
    ids_to_load=[0],
    target_name=None,
)

Out:

Reading ./0-epo.fif ...
    Found the data of interest:
        t =       0.00 ...    3900.00 ms
        0 CTF compensation matrices available
Adding metadata with 4 columns
Replacing existing metadata with 4 columns
48 matching events found
No baseline correction applied
0 projection items activated

Total running time of the script: ( 0 minutes 3.653 seconds)

Estimated memory usage: 404 MB

Gallery generated by Sphinx-Gallery