This section describes how to fine-tune the pretrained Morpheus model on your own PSG dataset, using sleep staging as an example.
Depending on the format of your raw data, use one of the two builder functions in data_provider_2.py:
| Raw data format | Function to call |
|---|---|
| EDF files + annotation CSV | sleep_staging_finetune_from_edf_to_h5 |
H5 files (BCH format, with annotations/stage) |
sleep_staging_finetune_from_h5_to_h5 |
Both functions produce the same output layout:
<save_path>/
βββ EEG_dataset.hdf5
βββ ECG_dataset.hdf5
βββ EMG_CHIN_dataset.hdf5
βββ EMG_LEG_dataset.hdf5
βββ EOG_dataset.hdf5
βββ SNORE_dataset.hdf5
βββ RESP_dataset.hdf5
βββ NP_dataset.hdf5
βββ labels.hdf5 β per-window sleep stage labels (0=W,1=N1,2=N2,3=N3,4=REM,5=unscored)
Edit the call at the bottom of data_provider_2.py β main_pretraining_dataset() to set your input directory and output path, then run:
sudo $(which python) data_provider_2.pyThe script is resumable β if interrupted, re-running will skip already-processed subjects (when rewrite=False).
Edit run_finetune_multimodal.sh and set DATA_ROOT to your output directory from Step 1:
DATA_ROOT="/your/finetune/data/path"Then run:
bash run_finetune_multimodal.shThe fine-tuned checkpoint will be saved to checkpoints/finetune_sleep_staging/.
Edit run_infer_sleep_staging.sh to set your checkpoint, input folder, and output folder:
CHECKPOINT="./checkpoints/finetune_sleep_staging/checkpoint-19.pth"
INPUT_DIR="/your/psg/data/" # folder of .edf or .h5 files
OUTPUT_DIR="predictions/" # one CSV per input file
STRIDE_SEC=10 # seconds between windows (10 = non-overlapping)Then run:
bash run_infer_sleep_staging.shEach input file produces one CSV in OUTPUT_DIR:
window_idx, t_start_sec, predicted_stage, stage_name, W_prob, N1_prob, N2_prob, N3_prob, REM_prob, [true_stage]
0, 0.0, 2, N2, 0.03, 0.05, 0.87, 0.04, 0.01
1, 10.0, 4, REM, 0.01, 0.03, 0.10, 0.02, 0.84
...
STRIDE_SEC=10: one prediction per 10 s (non-overlapping, fastest)STRIDE_SEC=5: one prediction per 5 s (50% overlap)STRIDE_SEC=1: one prediction per 1 s (finest resolution, slowest)
If the input files contain ground-truth stage labels (annotations/stage in H5, or equivalent in EDF), accuracy is computed automatically and the average across all files is printed at the end.
Note: The rest of this repository covers tokenizer pretraining (
train_tokenizer.py,run_pretrain_tokenizer*.sh), multimodal transformer pretraining (pretrain_multimodal.py,run_pretrain_multimodal*.sh), full data pipeline construction (data_provider_2.py), and the original Morgoth models for other EEG tasks (event-level detection, EEG-level classification, etc.). If you are only fine-tuning on your own dataset, you can skip all of that.
This section describes how to train a multimodal polysomnography (PSG) sleep staging model from scratch using 8 physiological signal modalities: EEG, ECG, EMG (chin & leg), EOG, snoring sound, respiratory effort, and nasal pressure.
The training pipeline has three stages:
[Raw PSG signals]
β
βΌ
Step 1 β Tokenizer Pretraining (one VQNSP tokenizer per modality)
β
βΌ
Step 2 β Transformer Pretraining (multimodal MEM pretraining)
β
βΌ
Step 3 β Fine-tuning (sleep staging head, supervised)
Pre-trained tokenizers and the multimodal transformer checkpoint are available on AWS S3:
aws s3 cp s3://bdsp-opendata-credentialed/morpheus/model/checkpoints morpheus-internal/ --recursiveThis will populate the checkpoints/ directory with:
checkpoints/
βββ tokenizers/
β βββ eeg_tokenizer.pth
β βββ ecg_tokenizer_3.pth
β βββ emg_chin_tokenizer.pth
β βββ emg_leg_tokenizer_1.pth
β βββ eog_tokenizer_3.pth
β βββ snore_tokenizer.pth
β βββ resp_tokenizer.pth
β βββ np_tokenizer.pth
βββ pretrain_multimodal_early/
βββ checkpoint-59.pth
If you use the pre-trained models, you can skip Step 1 and Step 2 and go directly to Step 3 β Fine-tuning.
All HDF5 datasets are built from raw PSG recordings using data_provider_2.py. Two source formats are supported:
Option A β EDF files (MGH format)
/data/MGH_PSG/
βββ edf/ β one <fileid>.edf per PSG recording
βββ annotation/ β one <fileid>.csv per recording (sleep staging labels)
Annotation CSV columns: time (HH:MM:SS, start of each 30-second epoch) and event (e.g. Sleep_stage_W, Sleep_stage_N1, Sleep_stage_N2, Sleep_stage_N3, Sleep_stage_R).
Option B β H5 files (BCH format)
/data/BCH_PSG/
βββ <subject_id>.h5 β one H5 file per PSG recording
Each H5 file layout:
<root>.attrs['sampling_rate'] int
signals/<ch_name> shape=(T, 1) float32, unit: V
annotations/stage shape=(T, 1) float32, per-sample AASM label (0=W,1=N1,2=N2,3=N3,4=REM,5=unscored)
The preprocessing pipeline (_extract_one_modality) is identical for both formats β MNE FIR bandpass, notch filter, resampling to 200 Hz, and amplitude clipping β so either source produces the same HDF5 output layout.
Each modality is extracted from the source recordings independently and saved as a separate HDF5. Edit data_provider_2.py β main_pretraining_dataset() and uncomment the call for each modality, then run:
sudo $(which python) data_provider_2.pyPer-modality preprocessing applied during HDF5 construction:
| Modality | Channel(s) in EDF | Bandpass (Hz) | Resample | Clipping | Reference |
|---|---|---|---|---|---|
| EEG | F3/C3/O1/F4/C4/O2 (6 ch) | 0.5 β 70 | 200 Hz | Β±500 Β΅V | Average |
| ECG | EKG | 0.3 β 70 | 200 Hz | 1% percentile | β |
| EMG_CHIN | CHIN1-CHIN2 | 10 β 70 | 200 Hz | 1% percentile | β |
| EMG_LEG | LAT, RAT | 10 β 70 | 200 Hz | 1% percentile | β |
| EOG | E1-M2, E2-M1 | 0.3 β 35 | 200 Hz | 1% percentile | β |
| SNORE | SNORE | 10 β 70 | 200 Hz | 1% percentile | β |
| RESP | CHEST, ABD, AIRFLOW | 0.1 β 15 | 200 Hz | 1% percentile | β |
| NP | PTAF (or CFLOW) | 0.03 β 99 | 200 Hz | 1% percentile | β |
For EEG, four candidate montages are tried in order (first match wins):
| Priority | Channels | Post-processing |
|---|---|---|
| 1 | F3, C3, O1, F4, C4, O2 | Average reference applied |
| 2 | F3-M2, C3-M2, O1-M2, F4-M1, C4-M1, O2-M1 | Re-referenced to average |
| 3 | F3-AVG, C3-AVG, O1-AVG, F4-AVG, C4-AVG, O2-AVG | Already average-referenced |
| 4 | F3-M1, C3-M1, O1-M1, F4-M2, C4-M2, O2-M2 | Re-referenced to average |
Outputs are written to /data/PRETRAIN_DATASET/:
/data/PRETRAIN_DATASET/
βββ EEG_dataset.hdf5 # <subject>/eeg [6, T]
βββ ECG_dataset.hdf5 # <subject>/eeg [1, T]
βββ EMG_CHIN_dataset.hdf5 # <subject>/eeg [1, T]
βββ EMG_LEG_dataset.hdf5 # <subject>/eeg [2, T]
βββ EOG_dataset.hdf5 # <subject>/eeg [2, T]
βββ SNORE_dataset.hdf5 # <subject>/eeg [1, T]
βββ RESP_dataset.hdf5 # <subject>/eeg [3, T]
βββ NP_dataset.hdf5 # <subject>/eeg [1, T]
Each dataset stores raw signal as float32, with HDF5 attributes chOrder, lFreq, hFreq, rsFreq on every dataset.
The fine-tuning builder (make_multimodal_hdf5_for_MGH_finetune) reads the same EDF files, extracts all 8 modalities using the same per-modality preprocessing, and aligns 10-second windows to the 30-second annotation epochs.
Edit the main_pretraining_dataset() block at the bottom of data_provider_2.py to set _edf_dir, _ann_dir, _out_dir, and _max_subjects, then run:
sudo $(which python) data_provider_2.pyKey rules:
- A subject is included only if EEG is found plus at least 4 other modalities.
- Each 30-second annotation epoch maps to 3 Γ 10-second windows.
- Windows with no matching annotation are labeled
5(unscored).
Outputs are written to /data/FINETUNE_PSG/:
/data/FINETUNE_PSG/
βββ EEG_dataset.hdf5 # <subject>/eeg [6, T]
βββ ECG_dataset.hdf5 # ...
βββ EMG_CHIN_dataset.hdf5
βββ EMG_LEG_dataset.hdf5
βββ EOG_dataset.hdf5
βββ SNORE_dataset.hdf5
βββ RESP_dataset.hdf5
βββ NP_dataset.hdf5
βββ labels.hdf5 # <subject>/label [n_windows] dtype=int8
One HDF5 file per modality. Each file contains unlabeled PSG windows.
| File | Modality | Notes |
|---|---|---|
EEG_dataset.hdf5 |
EEG | Multi-channel, shape [n_ch, n_timepoints] per window |
ECG_dataset.hdf5 |
ECG | |
EMG_CHIN_dataset.hdf5 |
Chin EMG | |
EMG_LEG_dataset.hdf5 |
Leg EMG | |
EOG_dataset.hdf5 |
EOG | |
SNORE_dataset.hdf5 |
Snoring sound | |
RESP_dataset.hdf5 |
Respiratory effort | |
NP_dataset.hdf5 |
Nasal pressure |
Each HDF5 file layout: <subject_key> / <signal_name>, shape [n_ch, n_timepoints], with a chOrder attribute listing channel names.
Tip: The first run on a large dataset will have many cold-page cache misses because data is accessed sequentially. Warm up the page cache beforehand:
cat /data/PRETRAIN_DATASET/*.hdf5 > /dev/null && bash run_pretrain_multimodal_early.sh
Same modality HDF5 files as above (same format), plus a label file:
| File | Description |
|---|---|
EEG_dataset.hdf5 β¦ NP_dataset.hdf5 |
Signal windows (same format as pretrain) |
labels.hdf5 |
<subject_key> / label, shape [n_windows], dtype int |
Label encoding for 6-class sleep staging:
| Value | Stage |
|---|---|
| 0 | Wake (W) |
| 1 | N1 |
| 2 | N2 |
| 3 | N3 |
| 4 | REM |
| 5 | Unscored / unknown |
Each modality is tokenized independently using a VQNSP (Vector-Quantized Neural Signal Processing) encoder-decoder. Train each tokenizer on its own HDF5 file.
Checkpoints used by downstream stages:
| Modality | Checkpoint file | Codebook size | Code dim |
|---|---|---|---|
| EEG | checkpoints/tokenizers/eeg_tokenizer.pth |
8192 | 64 |
| ECG | checkpoints/tokenizers/ecg_tokenizer_3.pth |
1024 | 64 |
| EMG_CHIN | checkpoints/tokenizers/emg_chin_tokenizer.pth |
512 | 64 |
| EMG_LEG | checkpoints/tokenizers/emg_leg_tokenizer_1.pth |
768 | 64 |
| EOG | checkpoints/tokenizers/eog_tokenizer_3.pth |
640 | 64 |
| SNORE | checkpoints/tokenizers/snore_tokenizer.pth |
512 | 64 |
| RESP | checkpoints/tokenizers/resp_tokenizer.pth |
512 | 64 |
| NP | checkpoints/tokenizers/np_tokenizer.pth |
256 | 64 |
Run (example for EMG_LEG and EMG_CHIN, 2 GPUs):
bash run_pretrain_tokenizer.shOr call train_tokenizer.py directly for a single modality:
torchrun --standalone --nproc_per_node=2 train_tokenizer.py \
--output_dir checkpoints/ecg_tokenizer_3/ \
--log_dir log/ecg_tokenizer_3/ \
--dataset ECG \
--dataset_dir /data/PRETRAIN_DATASET/ECG_dataset.hdf5 \
--model vqnsp_encoder_base_decoder_3x200x12 \
--codebook_size 1024 \
--codebook_n_emd 1024 \
--quantize_kmeans_init \
--lr 5e-4 --batch_size 128 --epochs 10 --warmup_epochs 1Evaluate tokenizers (codebook usage, reconstruction quality):
bash run_evaluate_tokenizers.shResults are written to eval_out/<MODALITY>/.
Pretrain a multimodal Transformer on all 8 tokenized modalities using Masked Embedding Modeling (MEM). Three fusion strategies are available β choose one:
| Strategy | Script | Description |
|---|---|---|
| Early Fusion (recommended) | run_pretrain_multimodal_early.sh |
All tokens concatenated into one shared Transformer (241 tokens total) |
| Late Fusion | run_pretrain_multimodal_late.sh |
One independent Transformer per modality; CLS tokens fused at the end |
| Cross-Attention Fusion | run_pretrain_multimodal_cross.sh |
EEG as primary modality; other modalities attend to EEG via cross-attention |
Run early fusion (2 GPUs, adjust CUDA_VISIBLE_DEVICES as needed):
bash run_pretrain_multimodal_early.shKey hyperparameters (set inside the script):
| Parameter | Value | Description |
|---|---|---|
--embed_dim |
200 | Transformer hidden size |
--depth |
12 | Number of Transformer layers |
--num_heads |
10 | Attention heads |
--mask_ratio |
0.5 | Fraction of tokens masked during pretraining |
--epochs |
60 | Total training epochs |
--batch_size |
128 | Per-GPU batch size |
Checkpoints are saved to checkpoints/pretrain_multimodal_early/.
Monitor training curves:
python plot_pretrain_curves.py \
--log_txt checkpoints/pretrain_multimodal_early/log.txt \
--output_dir eval_out/pretrainOutputs eval_out/pretrain/pretrain_curves.png showing loss, learning rate, gradient norm, and AMP loss scale.
Resuming: The script automatically resumes from the latest checkpoint. Make sure
--epochsis set to a value greater than the already-completed epoch count.
Fine-tune the pretrained backbone with a sleep staging head (average pooling β LayerNorm β Linear(200 β 6)) for 6-class PSG sleep staging.
Prerequisite: Step 2 must have produced a checkpoint, e.g. checkpoints/pretrain_multimodal_early/checkpoint-59.pth.
Run (2 GPUs):
bash run_finetune_multimodal.shKey settings inside the script:
| Parameter | Value | Description |
|---|---|---|
--finetune |
checkpoint-59.pth |
Pretrained backbone to load |
--label_file |
labels.hdf5 |
Per-window sleep stage labels |
--nb_classes |
6 | W / N1 / N2 / N3 / REM / Unscored |
--train_split |
0.8 | Fraction of subjects used for training |
--epochs |
20 | Fine-tuning epochs |
--layer_decay |
0.65 | Layer-wise LR decay for backbone |
--window_size |
2000 | Samples per epoch window |
Checkpoints are saved to checkpoints/finetune_sleep_staging/.
Evaluate the fine-tuned model:
bash run_evaluate_finetune.shEdit run_evaluate_finetune.sh to set:
CKPTβ path to the fine-tuned checkpoint (e.g.checkpoint-59.pth)--subject_start/--subject_endβ index range of validation subjects to evaluate
Results (accuracy, Cohen's kappa, per-class F1, confusion matrix, ROC/PR curves) are written to eval_out/finetune/.
The content below documents the original Morgoth EEG interpretation model (event-level and EEG-level tasks). It is kept here for reference.
This repository contains code and tools for running EEG analysis using Morgoth, including event-level and EEG-level detection with probabilities for:
- Normal / Abnormal (binary)
- Slowing: No Slowing / Focal Slowing / Generalized Slowing (in the same order as each column in the corresponding output file)
- Burst suppression: No / Burst suppression (binary)
- Spike detection: No / Spike (binary)
- Spike localization: No / Focal Spike / Generalized Spike (in the same order as each column in the corresponding output file)
- IIIC classification: Other / Seizure / LPD / GPD / LRDA / GRDA (in the same order as each column in the corresponding output file)
- IIIC classification with chewing artifact correction (binary per class, uses an auxiliary high-frequency artifact model)
- Sleep staging (5 stages): Awake / N1 / N2 / N3 / REM (in the same order as each column in the corresponding output file)
- Sleep staging (3 stages): Awake / N1 / N2 (in the same order as each column in the corresponding output file)
- Sleep staging (6 stages): Awake / N1 / N2 / N3 / REM / Other (in the same order as each column in the corresponding output file)
- Sleep arousal detection: No / Arousal (binary)
- Vertex wave detection: No / Vertex wave (binary)
- BIRD (Brief Ictal Rhythmic Discharge): No / BIRD (binary)
- BIPD (Bilateral Independent Periodic Discharges): No / BIPD (binary)
- Partial PD (focal periodic discharges with lateralization): No / Left / Right (in the same order as each column in the corresponding output file)
βββ test_data/ # EEG data (.mat/.pkl/.edf)
βββ checkpoints/ # Pretrained models and checkpoints
βββ xxx.py # Code files
βββ xxx.sh/bat # Run model
βββ README.md # Project overview
Before running the code, please download the pretrained model (checkpoints) and test dataset (test_data) from AWS S3 and place them in the appropriate folders:
aws s3 cp s3://bdsp-opendata-credentialed/morgoth2/models/202605/morgoth checkpoints/ --recursiveconda create -n morgoth python=3.12
conda activate morgoth
pip install -r requirements.txt / conda install --file requirements.txt
# If you have GPU with cuda driver
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidiaconda create -n morgoth python=3.12
conda activate morgoth
pip install -r requirements_cuda122.txt
# If you have GPU with cuda driver
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidiaThe model supports raw EEG data in both EDF and MAT formats as input.
This means you can use unaltered clinical recordings directly β no manual preprocessing is required. The model automatically performs all necessary preprocessing steps, including:
Bandpass filtering, Resampling, Montage, Clipping, Normalization, and Epoching.
-
Ensure the raw EEG file contains sampling rate and channel names, or
-
Provide this information explicitly via command-line arguments, or
-
Channel order must follow the standard as specified in the corresponding bash scripts.
Please refer to the continuous_event_level.sh, discrete_event_level.sh, or EEG_level.sh for example configurations and expected input parameters.
This design ensures a streamlined and reproducible workflow, allowing you to run the model on raw EEG files directly, without requiring prior signal processing expertise.
See the comments at the beginning of the Bash file to understand the meaning of each parameter in the command. You can modify parameters inside each script.
bash continuous_event_level.shSee the comments at the beginning of the Bash file to understand the meaning of each parameter in the command. You can modify parameters inside each script.
The data in test_data folder should be processed first.
bash discrete_event_level.shSee the comments at the beginning of the Bash file to understand the meaning of each parameter in the command. You can modify parameters inside each script.
The EEG_level inputs are the event-level output results with 1-second slipping step.
bash eeg_level.shTo train a Morgoth model from scratch using your own data:
You should modify the data_provider.py script according to your dataset
echo password | sudo -S ~/miniconda3/envs/torchenv/bin/python data_provider.pybash run_pretrain.shbash train_classification.shbash train_EEG_level_head.shYou may modify the script or config file to set the number of epochs, learning rate, batch size, model type, etc.
Make sure you have sufficient GPU memory for large models or long EEG recordings.
Reinstall NumPy using Conda with the command:
conda install numpy=1.26.4Add layer clip for cuda index out of boundary in EEG_level_head.sh, class CNNTransformerClassifier, def forward
# check the valid of index
if indices.max() >= x.size(1) or indices.min() < 0:
indices = torch.clamp(indices, 0, x.size(1) - 1)When encountering pandas and numpy compatibility issues, please reinstall NumPy 1.26.4 using Conda instead of pip.
conda install numpy=1.26.4If you cannnot run Morgoth on long EEG files because of GPU memory limitations. To perform continuous event-level detection, run the following command:
bash continuous_event_level_longeeg.sh
This script will automatically segment the EEG files and batch them through Morgoth. The final output will contain predictions for each segment. In the script, you'll need to specify the following arguments:
--data_format edf \ # format of the raw EEG data
--segment_duration 600 \ # duration of each segment in seconds (here, 10 minutes)
--eeg_dir test_data/longEEG/raw \ # directory containing the original raw EEG files
--eval_sub_dir test_data/longEEG/segments_raw \ # directory to store the segmented files
continuous_event_level_longeeg.sh included examples in the script for IIIC, spike, slowing, and burst suppression detection. You can follow a similar format for other tasks. Just make sure to keep the --eval_sub_dir path consistent across all commands.
If you need to merge the segment-level results into a full-length prediction, note that you can't just concatenate them β due to the sliding step, you'll need to align each segment by padding the predictions before merging.
Wrong code in utils.py resize_array_along_axis0()
Change
return arr[indices:,]to
return arr[indices,: ]If your GPU driver is β€ 550, please switch to a different CUDA version (e.g., pytorch-cuda=12.1)
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidiato
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidiaIt shows a torch._C error: this may be due to the Conda version itself. Newer Conda uses different download sources than older ones, so the CUDA installed by Conda conflicted with the Torch installed by pip. To fix this, I removed torch from requirements.txt and modified the Conda installation command to install Torch with Conda instead. In other words, the installation steps for Conda and pip needed to be swapped.
- Create and activate the environment
conda create -n morgoth python=3.12 conda activate morgoth- conda torch cuda
conda install -y -c pytorch -c nvidia pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 pytorch-cuda=12.4If 404:
conda install -y -c pytorch -c nvidia/label/cuda-12.4.127 pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 pytorch-cuda=12.4- pip others
change requirements.txt to:
apex==0.9.10dev
botocore==1.34.154
einops==0.8.0
h5py==3.12.1
hdf5storage==0.1.19
mat73==0.65
matplotlib==3.9.2
mne==1.8.0
mne_connectivity==0.7.0
mypy==1.11.2
pandas==1.5.3
pyhealth==1.1.6
pytz==2024.1
scikit-learn==1.5.2
scipy==1.14.1
sympy==1.13.2
tensorboardX==1.8
timm==1.0.11
tqdm==4.66.5
pytest==7.4.4
colorcet==3.1.0
protobuf==3.20.3
numpy==1.26.4pip install -r requirements.txt - If pandas or numpy error
conda install numpy=1.26.4 For questions, please contact:
Chenxi Sun β csun8@bidmc.harvard.edu
cxsun@stanford.edu