Automatic whole-spine segmentation of MR (and CT) images — a two-phase approach to multi-class semantic and instance segmentation, with VERIDAH ("Solving Enumeration Anomaly Aware Vertebra Labeling across Imaging Sequences") for anatomical vertebra labeling.
SPINEPS automatically and robustly segments the whole spine in sagittal images.
There is a new release that finally supports both CT and T2W with completely independent, modality specific models. We are already working on completely modality/sequence robust version that works on everything. Stay tuned for that.
📖 Online documentation: spineps.readthedocs.io
The documentation source lives in the docs/ folder and is built with MkDocs
(Material theme + mkdocstrings). To build and preview it locally:
pip install mkdocs mkdocs-material "mkdocstrings[python]"
mkdocs serve # then open http://127.0.0.1:8000Start with docs/index.md and the Getting Started guide.
If you are using SPINEPS, please cite the following:
SPINEPS:
Hendrik Möller, Robert Graf, Joachim Schmitt, Benjamin Keinert, Hanna Schön, Matan Atad,
Anjany Sekuboyina, Felix Streckenbach, Florian Kofler, Thomas Kroencke, Stefanie Bette,
Stefan N. Willich, Thomas Keil, Thoralf Niendorf, Tobias Pischon, Beate Endemann, Bjoern Menze,
Daniel Rueckert, Jan S. Kirschke. SPINEPS—automatic whole spine segmentation of
T2-weighted MR images using a two-phase approach to multi-class semantic and instance segmentation.
Eur Radiol (2024). https://doi.org/10.1007/s00330-024-11155-y
Source of the T2w/T1w Segmentation:
Robert Graf, Joachim Schmitt, Sarah Schlaeger, Hendrik Kristian Möller, Vasiliki
Sideri-Lampretsa, Anjany Sekuboyina, Sandro Manuel Krieg, Benedikt Wiestler, Bjoern
Menze, Daniel Rueckert, Jan Stefan Kirschke. Denoising diffusion-based MRI to CT image
translation enables automated spinal segmentation. Eur Radiol Exp 7, 70 (2023).
https://doi.org/10.1186/s41747-023-00385-2
SPINEPS:
Paper link: https://link.springer.com/article/10.1007/s00330-024-11155-y#citeas
Source of the T2w/T1w Segmentation:
Open Access link: https://doi.org/10.1186/s41747-023-00385-2
BibTeX citation:
@article{moller_spinepsautomatic_2024,
title = {{SPINEPS}—automatic whole spine segmentation of T2-weighted {MR} images using a two-phase approach to multi-class semantic and instance segmentation},
issn = {1432-1084},
url = {https://doi.org/10.1007/s00330-024-11155-y},
doi = {10.1007/s00330-024-11155-y},
abstract = {Introducing {SPINEPS}, a deep learning method for semantic and instance segmentation of 14 spinal structures (ten vertebra substructures, intervertebral discs, spinal cord, spinal canal, and sacrum) in whole-body sagittal T2-weighted turbo spin echo images.},
journaltitle = {European Radiology},
shortjournal = {Eur Radiol},
author = {Möller, Hendrik and Graf, Robert and Schmitt, Joachim and Keinert, Benjamin and Schön, Hanna and Atad, Matan and Sekuboyina, Anjany and Streckenbach, Felix and Kofler, Florian and Kroencke, Thomas and Bette, Stefanie and Willich, Stefan N. and Keil, Thomas and Niendorf, Thoralf and Pischon, Tobias and Endemann, Beate and Menze, Bjoern and Rueckert, Daniel and Kirschke, Jan S.},
urldate = {2024-11-14},
date = {2024-10-29},
langid = {english},
keywords = {Deep learning, Intervertebral disc, Magnetic resonance imaging, Spine, Vertebral body},
}
@article{graf2023denoising,
title={Denoising diffusion-based MRI to CT image translation enables automated spinal segmentation},
author={Graf, Robert and Schmitt, Joachim and Schlaeger, Sarah and M{\"o}ller, Hendrik Kristian and Sideri-Lampretsa, Vasiliki and Sekuboyina, Anjany and Krieg, Sandro Manuel and Wiestler, Benedikt and Menze, Bjoern and Rueckert, Daniel and others},
journal={European Radiology Experimental},
volume={7},
number={1},
pages={70},
year={2023},
publisher={Springer}
}
This installation assumes you know your way around conda and virtual environments.
SPINEPS supports Python 3.9 to 3.13. On Windows, Python 3.10 or newer is required: antspyx (pulled in via TPTBox) publishes no Windows wheel for 3.9, so installing it there would mean building it from source.
The order of the following instructions is important!
- Use Conda or Pip to create a venv for python 3.11, we are using conda for this example:
conda create --name spineps python=3.11
conda activate spineps
conda install pip- Go to https://pytorch.org/get-started/locally/ and install a correct pytorch version for your machine in your venv
- Confirm that your pytorch package is working! Try calling these commands:
nvidia-smiThis should show your GPU and it's usage.
python -c "import torch; print(torch.cuda.is_available())"This should throw no errors and return True
Install the package (required even for local use):
cd spineps
pip install -e .Model weights download automatically on first use, so you usually don't need to do anything else. To manage
weights manually instead, download them from the corresponding release page and extract each model folder into a
directory of your choice (default spineps/spineps/models/), structured like:
<models_folder>
├── <model_name 1>
├── inference_config.json
├── <other model-specific files and folders>
├── <model_name 2>
├── inference_config.json
├── <other model-specific files and folders>
...
Point SPINEPS at that directory via the SPINEPS_SEGMENTOR_MODELS environment variable (set it permanently in your .bashrc/.zshrc):
export SPINEPS_SEGMENTOR_MODELS=<PATH-to-your-folder>You can also execute the above line whenever you run this segmentation pipeline.
To check that you set the environment variable correctly, call:
echo ${SPINEPS_SEGMENTOR_MODELS}For Windows, this might help: https://phoenixnap.com/kb/windows-set-environment-variable
If you don't set the environment variable, the pipeline will look into spineps/spineps/models/ by default.
After installation (pip install spineps, or pip install -e . from a local clone), the spineps command is
available in your venv:
- Activate your venv.
- Run
spineps -hfor the subcommands, andspineps sample -h/spineps dataset -hfor their arguments. - For example, to segment a single scan:
spineps sample -i <path-to-nifty> --model-semantic <model_name> --model-instance <model_name>(replacing <model_name> with the model you want to use). You can also call SPINEPS from Python — see
Using the Code.
SPINEPS prints a short citation reminder on first use and at exit. Set SPINEPS_NO_CITATION_REMINDER=1
(or true/yes/on) to silence it.
- import issues: try installing via the requirements again, somethings it doesn't install everything
- pytorch / cuda issues: good luck! :3
The pipeline can process either:
- Single Nifty (.nii.gz) files
- Whole Datasets
spineps sample <args>:
Processes a single nifty file, will create a derivatves folder next to the nifty, and write all outputs into that folder
| argument | explanation |
|---|---|
| --input, -i | Absolute path to the single nifty file (.nii.gz) to be processed (required) |
| --model-semantic, -ms | The model used for the semantic segmentation (required) |
| --model-instance, -mv, -mi | The model used for the vertebra instance segmentation (default: instance) |
| --model-labeling, -ml | The (optional) VERIDAH model used for vertebra labeling (default: t2w_labeling) |
Plus the common processing options below, shared with dataset mode. Run spineps sample -h for the full list
with defaults.
| argument | explanation |
|---|---|
| --derivative-name, -dn | Name of the derivatives folder (default: derivatives_seg) |
| --save-debug, -sd | Saves debug data and intermediate results in a separate folder (default: False) |
| --save-softmax-logits, -ssl | Saves an .npz of the semantic model's raw softmax logits (default: False) |
| --save-modelres-mask, -smrm | Also saves the semantic mask at the model's native resolution (default: False) |
| --override-semantic, -os | Override existing seg-spine files (default: False) |
| --override-instance, -oi | Override existing seg-vert files (default: False) |
| --override-postpair, -opp | Override existing cleaned/paired files (default: False) |
| --override-ctd, -oc | Override existing centroid files (default: False) |
| --ignore-inference-compatibility, -iic | Don't skip inputs whose modality doesn't match the models (default: False) |
| --crop / --no-crop | Crop the input to the spine before semantic segmentation (default: on) |
| --n4 / --no-n4 | N4 bias field correction before semantic segmentation, MRI only (default: on) |
| --enforce-12-thoracic | Force the labeling model to predict exactly 12 thoracic vertebrae (default: False) |
| --batch-size, -bs | Vertebra cutouts per batched forward pass; higher is faster but uses more GPU memory. Only affects GPU memory; host RAM usage in the instance phase scales with scan length/vertebra count instead (default: 4) |
| --amp | Run the instance model's forward pass under CUDA autocast, faster but may slightly change output (default: False) |
| --step-size | Semantic model sliding-window tile step size; larger is faster but less accurate (default: model's own setting) |
| --tta / --no-tta | Force test-time mirroring augmentation on/off for the semantic model (default: model's own setting) |
| --cpu | Run on CPU instead of GPU, much slower (default: False) |
| --run-cprofiler, -rcp | Runs a cProfiler over the entire run (default: False) |
| --verbose, -v | Prints much more stuff, may fully clutter your terminal (default: False) |
There are a lot more arguments, run spineps sample -h to see them.
#T2w sagittal
spineps sample --ignore-inference-compatibility -i /path/sub-testsample_T2w.nii.gz --model-semantic t2w --model-instance instance
#T1w sagittal
spineps sample --ignore-inference-compatibility -i ~/path/sub-testsample_T1w.nii.gz --model-semantic t1w --model-instance instance(--ignore-bids-filter is a dataset-only option — see below — it isn't accepted by sample.)
spineps dataset <args>:
Processes all "suitable" niftys it finds in the specified dataset folder.
A dataset folder must have the following structure:
dataset-folder
├── <rawdata>
├── subfolders (optionally, any number of them)
├── One or multiple target files
├── One or multiple target files
├── <derivatives>
├── The results are saved/loaded here
A target file in a dataset must look like the following:
sub-<subjectid>_*_T2w.nii.gz
where * depicts any number of key-value pairs of characters.
Some examples are:
sub-0001_T2w.nii.gz
sub-awesomedataset_sequ-HWS_part-inphase_T2w.nii.gz
Anything that follows the BIDS-nomenclature is also supported (see https://bids-specification.readthedocs.io/en/stable/)
Meaning you can have some key-value pairs (like sub-<id>) in the name. Those key-value pairs are always separated by _ and combined with - (see second example above). Those will be used in creating the filename of the created segmentations.
To that end, we are using TPTBox (see https://github.com/Hendrik-code/TPTBox)
| argument | explanation |
|---|---|
| --directory, -i, -d | Absolute path to the dataset directory, preferably a BIDS dataset (required) |
| --model-semantic, -ms | The model used for the semantic segmentation (default: t2w) |
| --model-instance, -mv, -mi | The model used for the vertebra instance segmentation (default: instance) |
| --model-labeling, -ml | The (optional) VERIDAH model used for vertebra labeling (default: t2w_labeling) |
| --rawdata-name, -rn | Sets the name of the rawdata folder of the dataset (default: "rawdata") |
| --ignore-bids-filter, -ibf | If true, will search the BIDS dataset without the strict filters. Use with care! (default: False) |
| --ignore-model-compatibility, -imc | If true, will not stop the pipeline to use the given models on unfitting input modalities (default: False) |
| --save-log, -sl | If true, saves the log into a separate folder in the dataset directory (default: False) |
| --save-snaps-folder, -ssf | If true, additionally saves the snapshots in a separate folder in the dataset directory (default: False) |
It also accepts all of the common processing options listed above (--batch-size, --crop,
--n4, --amp, etc.). For a full list of arguments, call spineps dataset -h.
spineps dataset --ignore-bids-filter -i /path/to/dataset-folder --model-semantic t2w --model-instance instanceThe pipeline segments in multiple steps:
- Semantically segments 14 spinal structures (9 regions for vertebrae, Spinal Cord, Spinal Canal, Intervertebral Discs, Endplate, Sacrum)
- From the vertebra regions, segment the different vertebrae as instance mask
- Save the first as
seg-spinemask, the second asseg-vertmask - From the two segmentations, calculates centroids for each vertebrae center point, endplate, and IVD and saves that into a .json
- From the centroid and the segmentations, makes a snapshot showcasing the result as a .png
In the subregion segmentation:
| Label | Structure |
|---|---|
| 41 | Arcus_Vertebrae |
| 42 | Spinosus_Process |
| 43 | Costal_Process_Left |
| 44 | Costal_Process_Right |
| 45 | Superior_Articular_Left |
| 46 | Superior_Articular_Right |
| 47 | Inferior_Articular_Left |
| 48 | Inferior_Articular_Right |
| 49 | Vertebra_Corpus_border |
| 52 | Vertebral_Body_Endplate_Superior |
| 53 | Vertebral_Body_Endplate_Inferior |
| 60 | Spinal_Cord |
| 61 | Spinal_Canal |
| 62 | Endplate (only where the plate could not be assigned to a vertebra) |
| 100 | Vertebra_Disc |
| 26 | Sacrum |
CT only
| Label | Structure |
|---|---|
| 51 | Dens_axis (odontoid process of C2) |
| 70 | Sacrum_Sacral_Ala_Left |
| 71 | Sacrum_Sacral_Ala_Right |
| 72 | Sacrum_Posterior_Sacral_Elements |
| 73 | Sacrum_Body |
| 74 | Sacrum_Endplate |
| 80 | Metal |
In the vertebra instance segmentation mask, each label X in [1, 25] are the unique vertebrae, while 100+X are their corresponding IVD and 200+X their endplates.
To run the vertebra labeling after segmentation, specify a --model-labeling model (similar to --model-semantic and --model-instance).
If you use VERIDAH (labeling model) in addition to the segmentation models from SPINEPS, then a labeling model will run and give each vertebrae detected by SPINEPS a vertebra label. These are
| Label | Structure |
|---|---|
| 1 | C1 |
| 2 - 7 | C2 - C7 |
| 8 - 19 | T1 - T12 |
| 28 | T13 |
| 20 | L1 |
| 21 - 25 | L2 - L6 |
| 26 | Sacrum |
The labels 100+X still correspond to the vertebra's IVD and 200+X the respective endplate. For example, the label 119 is the IVD below the T12 vertebra.
The easiest way to run SPINEPS from Python is the one-call spineps.segment API, which loads the models and runs the whole pipeline:
import spineps
result = spineps.segment("/path/to/sub-test_T2w.nii.gz") # saves a derivatives folder next to the input
result = spineps.segment(nii, output_in_memory=True) # or get the masks back in memoryTo segment many images without reloading the models, use SpinepsPipeline; to group processing options, pass the
SemanticConfig / InstanceConfig / LabelingConfig / PostConfig objects.
For full control, load the models yourself with get_semantic_model() / get_instance_model() and call
segment_image() (single image) or process_dataset() (whole dataset) from spineps.seg_run.
Upgrading from 1.x? See MIGRATION.md for the renamed CLI flags, functions and classes.
This pipeline was created by Hendrik Möller, M.Sc. (he/him)
PhD Researcher at Department for Interventional and Diagnostic Neuroradiology
Developed within an ERC Grant at
University Hospital rechts der Isar at Technical University of Munich
Ismaninger Street 22, 81675 Munich
https://deep-spine.de/
https://aim-lab.io/author/hendrik-moller/
Copyright 2023 Hendrik Möller
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.