This software project accompanies the research paper, Sparse Autoencoders are Capable LLM Jailbreak Mitigators.
This repository contains the code for CC-Delta, a sparse autoencoder (SAE) based inference-time defense against jailbreak attacks on language models.
ccdelta/— main package: hooks, models, evaluations, scripts, configsscripts/— standalone scripts for data prep and SAE feature descriptionnotebooks/— analysis notebooks for paper results
Install uv to manage dependencies, then:
git clone <repo-url>
cd ml-ccdelta
uv syncDependency versions are pinned to those used in the paper for reproducibility; loosen them in pyproject.toml if exact reproduction isn't needed.
To also install the optional SAE feature interpretation dependency (for scripts/interpret_sae_features.py):
uv sync --extra interpretData must be generated locally before running experiments.
Downloads AdvBench and StrongREJECT, creates train/test splits:
uv run python -m ccdelta.scripts.prepare_dataset_splits --dataset all -vOutput: data/forbidden_prompts/0.5/
Applies jailbreak attacks to the base prompts. Some attack methods (PAP variants) use the OpenAI API:
export OPENAI_API_KEY=<your-key>
uv run python -m ccdelta.scripts.apply_strongrejectOutput: data/jailbroken/0.5/
Wraps jailbroken prompts in defense templates. Only needed for prompt-defense evaluation:
uv run python -m ccdelta.scripts.apply_prompt_defenseOutput: data/prompt_defense/0.5/
Downloads OR-Bench from HuggingFace for over-refusal evaluation:
uv run python scripts/prepare_or_bench.py prepare
uv run python scripts/prepare_or_bench.py partition \
--existing-sample data/or_bench/or_bench_80k_10pct_seed42.parquetTo also run prompt defense on OR-Bench:
uv run python -m ccdelta.scripts.apply_prompt_defense_orbenchOutput: data/or_bench/
Run all experiments (baseline, CC-Delta, CAA) with a single command:
uv run python -m ccdelta.scripts.run_experimentsBy default this runs in fast mode (small subset of prompts) for quick validation. To run on the full dataset:
uv run python -m ccdelta.scripts.run_experiments --fullRun specific stages:
# Only baseline and CC-Delta
uv run python -m ccdelta.scripts.run_experiments --stages baseline ccdelta
# Only CAA
uv run python -m ccdelta.scripts.run_experiments --stages caaOverride the model:
uv run python -m ccdelta.scripts.run_experiments --model gemma-2-9b-itResults are written to ./results/ by default. See notebooks/results_analysis.ipynb to analyze results.
Running on Mac (MPS) or CPU:
The default device is cuda. On a Mac or machine without a GPU, pass a device override (MPS or CPU):
uv run python -m ccdelta.scripts.run_experiments device=cpu --stages ccdeltaThis repository also includes implementations of the following baselines:
- CAA (Contrastive Activation Addition) — dense contrastive steering vectors
- LinearAcT — dense optimal transport activation steering (ml-act)
uv run python -m pytest ccdelta/tests/ -x -m "not integration"