Repository navigation
Security Deep #324
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Security Deep | |
| on: | |
| schedule: | |
| - cron: '0 20 * * *' # 20:00 UTC = 7 AM AEDT next day | |
| workflow_dispatch: # Allow manual triggers | |
| permissions: | |
| contents: read | |
| env: | |
| CARGO_TERM_COLOR: always | |
| RUST_BACKTRACE: 1 | |
| jobs: | |
| # Deep security analysis - nightly scheduled | |
| kani-verification: | |
| name: Kani Formal Verification | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| - name: Install Kani | |
| run: | | |
| cargo +stable install --locked kani-verifier --version 0.67.0 | |
| cargo-kani setup | |
| - name: Run Kani verification | |
| run: | | |
| cd rust | |
| cargo-kani --tests --no-default-features --features compression,checksum,messagepack,encryption | |
| fuzzing: | |
| name: Extended Fuzzing (3 targets × 1h) | |
| runs-on: ubuntu-latest | |
| # Budget: ~2 min rustup + ~10 min cargo-fuzz install + ~10 min ASAN builds | |
| # + 3 x 60 min fuzz time (-max_total_time is pure fuzz time, excluding | |
| # builds) + slack. Must stay under the 360 min GitHub-hosted hard cap. | |
| # No rust-cache here on purpose: a nightly job's 10 min install is noise | |
| # against 180 min of fuzz time, and it would be one more cache competing | |
| # for the repo's 10 GB budget (fuzz-smoke.yml caches the same toolchain | |
| # where PR latency matters). | |
| timeout-minutes: 240 | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| - name: Install nightly Rust | |
| run: | | |
| # Pin nightly: cargo-fuzz's rustix dependency uses | |
| # rustc_layout_scalar_valid_range_* attributes reserved after | |
| # nightly-2026-04-27. Last known-good date. Same pin as fuzz-smoke.yml. | |
| rustup toolchain install nightly-2026-04-27 | |
| rustup default nightly-2026-04-27 | |
| - name: Install cargo-fuzz | |
| # Version-pinned: a floating `cargo install` hands a hijacked cargo-fuzz | |
| # release immediate code exec on the CI runner. Same pattern as | |
| # kani-verifier above and fuzz-smoke.yml. | |
| run: cargo install --locked cargo-fuzz --version 0.13.2 | |
| - name: Verify fuzz lockfile is current | |
| run: | | |
| cd rust/fuzz | |
| # See fuzz-smoke.yml: cargo fuzz has no --locked passthrough, so this is | |
| # the only step enforcing the committed lock against manifest drift. | |
| cargo +nightly-2026-04-27 fetch --locked | |
| - name: Build fuzz targets | |
| run: | | |
| cd rust/fuzz | |
| # Build before fuzzing so a compile error surfaces in ~10 min as a build | |
| # failure, not three hours in as a confusing fuzz-step error. | |
| cargo +nightly-2026-04-27 fuzz build | |
| # The fuzz crate (cachekit-storage-fuzz) declares no [features] table — its | |
| # cachekit-core feature set is fixed in rust/fuzz/Cargo.toml. Passing | |
| # --features here makes cargo error out before any fuzzing happens, and the | |
| # old `|| true` swallowed exactly that for three core versions. A non-zero | |
| # exit (build failure or crash) must fail the job; a clean hour exits 0 via | |
| # -max_total_time. | |
| # | |
| # Deliberate subset for the time budget: these three cover the highest-value | |
| # attack surfaces (compression bombs, envelope decode, key derivation); every | |
| # target gets 60 s on every PR via fuzz-smoke.yml. | |
| - name: Fuzz byte_storage_compress (1 hour) | |
| run: | | |
| cd rust | |
| cargo fuzz run byte_storage_compress -- -max_total_time=3600 | |
| - name: Fuzz byte_storage_decompress (1 hour) | |
| run: | | |
| cd rust | |
| cargo fuzz run byte_storage_decompress -- -max_total_time=3600 | |
| - name: Fuzz encryption_key_derivation (1 hour) | |
| run: | | |
| cd rust | |
| cargo fuzz run encryption_key_derivation -- -max_total_time=3600 | |
| # No "Check for crashes" step: now that `|| true` is gone, a crash fails its | |
| # own fuzz step and kills the job, so a trailing check could only ever run in | |
| # the no-crash case and print ✅ — a named green step incapable of failing, | |
| # which is the exact manufactured-evidence pattern this PR removes. (It also | |
| # ran `find artifacts` without creating the dir, swallowing the error.) The | |
| # fuzz steps' exit codes are the signal; this upload preserves the evidence, | |
| # which would otherwise die with the ephemeral runner. | |
| - name: Upload crash artifacts | |
| if: failure() | |
| uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 | |
| with: | |
| name: extended-fuzz-crash-artifacts | |
| path: rust/fuzz/artifacts/ | |
| retention-days: 30 | |
| if-no-files-found: warn | |
| atheris-fuzzing: | |
| name: Atheris Python-Rust Fuzzing | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 60 | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| # No uv or Rust cache on this nightly job: the cold `uv sync` (maturin | |
| # build included) is noise against 30 min of fuzz time, and an unsuffixed | |
| # uv cache would just restore the cp312 wheel set the PR jobs save. | |
| - uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0 | |
| with: | |
| version: "0.12.12" # pinned: uv runs as step 1 of every job; a floating release is code exec | |
| enable-cache: false | |
| # Pin to 3.11: atheris 2.3.0's newest prebuilt wheel is cp311. On a newer | |
| # interpreter uv builds atheris from sdist, which needs clang plus a | |
| # matching libFuzzer archive (find_libfuzzer.sh) that runner images do not | |
| # reliably ship, so install fails before any fuzzing runs. Both uv sync and | |
| # uv run must pin, or uv run re-resolves to a different interpreter and | |
| # re-triggers the source build. | |
| - name: Install dependencies | |
| run: | | |
| uv sync --group dev --group fuzz --python 3.11 | |
| - name: Run Atheris fuzz targets (10 min each) | |
| run: | | |
| for fuzz_target in tests/fuzzing/fuzz_*.py; do | |
| if [ -f "$fuzz_target" ]; then | |
| echo "Fuzzing $fuzz_target..." | |
| timeout 10m uv run --python 3.11 python "$fuzz_target" -max_total_time=600 || true | |
| fi | |
| done | |
| - name: Upload crash corpus | |
| if: always() | |
| uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 | |
| with: | |
| name: atheris-crashes-${{ github.run_id }} | |
| path: tests/fuzzing/corpus/ | |
| - name: Report fuzzing results | |
| if: always() | |
| run: | | |
| if [ -d tests/fuzzing/corpus/ ] && find tests/fuzzing/corpus/ -mindepth 1 ! -name '.gitignore' -print -quit | grep -q .; then | |
| echo "⚠️ Crashes discovered during Atheris fuzzing!" | |
| ls -lh tests/fuzzing/corpus/ | |
| exit 1 | |
| fi | |
| echo "✅ No crashes discovered during Atheris fuzzing" | |
| miri-full: | |
| name: Miri Full Suite | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 30 | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| - name: Install nightly Rust with Miri | |
| run: | | |
| rustup toolchain install nightly --component miri | |
| rustup default nightly | |
| - name: Run full Miri test suite | |
| run: | | |
| cd rust | |
| cargo miri test --no-default-features --features compression,checksum,messagepack,encryption | |
| sanitizers: | |
| name: Sanitizers (ASan, TSan, MSan) | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 40 | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| sanitizer: [address, thread, memory] | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| - name: Install nightly Rust | |
| run: | | |
| rustup toolchain install nightly | |
| rustup default nightly | |
| - name: Add source for sanitizers | |
| if: matrix.sanitizer == 'memory' || matrix.sanitizer == 'thread' | |
| run: rustup component add rust-src --toolchain nightly | |
| - name: Run AddressSanitizer | |
| if: matrix.sanitizer == 'address' | |
| env: | |
| RUSTFLAGS: -Zsanitizer=address | |
| run: | | |
| cd rust | |
| cargo +nightly test --lib --target x86_64-unknown-linux-gnu \ | |
| --no-default-features --features compression,checksum,messagepack,encryption | |
| - name: Run ThreadSanitizer | |
| if: matrix.sanitizer == 'thread' | |
| env: | |
| RUSTFLAGS: -Zsanitizer=thread | |
| TSAN_OPTIONS: suppressions=${{ github.workspace }}/rust/tsan_suppressions.txt | |
| run: | | |
| cd rust | |
| cargo +nightly test --lib --target x86_64-unknown-linux-gnu \ | |
| -Zbuild-std --no-default-features --features compression,checksum,messagepack,encryption | |
| - name: Run MemorySanitizer | |
| if: matrix.sanitizer == 'memory' | |
| env: | |
| RUSTFLAGS: -Zsanitizer=memory -Zsanitizer-memory-track-origins | |
| run: | | |
| cd rust | |
| cargo +nightly test --lib --target x86_64-unknown-linux-gnu \ | |
| -Zbuild-std --no-default-features --features compression,checksum,messagepack,encryption | |
| generate-security-report: | |
| name: Generate Security Report | |
| runs-on: ubuntu-latest | |
| needs: [kani-verification, fuzzing, atheris-fuzzing, miri-full, sanitizers] | |
| if: always() | |
| steps: | |
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | |
| - name: Generate security report | |
| run: | | |
| mkdir -p reports/security | |
| TIMESTAMP=$(date +%Y%m%d-%H%M%S) | |
| REPORT="reports/security/deep-analysis-$TIMESTAMP.md" | |
| cat > "$REPORT" <<EOF | |
| # Deep Security Analysis Report | |
| Generated: $(date -u +"%Y-%m-%d %H:%M:%S UTC") | |
| Workflow Run: ${{ github.run_id }} | |
| ## Summary | |
| - **Kani Verification**: ${{ needs.kani-verification.result }} | |
| - **Extended Fuzzing**: ${{ needs.fuzzing.result }} | |
| - **Atheris Python-Rust Fuzzing**: ${{ needs.atheris-fuzzing.result }} | |
| - **Miri Full Suite**: ${{ needs.miri-full.result }} | |
| - **Sanitizers**: ${{ needs.sanitizers.result }} | |
| ## Results | |
| ### Formal Verification (Kani) | |
| Status: ${{ needs.kani-verification.result }} | |
| - ByteStorage roundtrip properties | |
| - Checksum integrity verification | |
| - Decompression bomb protection | |
| - Encryption roundtrip correctness | |
| - Key derivation determinism | |
| ### Fuzzing (3 hours total) | |
| Status: ${{ needs.fuzzing.result }} | |
| - byte_storage_compress (1 hour) | |
| - byte_storage_decompress (1 hour) | |
| - encryption_key_derivation (1 hour) | |
| ### Undefined Behavior Detection (Miri) | |
| Status: ${{ needs.miri-full.result }} | |
| - Full test suite with strict provenance | |
| - Symbolic alignment checking | |
| - Memory leak detection | |
| ### Runtime Sanitizers | |
| Status: ${{ needs.sanitizers.result }} | |
| - AddressSanitizer (memory errors) | |
| - ThreadSanitizer (race conditions) | |
| - MemorySanitizer (uninitialized memory) | |
| ## Conclusion | |
| $(if [[ "${{ needs.kani-verification.result }}" == "success" ]] && \ | |
| [[ "${{ needs.fuzzing.result }}" == "success" ]] && \ | |
| [[ "${{ needs.miri-full.result }}" == "success" ]] && \ | |
| [[ "${{ needs.sanitizers.result }}" == "success" ]]; then | |
| echo "✅ All deep security checks passed" | |
| else | |
| echo "❌ One or more deep security checks failed" | |
| fi) | |
| EOF | |
| cat "$REPORT" | |
| - name: Archive security report | |
| uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 | |
| with: | |
| name: security-deep-report | |
| path: reports/security/ | |
| retention-days: 90 | |
| # Summary job | |
| security-deep-success: | |
| name: Security Deep Success | |
| runs-on: ubuntu-latest | |
| needs: [kani-verification, fuzzing, atheris-fuzzing, miri-full, sanitizers] | |
| if: always() | |
| permissions: | |
| contents: read | |
| issues: write | |
| steps: | |
| - name: Check all deep security checks passed | |
| run: | | |
| if [[ "${{ needs.kani-verification.result }}" != "success" ]] || \ | |
| [[ "${{ needs.fuzzing.result }}" != "success" ]] || \ | |
| [[ "${{ needs.atheris-fuzzing.result }}" != "success" ]] || \ | |
| [[ "${{ needs.miri-full.result }}" != "success" ]] || \ | |
| [[ "${{ needs.sanitizers.result }}" != "success" ]]; then | |
| echo "❌ One or more deep security checks failed" | |
| echo "Kani: ${{ needs.kani-verification.result }}" | |
| echo "Fuzzing: ${{ needs.fuzzing.result }}" | |
| echo "Atheris: ${{ needs.atheris-fuzzing.result }}" | |
| echo "Miri: ${{ needs.miri-full.result }}" | |
| echo "Sanitizers: ${{ needs.sanitizers.result }}" | |
| exit 1 | |
| fi | |
| echo "✅ All deep security checks passed" | |
| # Surface nightly failures: a schedule-only job that fails silently is worse | |
| # than no job. De-duplicate by title so we don't open a new issue every night. | |
| - name: File / update tracking issue on failure | |
| if: failure() | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| TITLE: "Security Deep nightly is failing" | |
| run: | | |
| body=$(printf '%s\n' \ | |
| "Security Deep failed on run ${{ github.run_id }}." \ | |
| "" \ | |
| "- Kani: ${{ needs.kani-verification.result }}" \ | |
| "- Fuzzing: ${{ needs.fuzzing.result }}" \ | |
| "- Atheris: ${{ needs.atheris-fuzzing.result }}" \ | |
| "- Miri: ${{ needs.miri-full.result }}" \ | |
| "- Sanitizers: ${{ needs.sanitizers.result }}" \ | |
| "" \ | |
| "Run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}") | |
| existing=$(gh issue list --repo "${{ github.repository }}" --state open \ | |
| --search "$TITLE in:title" --json number --jq '.[0].number // empty') | |
| if [ -n "$existing" ]; then | |
| gh issue comment "$existing" --repo "${{ github.repository }}" --body "$body" | |
| else | |
| gh issue create --repo "${{ github.repository }}" --title "$TITLE" --label bug --body "$body" | |
| fi |