Nullify / PromptLeak Scanner Core

This is a cleaned public bundle of the core scanning logic from Nullify.

It includes:

• the scanner engine in scanner/attacker.py
• the analysis and scoring logic in scanner/analyzer.py
• the payload library in scanner/payloads.py
• one anonymized sample report in sample_reports/acmecorp_demo_report.pdf

It intentionally does not include local app wrappers, deployment files, private notes, or any API keys.

What It Does

Nullify fires a fixed corpus of prompt-injection payloads at a target endpoint and then classifies the responses for:

• direct prompt leakage
• role confusion or identity takeover
• instruction override behavior

The scanner works best against HTTP or HTTPS chat-style endpoints, especially OpenAI-compatible /chat/completions targets.

Repo Layout

nullify_github/
  README.md
  requirements.txt
  .gitignore
  scanner/
    attacker.py
    analyzer.py
    payloads.py
  sample_reports/
    acmecorp_demo_report.pdf

Quick Start

python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt

Minimal Usage

import asyncio

from scanner.attacker import run_scan
from scanner.analyzer import analyze


async def main():
    scan_result = await run_scan("https://example.com/v1/chat/completions")
    report = await analyze(scan_result)

    print("Risk:", report.overall_risk.value)
    print("Findings:", report.total_findings)

    for finding in report.findings:
        print(f"{finding.payload_id}: {finding.description}")
        print(f"Evidence: {finding.evidence}")


asyncio.run(main())

Environment Variables

The scanner supports these useful environment variables:

• OPENAI_COMPAT_API_KEY
• OPENAI_COMPAT_MODEL
• OPENAI_COMPAT_MAX_TOKENS
• SCAN_AUTH_BEARER_TOKEN
• SCAN_TIMEOUT_S
• SCAN_CONCURRENT_LIMIT
• SCAN_REQUEST_DELAY_S
• ALLOW_LOCAL_TARGETS=1 for local lab testing

The analyzer supports:

• NVIDIA_API_KEY for the NVIDIA-hosted judge path
• ANTHROPIC_API_KEY for the Claude judge path

Do not commit your .env file. This repo includes a .gitignore entry for it on purpose.

Sample Report

sample_reports/acmecorp_demo_report.pdf is an anonymized demo report using fake AcmeCorp target details. It is included only as an example artifact for formatting and presentation.

Real Finding Example

License

This project is released under the MIT License.

That means other people can use, modify, and share it, but your copyright notice remains attached to the project in the license file.

Notes

• The payload library is intentionally fixed so results remain comparable across targets.
• Public endpoints may rate-limit or partially fail, so incomplete coverage should not be treated as a clean pass.
• Visible reasoning traces can create false positives if they are not sanitized before analysis.

Built by @pulkitbuilds and Pulkit Srivastava -- follow for findings and updates.