The world's first observability bridge between zero-knowledge blockchain infrastructure and Splunk — now with zkZap, a privacy-native security layer.
You can't watch what you can't see. Unless you're us.
Built for the Splunk Agentic Ops Hackathon 2026 by EnterpriseZK Labs LLC
📌 About this project. This repository was built for the Splunk Agentic Ops Hackathon 2026 (organized by Splunk, sponsored by Cisco) — Observability track. ZKSplunk is the first Splunk observability connector for zero-knowledge blockchain infrastructure, built on the Midnight Network. For a guided tour, watch the repo & architecture walkthrough video. New to the docs? Start at
docs/, which lays out a numbered reading order.
"Privacy is a feature. Observability is a superpower. zkZap turns both into a defense."
Midnight is a privacy-preserving blockchain where computation happens inside zero-knowledge proofs — by design, you can't see what's happening inside. That's the whole point.
But operators still need to know: Is the proof server alive? Why did that proof take 47 seconds? Is the wallet connected? Did a contract just go dark — or is someone attacking it?
ZKSplunk is a purpose-built telemetry pipeline that understands ZK-proof lifecycles, shielded-state semantics, and privacy-preserving contracts — and streams that understanding into Splunk, where AI agents turn raw blockchain telemetry into autonomous incident response.
zkZap is the security layer on top: it re-reads that same telemetry as threat signals (proof floods, brute-force bursts, mint anomalies, wallet drains). Today it provides detection surfaces and an evidence-backed analyst loop; the planned response path is observe → decide → act, with critical incidents anchored on-chain as anonymous, unlinkable attestations: only the anonymized incident class is public (so the whole network gains awareness) — never the operator or node.
No one has built this before. Splunk has connectors for Ethereum, Hyperledger, and Quorum. ZKSplunk is the first connector for any ZK-proof blockchain infrastructure on Earth — and zkZap is the first privacy-native SOC pattern for one.
ZKSplunk is a working local Splunk Enterprise observability app plus connector/agent code, with several forward-looking pieces clearly separated below.
| Area | Status | Notes |
|---|---|---|
| Live Midnight component monitoring | Running / implemented | zkMonitor / realDeal probes proof-server, indexer, node, and wallet-facing public metadata, then forwards events to Splunk HEC. |
| Splunk app dashboards | Running / implemented | splunk-app/zksplunk ships the zksplunk index, saved searches, alerts, component detail view, operator map, AI tabs, attestation view, and packaged .spl. The Global Map includes the KPI strip below the map: critical components, proof/indexer p95, HEC failures, contract state, and on-chain attestation count. |
| Splunk-native AI Toolkit analyst tab | Running / implemented | splunk-app/zksplunk ships ZKSplunk AI Toolkit Analyst, a Splunk tab that aggregates index=zksplunk evidence and calls Splunk AI Toolkit ` |
| Local AI analyst chat | Running / implemented | ai-agent queries Splunk via MCP when configured, falls back to REST, and phrases evidence-backed answers through Splunk AI Toolkit ` |
| Splunk MCP integration | Implemented for Splunk evidence | The local analyst can use Splunk MCP or Splunk REST to gather evidence. |
| Dual MCP bridge: Splunk MCP + Midnight MCP | Future / planned | The cross-platform bridge described below is the target architecture: Splunk evidence plus Midnight MCP contract/docs/tooling context in one investigation loop. |
| On-chain attestation | Demo runtime implemented / unaudited | contract/src/zksplunk.compact, zkMonitor deploy tooling, the attestation relayer, and the read-only on-chain status reader are wired for the demo pipeline. The contract is not audited, so this is not treated as a production attestation system. Setup: docs/09_SETUP_BLOCKCHAIN_PIPELINE.md. |
| zkZap automated response / SOAR | Future / planned | Detection concepts and alert surfaces exist; automated SOAR/on-chain response remains future work. |
- Repo & architecture walkthrough: what ZKSplunk is and how it's built
- Blockchain attestation demo: critical event -> relayer -> Midnight contract -> Splunk
Three implemented demo flows run through ZKSplunk today: a telemetry ingestion path (green) that streams Midnight infrastructure health into Splunk, an AI analyst path (orange) where operators ask questions inside the Splunk app, and an on-chain status/attestation path (purple) that surfaces the deployed Midnight contract, operator count, attestation count, relayer health, and public incident classes in Splunk. The primary analyst surface is the ZKSplunk AI Toolkit Analyst tab, which runs SPL over index=zksplunk and invokes Splunk AI Toolkit with | ai prompt="{prompt}" provider=Gemini model=gemini-2.5-flash. The local ai-agent chat remains available for the MCP-backed flow and also prefers Splunk AI Toolkit for phrasing. The dual Splunk MCP + Midnight MCP bridge is shown as the planned extension point, not as a completed production bridge.
The full hackathon-required architecture diagram lives at architecture_diagram.md. It shows the live Midnight telemetry path into Splunk HEC, the Splunk app surfaces, and the runtime AI analyst path through Splunk MCP Server.
graph TD
A1[Midnight proof server] --> B1[zkMonitor probes]
A2[Midnight indexer] --> B1
A3[Midnight node] --> B1
A4[Wallet public boundary] --> B1
B1 --> C1[SplunkForwarder]
C1 --> C2[HEC client]
C2 --> D1[Splunk HEC]
D1 --> D2[index zksplunk]
D2 --> D3[Dashboards and alerts]
D2 --> D4[Splunk MCP Server]
D2 --> D5[Splunk REST fallback]
D2 --> D6[Splunk AI Toolkit]
B1 --> G1[Attestation trigger on critical vitals]
G1 --> G2[ZK proof client]
G2 --> G3[Attestation relayer]
G3 --> G4[Midnight zksplunk compact contract]
G4 --> G5[Read-only on-chain status reader]
G5 --> D1
F1[Operator] --> E0[Splunk AI Toolkit Analyst tab]
E0 --> E6[SPL evidence aggregation]
E6 --> D2
E6 --> D6
D6 --> F2[Evidence backed answer]
F1 --> E1[Local AI analyst UI fallback]
E1 --> E2[zkZap analyst]
E2 --> E3[Predefined SPL evidence library]
E3 --> D4
D4 --> E3
E3 -. fallback .-> D5
E2 --> E4[Splunk AI Toolkit phrasing]
E4 --> D6
D6 --> E4
E2 -. external fallback only .-> E5[OpenAI compatible LLM]
E2 --> F2
| Layer | Package | Role |
|---|---|---|
| Collector | zkMonitor/ |
probes live Midnight infra, emits VitalCheckResults |
| Forwarder | connector/ |
adapts + ships events to Splunk HEC (+ optional on-chain attestation) |
| Shared types | vitals/ |
VitalCheckResult and event schemas |
| Storage + UI | splunk-app/zksplunk/ |
index=zksplunk, dashboards, alerts, saved searches |
| Splunk-native analyst | splunk-app/zksplunk/default/data/ui/views/zksplunk_ai_toolkit_analyst.xml |
Splunk tab that asks questions over index=zksplunk and calls Splunk AI Toolkit ` |
| AI agent | ai-agent/ |
zkZap analyst chat — queries Splunk via MCP/REST and prefers Splunk AI Toolkit for answer phrasing |
| On-chain contract | contract/ |
Compact contract for anonymous, unlinkable critical-incident attestation; demo-wired, not audited yet |
| On-chain runtime | zkMonitor/src/deploy-attestation.ts, zkMonitor/src/attestation-relayer.ts, zkMonitor/src/onchain-status-reader.ts |
deploy/register, relay critical-incident proofs, and emit zksplunk:onchain status/incidents back into Splunk |
| Splunk attestation UI | splunk-app/zksplunk/default/data/ui/views/zksplunk_attestation.xml |
contract deployment state, operators, on-chain attestation count, relayer liveness, public incident classes |
A simulated, offline twin of the collector lives in
demoLand/— sameconnector/vitalscode, mock source and local sink. Seedocs/06_DEMOLAND_VS_ZKMONITOR.md.
ZKSplunk observes public infrastructure metadata and forwards it to Splunk; zkZap responds by reading those Splunk signals as operational and security evidence. The diagrammed version of this flow is included in architecture_diagram.md.
graph LR
A[MidnightVitals] --> B[ZKSplunk connector]
B --> C[Splunk HEC]
C --> D[Dashboards and SPL]
D --> E[zkZap analyst]
E --> F[Operator action]
B --> G[Telemetry commitment]
G -. optional .-> H[zksplunk compact]
E -. critical incident only .-> H
H --> I[Anonymous public incident class]
Two lenses over one pipeline:
| ZKSplunk Me | ZKSplunk Macro | |
|---|---|---|
| Who | An individual DApp / operator | Ecosystem watchers (DevRel / elected) |
| Sees | Its own stack (consented self-monitoring) | Public chain Effects + infra health |
| Catches | Local brute-force, wallet drain, proof abuse | Systemic floods, mint storms, outages |
| Privacy | Never reads private state | Built from public data — nobody shares secrets |
Current demo flow: operators can emit anonymous, unlinkable critical-incident attestations through the collector → relayer → Midnight contract path. Only the anonymized incident class, severity, epoch, payload commitment, and nullifier are public; the operator and node are not published. The read-only on-chain status reader sends zksplunk:onchain events back to Splunk so the Overview, Global Map, and zkZap Attestation dashboards can show deployment state, operator count, and attestation count. The contract remains unaudited and the Macro aggregation view remains future work. (See docs/04_ZKZAP_SECURITY_PROTOCOL.md §3.3 and docs/09_SETUP_BLOCKCHAIN_PIPELINE.md.)
Zero-knowledge blockchains have a fundamental operational paradox:
| Challenge | Why it's unique to ZK |
|---|---|
| Proofs take 17–28 s | Traditional chains confirm in milliseconds; ZK proofs are expensive and can fail silently mid-generation. |
| Proof servers are fragile | Docker containers running Halo 2 / UltraPlonk circuits that OOM or crash with zero external signal. |
| Private state is invisible by design | You cannot inspect a shielded contract's state — that's the guarantee, and the debugging nightmare. |
| Attacks hide in the metadata | You can't see what an attacker tried — but failed/rejected calls, mint spikes, and unshielded drains are public. zkZap watches those. |
Traditional observability sees HTTP 200s and container uptime. It has zero understanding of proof lifecycles, circuit pipelines, or shielded-state transitions. ZKSplunk translates that entire domain into Splunk's language.
Midnight is a privacy chain, so the governing rule is simple: metadata and volumes are public, contents are private. That public surface is enough to fight on-chain attacks and outages without ever touching private state. Full reference: docs/03_PUBLIC_LEDGER_OBSERVABILITY.md.
Three public layers we read:
| Layer | Public signal | Source |
|---|---|---|
Per-call Effects |
which contract + circuit fired & how often; mint amounts; unshielded (addr + amount); shielded spend activity | midnight-ledger spec |
| Transaction status | failure (rejected by ledger rules, lands in a block) and rejected (never included) |
tx lifecycle |
| Indexer + block data | block height/cadence/author, contract state transitions, epoch timing, DUST generation health | indexer GraphQL |
What stays invisible by design: private state plaintext, circuit arguments (the witness), and the parties/amounts of shielded transfers. We never claim to detect stealthy attacks on private state — that's impossible on a privacy chain, and saying otherwise would be dishonest.
Public signal → detection map:
| Threat / outage | Public evidence | Field |
|---|---|---|
| Failed-auth / brute force | failed/rejected calls to one entry point | claimed_contract_calls + tx status |
| Contract griefing / spam | abnormal call-rate, rising failure ratio | claimed_contract_calls |
| Mint anomaly | mint-rate / amount spike | shielded_mints / unshielded_mints |
| Wallet drain (unshielded) | rapid drawdown (addr + amount) | claimed_unshielded_spends |
| Wallet drain (shielded) | nullifier burst (amounts hidden) | claimed_nullifiers |
| Indexer / network outage | block height stalls, sync lag, reconnect storm | Block.height / timestamp |
| Consensus / liveness anomaly | block cadence + epoch drift | Block, EpochInfo |
| Fee-resource starvation | DUST generation stalls vs NIGHT registered | DustGenerationStatus |
| Proof-server flood (DDoS) | latency spike + queue depth (operator-side) | MidnightVitals local |
Rules confirmed May 13, 2026 · Sponsor: Cisco · Submission deadline June 15, 2026 · Team cap: 2
| Track | Prize | Our Edge |
|---|---|---|
| 🏆 Grand Prize | $7,000 + .conf26 | An entirely new observability domain (ZK-proof infrastructure) + production-grade code + a privacy-native security layer (zkZap). |
| 🥇 Best of Observability | $3,000 + .conf26 | Home turf. First Splunk connector for ZK infra — purpose-built field extraction, saved searches, dashboards, alerts, and a demo-wired, unaudited tamper-evident attestation layer. |
| Best Use of Splunk MCP Server | $1,000 | Current: Splunk MCP-backed analyst over ZKSplunk evidence. Future: Splunk MCP + Midnight MCP bridge for cross-platform diagnostics neither server could do alone. |
| Most Valuable Feedback | $200 ×5 | Thoughtful, actionable feedback during the feedback period. |
Allowed stack: one Grand + one Bonus. We aim Grand-first, with Observability as the safe floor.
| Metric | Tells you | Why it matters |
|---|---|---|
proof.server.status |
alive / degraded / dead | Is the prover container running? |
proof.server.latency_ms |
response time to :6300 |
Latency spikes predict OOM crashes |
proof.generation.duration_s |
last proof time | Normal 17–28 s; >45 s = trouble |
proof.generation.success |
did it verify? | Silent failures are the #1 ZK hazard |
| Metric | Tells you |
|---|---|
wallet.connected / wallet.balance_dust |
Can the user transact / pay fees? |
network.indexer.status / network.sync_lag_s |
Is the indexer healthy and current? |
contract.readable / contract.last_interaction |
Is the deployed contract responsive? |
| Signal | Observable evidence |
|---|---|
| proof-flood | sustained proof-server latency blow-up |
| failed-auth-bruteforce | burst of failed/rejected calls to one entry point |
| wallet-drain | rapid claimed_unshielded_spends (addr + amount public) |
| mint-anomaly | shielded_mints / unshielded_mints rate spike |
| indexer-outage | health-check failures + growing sync lag |
Status: demo runtime implemented, not audited yet. The contract, deploy/register script, relayer, and on-chain status reader define the tamper-evident anchoring path used by the Splunk dashboards, but this is not yet a production-audited attestation system.
Design intent: telemetry lives off-chain (Splunk); only anonymized incident classes and commitments go on-chain — never operator identity. Operators are registered as commitments in a Merkle tree and prove membership in zero knowledge; a one-time scoped nullifier makes critical reports unlinkable and replay-proof. Built on the sealed-ledger + persistentHash-derived key patterns (pragma language_version >= 0.23).
enum Severity { info, warning, degraded, critical, outage }
enum IncidentClass { proofServerOutage, authBruteforceBurst, mintAnomaly, blockStall, walletDrain }
struct IncidentRecord { incidentClass, severity, epoch, payloadCommitment, nullifier }
sealed ledger networkId, adminPublicKeyHash, observabilitySchemaVersion
ledger operators: HistoricMerkleTree<16, Bytes<32>> // anonymous operator commitments
ledger spentNullifiers: Set<Bytes<32>> // unlinkability + anti-replay
ledger attestationCount: Counter
ledger incidentLog: Map<Field, IncidentRecord> // public append-only log
| Circuit | Role |
|---|---|
selfRegisterAsOperator() |
Admin (deployer) registers itself as operator 0 (the constructor also seeds it) |
registerOperator(operatorCommitment) |
Admin inserts an external operator's pre-computed commitment leaf |
attestCriticalIncident(incidentClass, severity, epoch, scopeTag, payloadCommitment) |
A registered operator anonymously anchors a CRITICAL incident: proves Merkle membership in ZK, spends a one-time scoped nullifier, appends an anonymized record |
getAttestationCount() / isNullifierSpent(nul) |
Read-only views over the public log and spent-nullifier set |
Witnesses (witnesses.ts): localSecretKey() and operatorPath(leaf) supply the operator secret and Merkle authentication path — both stay on the prover's machine. The payloadCommitment lets an auditor independently re-hash the off-chain Splunk detail and verify it matches what was attested. The contract is demo-wired and not audited yet.
ZKSplunk already uses a local AI analyst over Splunk evidence. The next step is the dual-MCP bridge: two Model Context Protocol servers feeding one bidirectional AI intelligence layer.
The implemented loop uses Splunk MCP or Splunk REST evidence today. The planned extension adds Midnight MCP context to the same investigation loop.
graph TD
A[Splunk MCP Server] --> C[Grounded AI diagnosis]
B[Midnight MCP Server] --> C
D[midnight manual corpus] --> C
C --> E[Evidence backed answer]
E --> F[Alert SOAR or on chain report]
C --> A
C --> B
Current loop: detect in Splunk → investigate via Splunk MCP/REST evidence → respond with operator guidance. Planned bridge: add Midnight MCP + the mnm corpus for contract/docs/tooling context, then correlate and respond through alert / SOAR / on-chain attestCriticalIncident. Neither MCP can do this alone — the bridge is the innovation, and it is future work.
ZKSplunk follows the DIDzMonolith demoLand / zkMonitor convention. Both sides share the same connector / vitals / contract code — only the source and sink change. Full architecture: docs/06_DEMOLAND_VS_ZKMONITOR.md.
# demoLand — simulated, offline, safe to record. Runs the zkZap attack scenarios.
cd demoLand && npm install && npm run demo:dashboard
# → console events + out/events.jsonl + a tabbed metrics dashboard (out/dashboard.html)
# zkMonitor — live vitals → real Splunk HEC (+ optional experimental attestation)
cd zkMonitor && cp .env.zkmonitor .env # set SPLUNK_HEC_TOKEN, then:
npm install && npm run start| demoLand | zkMonitor | |
|---|---|---|
| Vitals source | MockVitalsProvider |
live HTTP checks |
| Splunk sink | console + out/events.jsonl |
real Splunk Cloud HEC |
| Attestation | MockAttestationClient |
experimental; contract exists, not audited yet |
| Infra needed | none | Midnight Docker + Splunk Cloud |
npm run demo:dashboard emits a self-contained out/dashboard.html (no CDNs — inline SVG) with a tab per metric — safe to show on a recorded demo:
| Tab | Shows |
|---|---|
| Proof Latency | proof-server response time + 2 s threshold |
| zkZap Incidents | incidents grouped by threat type |
| Vital Health | status mix per vital + overall health % |
| Attestations | cumulative on-chain commitments + avg attest latency |
const forwarder = new SplunkForwarder(config, { dappName: 'BlindOracle', environment: 'production' });
await forwarder.connect();
<VitalsProvider splunkCallbacks={{
onVitalCheck: forwarder.handleVitalCheck,
onLogEntry: forwarder.handleLogEntry,
onDiagnosticReport: forwarder.handleDiagnosticReport,
}}>
<App />
</VitalsProvider>Every vital check, log entry, and diagnostic report flows to Splunk automatically. Any Midnight DApp is a candidate integration (see docs/10_CANDIDATE_INTEGRATIONS.md).
| Layer | Technology | Role |
|---|---|---|
| Telemetry source | MidnightVitals | React 18 / TS 5 diagnostic console |
| Blockchain | Midnight Network | Privacy-preserving L1 (Cardano partner chain) |
| Smart contracts | Compact v0.23 | ZK-proof DSL (>= 0.16 && <= 0.23) |
| ZK proving | Midnight Proof Server | Halo 2 / UltraPlonk prover (Docker) |
| Wallet / SDK | Lace + Midnight.js | Keys, coin selection, ZSwap |
| Ingestion | Splunk HEC | Batched, retrying HTTP Event Collector |
| Analytics | Splunk Cloud | Dashboards, SPL, saved searches |
| AI | Splunk MCP / Splunk REST + Splunk AI Toolkit ` | ai` |
| Future AI | Splunk MCP + Midnight MCP | Planned dual-MCP bridge with midnight-manual grounding |
| Future automation | Splunk SOAR + zkZap | Planned playbook + on-chain incident response |
This isn't a pitch deck — it's working, type-checked code.
connector/— production HEC client (batch + exponential retry + heartbeat),SplunkForwarderlifecycle, type-safevitals-adapter, canonicaltelemetry-commitment,attestation-client, 14 field extractions + 11 SPL saved searches.contract/—zksplunk.compact(sealed ledger +persistentHashkeys; Merkle-membership operator registry + nullifier-based anonymous attestation;Severity/IncidentClassenums). The contract is demo-wired but not audited yet.vitals/— full MidnightVitals module (mock + live provider interface, UI components).demoLand/+zkMonitor/— offline simulated runner with zkZap attack scenarios + a tabbed metrics dashboard, and live HTTP→HEC wiring.splunk-app/zksplunk/— installable Splunk app with global operator map, component detail dashboard, overview dashboard, saved searches, alert configuration, and packaged.spl.ai-agent/— local analyst chat over Splunk evidence using Splunk MCP when configured, REST fallback otherwise, and Splunk AI Toolkit| aiphrasing when enabled.zkMonitor/src/onchain-status-reader.ts— read-only Midnight indexer poller that emitszksplunk:onchainstatus and public incident events into Splunk.
- Dual MCP bridge — combine Splunk MCP evidence with Midnight MCP contract/docs/tooling context in one agent loop.
- Audited on-chain attestation — audit and harden
zksplunk.compact, then validate live attestation end to end. - zkZap automated response — turn detections into Splunk SOAR playbooks and optional on-chain
attestCriticalIncidentcalls. - Macro consortium view — aggregate anonymous incident classes across operators without exposing node/operator identity.
ZKSplunk_Splunking_w_Midnight/
├── connector/ # HEC client, forwarder, adapter, commitments, attestation
├── vitals/ # MidnightVitals: provider interface + mock + UI
├── contract/ # zksplunk.compact (attestations + incidents)
├── demoLand/ # simulated runner + zkZap scenarios + dashboard generator
├── zkMonitor/ # live vitals → Splunk HEC
├── splunk-app/ # Splunk app dashboards, alerts, saved searches, .spl package
├── ai-agent/ # local analyst over Splunk MCP/REST evidence
└── docs/ # PUBLIC_LEDGER_OBSERVABILITY · ZKZAP_SECURITY_PROTOCOL · DEMOLAND_VS_ZKMONITOR · CANDIDATE_INTEGRATIONS · ai-chat/
| Phase | Window | Goal |
|---|---|---|
| Build sprint | now → Jun 13 | End-to-end live telemetry, Splunk app dashboards, local analyst loop |
| Submit | Jun 13 → Jun 15 | Demo video, architecture_diagram at root, public repo, Devpost submission |
| Post-hackathon | — | Dual-MCP bridge, contract audit, SOAR playbooks, Splunkbase publish, opt-in Macro consortium, cross-ZK-chain support |
ZKSplunk is part of the DIDzMonolith ecosystem — privacy-preserving products + books, all on Midnight (see DIDzMonolith_overview.md). It's the operational spine: any product with a contract in a production loop can stream MidnightVitals → ZKSplunk, and zkZap then provides privacy-preserving incident awareness across the whole suite.
Apache 2.0 — see LICENSE.
Built by EnterpriseZK Labs LLC and Alex Pestchanker.