This tutorial walks through all 8 Kates test types with real commands and expected outputs. By the end, you'll understand which test to use for every scenario.
- Kates stack deployed, and the CLI configured with a context that carries the API key, as the Quick Start in Introduction sets it up
- System health verified:
kates health
The LOAD test establishes your cluster's steady-state performance at a controlled rate.
When to use: Before any other test. This provides the baseline you'll compare everything else against.
# List the built-in LOAD templates (quick-load, production-load, ci-gate)
kates test scaffold --type LOAD
# Run a basic load test
kates test create --type LOAD \
--records 100000 \
--producers 2 \
--acks all \
--waitWhat to record: Note the P99 latency and throughput. These are your baseline numbers.
graph LR
A["Your Baseline"] --> B["P99: ___ms"]
A --> C["Throughput: ___rec/s"]
A --> D["Errors: 0%"]
The STRESS test ramps load until the cluster can no longer keep up.
When to use: Capacity planning. You need to know how much headroom exists.
# Export the built-in STRESS template
kates test scaffold export stress-test -f stress.yaml
# Quick stress test with 4 producers ramping up
kates test create --type STRESS \
--records 500000 \
--producers 4 \
--duration 180 \
--waitWhat to watch: Look for the phase where P99 latency jumps significantly. That's your saturation point.
graph LR
subgraph Results
R1["Phase 1: Good ✅<br/>P99 < 20ms"]
R2["Phase 2: OK ✅<br/>P99 < 50ms"]
R3["Phase 3: Warning ⚠<br/>P99 > 100ms"]
R4["Phase 4: Breaking ❌<br/>P99 > 500ms"]
end
The SPIKE test hits the cluster with a sudden burst of traffic, then drops back to normal.
When to use: Preparing for events with sudden traffic increases (marketing campaigns, product launches).
# Export the built-in SPIKE template
kates test scaffold export spike-test -f spike.yaml
# View the exported YAML
cat spike.yaml
# Apply it
kates test apply -f spike.yaml --waitThe spike-test template is a single burst: 500,000 records through 32 parallel
producers over 60 seconds, acks: "1" and lingerMs: 0 so the load arrives as
fast as the client can push it. Its gate allows a P99 of 500 ms and an error
rate of 1% — deliberately looser than a LOAD test, because absorbing a burst is
the thing being measured, not steady-state latency.
Key question: How long does P99 take to return to baseline after the spike?
The ENDURANCE test runs at moderate load for an extended period to detect slow resource leaks.
When to use: Before major releases. Run overnight to catch memory leaks, thread leaks, and log accumulation.
# Export the built-in ENDURANCE template (a 1-hour soak at 5k msg/s)
kates test scaffold export endurance-soak -f endurance.yaml
# Start a 30-minute endurance test
kates test create --type ENDURANCE \
--records 1000000 \
--producers 2 \
--duration 1800 \
--waitWhat to watch: Compare metrics from the first 5 minutes vs. the last 5 minutes. Any drift indicates a leak.
graph LR
subgraph Healthy
H1["Start: P99 = 15ms"] --> H2["End: P99 = 16ms<br/>✅ Stable"]
end
subgraph Leaking
L1["Start: P99 = 15ms"] --> L2["End: P99 = 150ms<br/>❌ 10x degradation"]
end
The VOLUME test focuses on large messages and data volumes to stress storage and replication.
When to use: When your production workload includes large messages (images, documents, large events).
# The scenario library ships no VOLUME template — drive this one from flags
# Run with large messages
kates test create --type VOLUME \
--records 10000 \
--record-size 102400 \
--acks all \
--waitThe 100KB record size × 10,000 records = ~1GB of data through the cluster.
What to watch: Does throughput (MB/s) scale linearly, or does it plateau?
The CAPACITY test removes all rate limiting and finds the absolute ceiling.
When to use: When you need hard numbers for capacity planning documents.
# The scenario library ships no CAPACITY template — drive this one from flags
# Run with unlimited throughput
kates test create --type CAPACITY \
--records 1000000 \
--producers 8 \
--throughput -1 \
--waitKey metric: The maximum sustained throughput before errors appear.
graph TB
subgraph Discovery
P1["1 producer<br/>~50K rec/s"] --> P2["2 producers<br/>~95K rec/s"]
P2 --> P3["4 producers<br/>~170K rec/s"]
P3 --> P4["8 producers<br/>~200K rec/s ← MAX"]
P4 --> P5["16 producers<br/>~195K rec/s ← Overloaded"]
end
The ROUND_TRIP test measures the complete journey from producer to consumer.
When to use: When you need to SLA on consumer-side delivery latency, not just producer acknowledgment.
# Export the built-in ROUND_TRIP template (exactly-once, CRC-verified)
kates test scaffold export exactly-once -f roundtrip.yaml
# Run with 1 producer and 1 consumer
kates test create --type ROUND_TRIP \
--records 10000 \
--producers 1 \
--consumers 1 \
--waitKey insight: Round-trip latency is typically 2–5x higher than producer latency because it includes consumer fetch polling intervals.
The INTEGRITY test verifies that every message is persisted and deliverable.
When to use: Whenever you change Kafka configuration, upgrade brokers, or validate a new cluster.
# Export the built-in INTEGRITY template (transactional, zstd, CRC)
kates test scaffold export integrity-tx -f integrity.yaml
# Run data integrity verification
kates test create --type INTEGRITY \
--records 100000 \
--consumers 1 \
--acks all \
--waitExpected output:
Data Integrity
──────────────
Sent 100,000
Acked 100,000
Received 100,000
Lost 0
Mode idempotent
Verdict PASS ✅
For the ultimate validation — verify integrity while killing a broker. No
scenario template carries chaos: a scaffold describes a test, and pairing one
with a fault is what kates resilience run is for. It takes a config with a
testRequest and a chaosSpec side by side. The testRequest goes to the API
as written, so it takes the API's field names (type, numRecords), not the
flags above or a scenario file's records:
cat > integrity-chaos.json <<'EOF'
{
"testRequest": {
"type": "INTEGRITY",
"spec": {
"numRecords": 100000,
"throughput": 500,
"acks": "all"
}
},
"chaosSpec": {
"experimentName": "kafka-pod-kill",
"disruptionType": "POD_KILL",
"targetNamespace": "kafka",
"targetLabel": "strimzi.io/component-type=kafka,strimzi.io/broker-role=true"
},
"steadyStateSec": 30
}
EOF
kates resilience run -f integrity-chaos.jsonThe fault comes after the 30 seconds of steadyStateSec, so the run must still
be producing then. An INTEGRITY run is unthrottled unless you set throughput,
and at 500 records per second these 100,000 records take about 200 seconds. The
selector adds strimzi.io/broker-role=true because the default,
strimzi.io/component-type=kafka, also matches the KRaft controllers.
Tutorial 4 works through this in full.
graph TD
Q1{What do you want<br/>to know?}
Q1 -->|"Normal performance"| LOAD[Run LOAD test]
Q1 -->|"Breaking point"| STRESS[Run STRESS test]
Q1 -->|"Burst handling"| SPIKE[Run SPIKE test]
Q1 -->|"Long-term stability"| ENDURANCE[Run ENDURANCE test]
Q1 -->|"Large message support"| VOLUME[Run VOLUME test]
Q1 -->|"Maximum throughput"| CAPACITY[Run CAPACITY test]
Q1 -->|"Consumer delivery time"| RT[Run ROUND_TRIP test]
Q1 -->|"Data safety"| INT[Run INTEGRITY test]
Q1 -->|"Data safety under failure"| INTC[Run INTEGRITY via kates resilience run]
After running multiple tests:
# List all your test runs
kates test list
# View trends
kates trend --type LOAD --metric p99LatencyMs --days 1
kates trend --type LOAD --metric avgThroughputRecPerSec --days 1- Tutorial 3: Chaos Engineering — inject broker failures
- Tutorial 5: Heatmaps and Exports — deep analysis