Skip to content

KAFKA-14597: record-e2e-latency-max only considers consumption latencies but not processing delays#22452

Open
alanlau28 wants to merge 2 commits into
apache:trunkfrom
alanlau28:KAFKA-14597-e2e-latency-processing-delay
Open

KAFKA-14597: record-e2e-latency-max only considers consumption latencies but not processing delays#22452
alanlau28 wants to merge 2 commits into
apache:trunkfrom
alanlau28:KAFKA-14597-e2e-latency-processing-delay

Conversation

@alanlau28
Copy link
Copy Markdown

@alanlau28 alanlau28 commented Jun 2, 2026

Jira: https://issues.apache.org/jira/browse/KAFKA-14597

The record-e2e-latency metric at terminal nodes was computed as currentSystemTimeMs() - record.timestamp(), where currentSystemTimeMs() returns a wall-clock value cached once at the start of processing a record (set by updateProcessorContext before currNode.process() runs). Because this value is captured before the topology executes, processing delays are invisible to it.

The fix reads the wall clock freshly at the terminal site, after child.process() returns. To avoid a clock read per output record the read is done lazily: currentRecordE2ELatencyTimeMs on StreamTask caches the first fresh read and is reused for every subsequent terminal recording within the same input record's processing. updateProcessorContext resets the field to -1 at the start of each record and punctuation, so each input record gets its own reading taken after its processing completes.

@github-actions github-actions Bot added streams small Small PRs triage PRs from the community labels Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

small Small PRs streams triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant