Skip to content

[WIP] Remove global delayed-data runtime plumbing#2472

Draft
lquerel wants to merge 32 commits intoopen-telemetry:mainfrom
lquerel:feature/remove-global-delayed-data
Draft

[WIP] Remove global delayed-data runtime plumbing#2472
lquerel wants to merge 32 commits intoopen-telemetry:mainfrom
lquerel:feature/remove-global-delayed-data

Conversation

@lquerel
Copy link
Copy Markdown
Contributor

@lquerel lquerel commented Mar 31, 2026

Not ready for review. #2471 must be merged first to reduce the scope of this PR

Change Summary

Remove the old runtime-managed delayed-data path now that node-local wakeups and node-local delayed resume are in place.

This PR deletes the remaining global delayed-data plumbing from the engine, including the runtime control path and the delayed-data-specific node control message flow. After the previous PRs:

  • wakeup-style local scheduling is handled locally in ProcessorInbox
  • retry-style delayed pdata resume is handled locally via requeue_later(...)

This is the cleanup step in the redesign tracked in #2465. It simplifies the runtime manager, reduces global contention, and makes shutdown behavior easier to reason about by removing delayed-data-specific special cases from the global control path.

What issue does this PR close?

How are these changes tested?

  • Removed or updated engine tests that previously depended on the global
    delayed-data path
  • Re-ran the existing coverage for:
    • local wakeups
    • local delayed resume
    • retry behavior
    • shutdown behavior
  • Verified with:
    • cargo xtask check

Are there any user-facing changes?

No user-facing changes.

This PR is an internal runtime cleanup only.

lquerel added 30 commits March 26, 2026 23:07
# Conflicts:
#	rust/otap-dataflow/crates/engine/src/message.rs
#	rust/otap-dataflow/crates/engine/src/processor.rs
# Conflicts:
#	rust/otap-dataflow/crates/engine/src/message.rs
#	rust/otap-dataflow/crates/engine/src/processor.rs
@github-actions github-actions bot added the rust Pull requests that update Rust code label Mar 31, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 31, 2026

Codecov Report

❌ Patch coverage is 77.98834% with 906 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.17%. Comparing base (5ef4ab5) to head (b33cf2b).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2472      +/-   ##
==========================================
- Coverage   88.35%   88.17%   -0.19%     
==========================================
  Files         603      608       +5     
  Lines      212379   215944    +3565     
==========================================
+ Hits       187646   190403    +2757     
- Misses      24207    25015     +808     
  Partials      526      526              
Components Coverage Δ
otap-dataflow 90.09% <77.98%> (-0.29%) ⬇️
query_abstraction 80.61% <ø> (ø)
query_engine 90.74% <ø> (ø)
syslog_cef_receivers ∅ <ø> (∅)
otel-arrow-go 52.44% <ø> (ø)
quiver 91.94% <98.90%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant