Skip to content

feat(config): extensions section, capabilities bindings, and config-level validation#2538

Open
gouslu wants to merge 2 commits intoopen-telemetry:mainfrom
gouslu:gouslu/extension-system-pr1-config
Open

feat(config): extensions section, capabilities bindings, and config-level validation#2538
gouslu wants to merge 2 commits intoopen-telemetry:mainfrom
gouslu:gouslu/extension-system-pr1-config

Conversation

@gouslu
Copy link
Copy Markdown
Contributor

@gouslu gouslu commented Apr 3, 2026

Change Summary

This is PR 1 based on #2510.

Add config parsing for pipeline extensions — the first building block of the extension system (architecture doc).

Extensions are standalone pipeline components that provide shared capabilities (auth, storage, etc.) to data-path nodes. This PR adds config-level support only — no runtime behavior. Extensions are parsed and validated but not created or started.

Config changes:

  • extensions: section in pipeline config, separate from nodes:
  • capabilities: field on nodes for binding capabilities to extension instances
  • NodeKind::Extension variant for URN parsing (e.g., urn:otel:extension:auth)
  • ExtensionConfig struct in engine config
  • PipelineConfigBuilder::add_extension() for programmatic config construction

Validations added:

  • Capability bindings must reference extensions that exist in the extensions: section
  • Extensions cannot have capabilities: bindings (they provide capabilities, not consume them)
  • Extension URN in nodes: section → ExtensionInNodesSection error
  • extensions: at group level rejected by serde (deny_unknown_fields)
  • Duplicate names across nodes and extensions rejected

Example config:

extensions:
  azure_auth:
    type: "urn:microsoft:extension:azure_identity_auth"
    config:
      method: "managed_identity"

nodes:
  exporter:
    type: "urn:microsoft:exporter:azure_monitor"
    capabilities:
      bearer_token_provider: "azure_auth"

What issue does this PR close?

  • Part of the extension system initiative (Phase 1, PR 1 of 8)

How are these changes tested?

13 new config tests covering:

  • Extensions parsed separately from nodes
  • Extension with config and capability bindings
  • Extension URN kind detection
  • Duplicate extension/node name rejection
  • Extension URN in nodes section detection
  • Non-extension URN in extensions section detection
  • Empty/missing extensions section
  • Empty capabilities
  • Multiple extensions with capability bindings
  • Capability binding to nonexistent extension (rejected)
  • Capability binding to existing extension (passes)
  • Capabilities on extension itself (rejected)
  • Extensions at group level (rejected by serde)

All 168 config tests pass. Example config YAML added to configs/.

Are there any user-facing changes?

Yes — pipeline YAML configs now support extensions: and capabilities: sections. These are optional and have no runtime effect in this PR.

@gouslu gouslu requested a review from a team as a code owner April 3, 2026 21:04
@github-actions github-actions bot added the rust Pull requests that update Rust code label Apr 3, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

❌ Patch coverage is 92.54386% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.38%. Comparing base (ce41c95) to head (a2ee9bb).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2538      +/-   ##
==========================================
+ Coverage   88.34%   88.38%   +0.04%     
==========================================
  Files         612      613       +1     
  Lines      221481   223188    +1707     
==========================================
+ Hits       195665   197266    +1601     
- Misses      25292    25398     +106     
  Partials      524      524              
Components Coverage Δ
otap-dataflow 90.28% <92.54%> (+0.03%) ⬆️
query_abstraction 80.61% <ø> (ø)
query_engine 90.74% <ø> (ø)
syslog_cef_receivers ∅ <ø> (∅)
otel-arrow-go 52.45% <ø> (ø)
quiver 91.92% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gouslu gouslu force-pushed the gouslu/extension-system-pr1-config branch 2 times, most recently from 4eec6cc to 18e9f65 Compare April 3, 2026 21:42
@gouslu gouslu force-pushed the gouslu/extension-system-pr1-config branch from 18e9f65 to a520a65 Compare April 3, 2026 21:44
@kamilon
Copy link
Copy Markdown
Contributor

kamilon commented Apr 3, 2026

The architecture doc link doesn't work. Can you please fix that? I'd like to understand what this is being built against.

@gouslu gouslu marked this pull request as draft April 3, 2026 23:39
@jmacd jmacd marked this pull request as ready for review April 6, 2026 19:34
@jmacd
Copy link
Copy Markdown
Contributor

jmacd commented Apr 6, 2026

I believe this is ready to review because this PR1 is consistent with the original extensions design document. If there are objections to #2510, let them surface now! I will merge that PR before I merge this one, but both are ready to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants