Skip to content

Commit 34c3604

Browse files
committed
update document to explain shared being per core per pipeline for phase 1
1 parent ff01875 commit 34c3604

File tree

1 file changed

+67
-17
lines changed

1 file changed

+67
-17
lines changed

rust/otap-dataflow/docs/extension-system-architecture.md

Lines changed: 67 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,24 @@ is addressed in the Phase 1 implementation:
3030
| Future hierarchical scopes | `CapabilityRegistry` and `resolve_bindings()` are scope-agnostic by design |
3131

3232
In addition to implementing the proposal's requirements,
33-
this design introduces two capabilities beyond what the
33+
this design introduces three refinements beyond what the
3434
proposal envisioned:
3535

36+
- **Pipeline-scoped shared is per-core.** The proposal
37+
defines shared extensions as "one runtime instance
38+
shared across cores." In Phase 1, both local and shared
39+
pipeline-scoped extensions are instantiated **per
40+
pipeline instance** (i.e., per core). The local/shared
41+
distinction at pipeline scope is about type constraints
42+
(`!Send` vs `Send + Clone`), not about cross-core
43+
sharing. This follows a consistent principle: **an
44+
extension's sharing boundary is determined by the scope
45+
it is declared in**, not by its execution model.
46+
Pipeline-scoped extensions are per pipeline instance,
47+
group-scoped extensions are shared across the group,
48+
and engine-scoped extensions are shared globally.
49+
The execution model (local vs shared) determines only
50+
the type constraints imposed on the implementation.
3651
- **Active/Passive lifecycle distinction.** The proposal
3752
describes extensions with background tasks, but does not
3853
distinguish extensions that only provide capabilities
@@ -768,9 +783,15 @@ evolve toward hierarchical scoping in future phases.
768783
### Pipeline Scope (Phase 1)
769784

770785
Extensions are declared at the pipeline level and consumed
771-
by nodes within that pipeline. Each pipeline instance (one
772-
per core in thread-per-core mode) gets its own extension
773-
instances.
786+
by nodes within that pipeline. Following the principle that
787+
**sharing boundary is determined by the scope**, both local
788+
and shared pipeline-scoped extensions are instantiated per
789+
pipeline instance (one per core in thread-per-core mode).
790+
The local/shared distinction at this scope controls type
791+
constraints only: local uses `Rc`-based `!Send` types,
792+
shared uses `Clone + Send` types. See the
793+
[refinements section](#how-this-document-relates-to-the-proposal)
794+
for the rationale behind this design choice.
774795

775796
```yaml
776797
groups:
@@ -814,18 +835,47 @@ extension wins.
814835
815836
| Scope | Execution Model | Sharing |
816837
|----------|-----------------|----------------------------------|
817-
| Pipeline | Local or Shared | Per pipeline instance (per core) |
818-
| Group | Shared | Across pipelines in the group |
819-
| Engine | Shared | Across all pipelines |
820-
821-
- Pipeline-scoped extensions can be local (`!Send`,
822-
`Rc`-based) or shared (`Send`, `Clone`-based).
823-
- Group and engine-scoped extensions must be shared
824-
(`Send + Clone`) since they are accessed from multiple
825-
pipeline instances running on different cores.
826-
827-
**No architectural changes required:** the Phase 1
828-
`CapabilityRegistry`, `resolve_bindings()`, and
838+
| Pipeline | Local | Per pipeline instance (per core) |
839+
| Pipeline | Shared | Per pipeline instance (per core) |
840+
| Group | Shared only | Across pipelines in the group |
841+
| Engine | Shared only | Across all pipelines |
842+
843+
At the pipeline level, both local and shared extensions
844+
are instantiated **per pipeline instance** (i.e., per
845+
core in thread-per-core mode). The distinction between
846+
local and shared at this scope is about **type
847+
constraints**, not about cross-core sharing:
848+
849+
- **Local** extensions use `Rc`-based, `!Send` types.
850+
They cannot leave the core they were created on.
851+
This enables lock-free designs using `Rc`, `RefCell`,
852+
and `Cell`.
853+
- **Shared** extensions use `Clone + Send` types. They
854+
are still instantiated per pipeline instance, but
855+
their `Send` bound means they *could* be shared across
856+
cores. At the pipeline scope this is not needed --
857+
it simply means the implementation uses thread-safe
858+
primitives (e.g., `Arc`, `RwLock`).
859+
860+
Cross-core sharing becomes meaningful when extensions
861+
are declared at **higher scopes** (group or engine) in
862+
Phase 2. At those scopes, a single extension instance
863+
serves multiple pipeline instances running on different
864+
cores, so only the shared (`Send + Clone`) execution
865+
model is permitted -- local extensions are not allowed
866+
at group or engine scope.
867+
868+
This design keeps Phase 1 simple: pipeline-scoped
869+
extensions are always per-core, with no cross-core
870+
coordination. Extension authors who only need pipeline
871+
scope can choose local for maximum performance or shared
872+
for convenience, knowing both are core-local. When
873+
Phase 2 introduces higher scopes, shared extensions
874+
naturally promote to cross-core usage without API
875+
changes.
876+
877+
**No architectural changes required for Phase 2:** the
878+
Phase 1 `CapabilityRegistry`, `resolve_bindings()`, and
829879
`require_local()` / `require_shared()` mechanisms are
830880
scope-agnostic. Adding higher scopes requires:
831881

@@ -834,7 +884,7 @@ scope-agnostic. Adding higher scopes requires:
834884
2. A scope-aware resolution pass in `resolve_bindings()`
835885
that merges registries from inner to outer scope
836886
3. Validation that group/engine-scoped extensions only
837-
use shared execution model
887+
use the shared execution model
838888

839889
The `SharedAsLocal` adapter ensures that even when a
840890
group or engine-scoped shared extension is consumed by a

0 commit comments

Comments
 (0)