Skip to content

chore(release): 0.14.0 for the engine and all four SDKs - #89

Merged
rrrodzilla merged 232 commits into
mainfrom
release/0.14.0
Sep 20, 2026
Merged

rrrodzilla merged 232 commits into
mainfrom
release/0.14.0

Conversation

@rrrodzilla

Copy link
Copy Markdown
Contributor

Release 0.14.0 for the engine and all four SDKs. This pull request exists so that CI runs on GitHub before any SDK tag is pushed: the publish workflows do not run tests themselves.

Every local gate passes on this branch: fmt, clippy with 0 warnings, 534 Rust tests, doctests, the scaffold compile test, cargo package -p emergent-client, 398 Python tests with ruff and mypy clean, 113 TypeScript tests, the Go race suite and the Go and Deno examples.

Draft release notes follow.

Emergent 0.14.0 (engine and all four SDKs)

This release fixes message loss, silent failures and local exposure across the engine and every SDK. It contains breaking changes in each SDK and new load-time checks in the engine, so read "Before you upgrade" first. The engine jumps from 0.10.10 to 0.14.0 so that the engine and the SDKs share one version from here on.

Before you upgrade

  • Upgrade in order. Publish or install the SDKs first, then emergent-primitives 0.12.0, then the engine. The marketplace catalog is now the primitives release itself.
  • retention_days is enforced, and it defaults to 30. It was parsed and ignored before. On first start the engine prunes events and rotated JSON logs older than that, whether or not your config sets the key. Set retention_days = 0 to keep everything.
  • Unknown config keys are load errors. A misspelled key used to be ignored silently.
  • max_connections is checked at load. The engine refuses to start when the limit cannot cover the enabled primitives plus 4.
  • Primitive names are validated. A name that cannot form a message type is rejected by both the loader and emergent scaffold.
  • Rebuild your primitives on SDK 0.14.0. Wildcard subscriptions are only delivered by SDK 0.14.0 and later; an older SDK sends prefix.* as an exact name.

Engine

Security

  • The HTTP API answers only to IP literals, localhost and names listed in the new [engine].api_allowed_hosts. Any other Host gets 421 Misdirected Request, over HTTP/1.1 and HTTP/2. This closes DNS rebinding against /api/topology. If you reach the API through a reverse proxy under a host name, list that name.
  • New [engine].enforce_declarations = "off" | "warn" | "strict" (default off). In strict the broker refuses a publish or subscribe outside the primitive's declared publishes and subscribes.
  • The engine names an IPC connection by the child process it descends from, not by the name the client sends, so a primitive cannot publish under another's name. Primitives started through a wrapper such as uv run or sh -c are named like direct spawns. A process the engine did not start (a CLI query, the topology viewer, a primitive you run by hand) gets the protocol topics and no primitive's name, so with enforce_declarations at warn or strict a hand-started primitive is reported or refused. When a primitive exits, connections still held under its name are closed. Ancestry is read from /proc, so wrapped primitives are named on Linux only; elsewhere the engine falls back to the name on the message and says so at start.
  • New [engine].authenticate_connections = "off" | "warn" | "strict" (default off). strict refuses a peer running as another user. It still admits a same-user peer, because the kernel cannot tell a viewer from an impostor at admission.
  • Marketplace archives are checksum-verified before extraction.

Reliability

  • Startup waits for each tier's primitives to subscribe instead of sleeping 50 ms per tier. Events published by a fast source no longer miss a slow sink, and an all-Rust topology reaches "Engine ready" in about 25 ms instead of about 200 ms. startup_ready_timeout_ms bounds the wait.
  • Shutdown uses deadlines (shutdown_drain_ms, shutdown_grace_ms), then SIGKILL to the process group. Measured exit: about 130 ms, down from 7 to 14 s.
  • New opt-in per-primitive restart policy.
  • Primitives die with the engine on Linux, even when the engine is killed.
  • Trailing-wildcard subscriptions (system.started.*) now route. Overlapping topics deliver one copy. A wildcard anywhere but the end is a load error.
  • publish_ack of a system.request.* message is acknowledged instead of reported as failed, and the broker no longer sends itself a reply for every fire-and-forget publish.
  • /api/topology and the pub/sub topology answer report live state and pid in a stable order.
  • The retention prune runs off the async runtime. Lifecycle events can no longer abort the engine. The engine logs to stderr when its log file cannot be opened.

Marketplace and scaffold

  • The registry is fetched over HTTPS from the emergent-primitives release assets. The separate emergent-registry repository is frozen.
  • emergent scaffold pins the SDK version it was built with, fails when it cannot write every file, and rejects names the loader would reject.

SDKs (Rust, Python, TypeScript, Go)

Breaking changes

  • Rust: run_handler passes the handler to the closure by value (EmergentHandler is now Clone), so the closure can publish through it. ClientError gains the variant InvalidSubscriptionTopic, which breaks exhaustive matches. MessageStream is no longer UnwindSafe. New: PublishStats and publish_stats() for refused fire-and-forget publishes, fallible EmergentMessage::try_new constructors, the subscribe module, and VERSION.
  • TypeScript: publish() throws PublishError (with cause) when the socket refuses a write. The exported subscription, publish and discovery error classes are now the ones actually thrown.
  • Python: a refused write raises PublishError or ConnectionError from the SDK, not a raw OSError. The default wire format is MessagePack. Unknown envelope fields are ignored.
  • Go: SubscriptionError, PublishError, DiscoveryError and ConnectionError carry Err and implement Unwrap. Unkeyed struct literals of these types no longer compile.
  • All: EMERGENT_UNWRAP_STDOUT is on only for true or 1, trimmed and case-insensitive.

Fixes in every SDK

  • Wildcard subscriptions are delivered.
  • When the engine closes the connection, in-flight requests fail at once instead of timing out, streams end, and run_source fires its shutdown signal, so a hand-started source exits instead of publishing into a dead socket.
  • A malformed frame is skipped instead of ending the read loop (Python, TypeScript, Go keep the frames behind it).
  • Requests answered with an ERROR frame settle instead of timing out.
  • The shutdown kind and the discovery reply are read from where the engine actually puts them.
  • A closing stream no longer unregisters its replacement, a failed subscribe closes its stream, and a second subscribe() ends the stream it replaces.
  • TypeScript writes every byte of a frame, one frame at a time. Go makes MessageStream push and Close safe against each other.

Examples, CI and the agent skill

  • The Deno examples run on a fresh clone again (examples/deno.json).
  • CI runs every language's gates on push and pull request.
  • skills/emergent ships a coding-agent skill, verified against the engine, the four SDKs and primitives 0.12.0.

A coding agent left to its defaults wraps a whole workflow in one
exec-handler and calls it a pipeline. Ship the skill that teaches the
idiom alongside the engine, so the guidance versions with the code it
describes.
The stopping rules bound a primitive's size but said nothing about what
it is built from, so a topology could pass them with a score.sh behind
every block. Add a ladder that prefers marketplace primitives, then bare
commands, then one shape-act-shape pipe, with no rung for a script file,
plus the tells that a command has become a program and the split each
one implies.
The reference predated primitives 0.10.0. Most consequentially it still
said a non-zero exit with empty stderr is a silent filter; since 0.10.0
that is an error event unless the code is listed in --silent-exit-codes,
and jq select() filters by exiting 0 with no output. Also fixes wrong
short flags and env names, documents the missing flags and payload
shapes, and records that the publishes array overrides topic flags by
position.
Subscriptions are exact-match, unknown keys are ignored, retention_days and
wire_format have no effect, and an invalid primitive name hangs startup. The
topology-viewer note now says what it renders today: the engine node only.
Wire format is always MessagePack, subscriptions are exact-match, shutdown runs
on fixed timers with no SIGKILL, nothing is restarted, and the engine answers
only subscription requests.
The Rust run_handler example could not compile when the handler publishes, so
it is now the subscribe loop. TypeScript, Go, and the lint guidance are fixed
against the real signatures. Rust and TypeScript snippets were compiled.
stream-runner replaces the shell splitter, every shell body holds a single act,
the routers are exclusive and exhaustive, and every path ends in the ack. The
old config lost the issue title on re-entry, never bounded its loop, and read
stdin twice. The new one was run end to end against stubs.
The join is now an accumulator, a router, and a sink. Scripts, shell branches,
and wildcard subscriptions are gone, fragments carry their path, and the
exit-code, concurrency, and stream-runner claims match primitives 0.12.0. The
join, fan-in, reaper, and self-seeding patterns were run live.
The join is three primitives instead of a shell conditional that published
errors, shell bodies are TOML literal strings, the one-event and silent-exit
claims match the code, and Gate 2 gains a dead-end test covering unrouted
events and wildcard subscriptions.
…havior

An invalid primitive name aborts the engine and orphans the spawned child, it
does not hang. The ignored system.shutdown kind is confirmed at runtime for the
Rust, Python and TypeScript SDKs. Known limitations now cite their issues.
The Python client treated any non-empty EMERGENT_UNWRAP_STDOUT as on, so
setting it to "false" or "0" to switch unwrapping off enabled it instead.
TypeScript accepted only "true". Both now parse the flag through one pure
function that accepts "true" and "1", case-insensitively, which is what
the Rust and Go SDKs already do.

Closes #47
get_topology() in every SDK published system.request.topology and waited
for system.response.topology, but the engine only forwarded the request to
subscribers and expected a "topology-query handler" that never shipped.
Every call timed out after 30 s.

The engine now answers the request itself, in the same style as
system.request.subscriptions: it replies with system.response.topology,
source emergent-engine, the request's correlation_id copied onto the
envelope where the SDKs match it, and the primitive list as the payload.

The primitive list is built by a new pure function,
emergent_engine::topology::build_topology_payload, which the HTTP
GET /api/topology handler now uses too, so the two transports cannot
report different topologies.

Refs #46
EmergentHandler::messages and EmergentSink::messages took a types
argument and threw it away, always subscribing to the engine-configured
list instead. EmergentSink::messages("x", ["timer.tick"]) against a
config with an empty subscribes list received nothing and reported no
error. The Python and TypeScript sinks had the same defect.

The rule is now: the topics you pass win, and an empty list falls back to
the engine's configured subscribes list. The engine is only queried when
the caller requested nothing, so the explicit case also drops a round
trip. The choice lives in two pure functions per SDK, resolve_topics and
needs_configured_topics, each with unit tests, and is documented on every
method that takes the argument.

This is not a breaking change: callers that passed topics matching their
config keep the same behavior, and callers that passed an empty list keep
the config-driven behavior.

The Go SDK has no messages() convenience, so it needed no change.

Refs #46
An agent given only this skill designed an invoice-intake topology and
reported where the skill misled it. Each fix below was checked against a
live engine before it was written down.

- Open every piped shell body with `set -o pipefail`. Without it a failed
  act in the middle of a pipe left the trailing jq to exit 0, so the
  handler published nothing (a permanent stall behind a stream-runner) or
  published a success event with empty content.
- Merge named fields from the act instead of `$orig + .`, so untrusted
  model output cannot overwrite an item's identity.
- Add the `act && shape` form of rung 3 for acts with no stdin or stdout,
  with a tested claim-by-rename example, and say why `&&` is not a branch.
- Split the retry pattern into guard, then delay, so an exhausted item
  escalates at once instead of sleeping its longest sleep first. Note that
  retried acts must be idempotent.
- Add HTTP status as data, so a 4xx is routed to a person and only
  transport failures and 5xx reach the retry loop.
- Say where a stream-runner ack can go and what each placement costs, that
  a poll needs a seen-set, that exec-source cannot wrap a streaming
  command, and that `--ack-topic` takes one topic.
- Reconcile statements that disagreed: which error topics need a name,
  the dead-end test versus deliberate "nobody yet" events, the long-timeout
  tell versus delay handlers, the name test versus router names, and the
  subscriber test for steps inserted between stages.
- Add the requirement test, the one gate that argues for fewer primitives,
  and stop the straight-line check from inviting invented loops.
- Give a judge that reads outside text no tools (`--tools ""`).
…ting

The retry handler slept before checking its guard and disagreed with the
pattern it was meant to illustrate. It is now two exclusive routers on the
failure event followed by a delay whose only act is the pause, with the
backoff read from the payload. The feedback edge is renamed for what became
true (`issue.backoff-elapsed`).

Piped bodies open with `set -o pipefail`, the scoring merge takes three
named fields from the model, the judge runs with no tools, and an idle poll
publishes no list event. The catalog says which events are unconsumed on
purpose, and Gate 2 gains the requirement and dead-end rows.

Re-run end to end against stubbed gh, claude, and curl: three issues in,
three settled, nothing on exec.error, and the third scoring failure
escalated with no pause in front of it.
EmergentSink::messages connected, subscribed, and returned only the
stream. The sink was dropped when the function returned, which dropped
the last reference to the IPC client, closed the connection, and closed
the stream. Callers got a stream that yielded None on the first read
instead of messages, with no error.

MessageStream can now own the connection that feeds it, and
EmergentSink::messages hands its client over before returning. Nothing
else changes: the field is private and unused by every other
constructor, which still returns the primitive alongside its stream.

Verified against a live engine: a sink with an empty configured
subscribes list now receives a message published on the topic it asked
for, where before the stream closed immediately.

The Python and TypeScript sinks are async generators that hold their
sink for the life of the iteration, so neither shares this defect.

Refs #46
The pub/sub topology section told readers to run a handler that
subscribes to system.request.topology, reads the HTTP API, and publishes
the response. The engine answers the request itself now, so that handler
is dead weight.

Removes examples/handlers/topology-api, which also contradicted the
engine: it put the correlation id inside the payload as correlationId,
while every SDK matches on the envelope's correlation_id. The source and
viewer examples still apply and are unchanged; the engine simply plays
the part the handler used to.

Closes #46
The engine now answers system.request.topology itself and messages()
honors its types argument, so the references no longer tell readers to add
a responder handler or to expect the argument to be ignored. Each statement
names the last release with the old behavior.

Refs #46
mypy rejected one variable reused for three request types; each pending
lookup now has its own name. The frozen-model tests assert pydantic's
ValidationError instead of a blind Exception, the stream tests keep and
await the tasks they spawn, and two annotation-only imports move under
TYPE_CHECKING. ruff check, ruff format, and mypy are all clean.
The skill's SDK reference described the Rust helper only. It now says
that the Python, TypeScript and Go helpers fire the same shutdown signal
when the engine closes the connection, what the earlier releases did,
and which Sources were exposed. The Go SDK page says the same in short.

Refs #83
The startup line and the middleware want the same list, and parsing it twice
invited the two to disagree. Share one Arc.

Refs #81
sdks/ts has a package.json, which puts Deno in manual node_modules mode. In a clean copy of the tree deno check fails with "Could not find a matching package for 'npm:typeid-js@1.2.0'" until deno install runs; with it, check passes and 106 tests pass. The JSR publish workflow already installs.

Refs #70
The API answered any Host header, so a page on any domain could rebind
that domain to 127.0.0.1 and read the topology from the browser. It now
answers to IP literals, localhost and the names in
[engine].api_allowed_hosts, and refuses the rest with 421, over HTTP/1.1
and HTTP/2 alike.

Closes #81
Python and TypeScript no longer orphan a newer message stream when an
older one closes, and close the stream of a subscribe that throws.
run_source in Python, TypeScript and Go now fires its shutdown signal
when the engine closes the connection, so a hand-started Source stops
instead of publishing into a dead socket.

Closes #82
Closes #83
…dules

The root package.json puts Deno in manual node_modules mode for every
script below it, so on a fresh clone console_color, topology-viewer and
topology-api exited 1 on the SDK's npm:typeid-js import. examples/deno.json
turns that mode off for the examples, and CI now type-checks all three.

Closes #85
Admission walked nowhere: every peer was Unmanaged, so declaration
enforcement fell back to the source a client writes itself and a
subscribe frame could not be judged at all.

The resolver now walks the peer's process ancestry to a pid the engine
spawned, which covers a launcher such as `uv` that stays alive as the
engine's child while its interpreter opens the socket. The walk is a
pure function over the child table and a parent lookup; /proc is the
only thing the shell around it touches.

A pid is a primitive exactly while the engine holds that child as live:
an unreaped child's pid cannot be recycled, and a grandchild pid that is
recycled no longer climbs to a recorded child. The module docs make the
argument and the table tests cover it, along with the uv chain, deeper
wrappers, a foreign process, the engine's own pid, a missing peer pid,
the no-ancestry collapse, a cycle, pid 1 and a comm full of parentheses.

Connections admitted as a primitive are recorded, and every one of them
is revoked when that primitive's child exits.

[engine].authenticate_connections says what happens to a peer the engine
did not spawn: off admits it, warn says so, strict refuses another user
id and still admits a same-user one, because the CLI and a primitive run
by hand cannot be told from an impostor at admission.

Refs #24
A client feeds one registered stream. A second subscribe() took the
registration and left the earlier stream open and unfed, and from then
on nothing could end it: not connection loss, not system.shutdown, not
close(). An async for over it waited for ever.

The stream being replaced is now closed at the moment the new one is
registered, so its consumer stops. The engine keeps the earlier
subscriptions, so the new stream receives the earlier topics too, which
the subscribe docs and the README now say.

Refs #86
A client feeds one registered stream. A second subscribe() took the
registration and left the earlier stream open and unfed, and from then
on nothing could end it: not connection loss, not system.shutdown, not
close(). A for await over it waited for ever.

The stream being replaced is now closed at the moment the new one is
registered, so its consumer stops. The engine keeps the earlier
subscriptions, so the new stream receives the earlier topics too, which
the subscribe docs and the README now say.

Refs #86
A client feeds one registered stream. A second Subscribe took the
registration and left the earlier stream open and unfed, and from then
on nothing could close it: not connection loss, not system.shutdown,
not Close. A range over its channel waited for ever.

The stream being replaced is now closed when the new one is registered,
after c.mu is released because its onClose callback takes c.mu. The
engine keeps the earlier subscriptions, so the new stream receives the
earlier topics too, which the Subscribe docs and the README now say.

Refs #86
A Rust client hands its push channel to its first stream, so a second
stream can never be registered and the first is never orphaned. The
second subscribe was refused, but only after the engine had accepted
it: the refused topics started arriving on the first stream while the
caller held SubscriptionFailed and subscribed_types() did not list
them.

The refusal now comes first, from a pure check on whether this client
has handed out its stream, so the first stream and its subscriptions
stay as they were. Tests run both a Handler and a Sink against a real
engine.

Refs #86
One stream per client in all four SDKs. In Python, TypeScript and Go a
second subscribe ends the earlier stream and the new one receives the
earlier topics too, because the engine keeps those subscriptions. In
Rust the second call is refused before anything is sent. The skill's
SDK reference and the Go and Rust SDK pages now say so, with what the
earlier releases did.

Refs #86
In Python, TypeScript and Go a second subscribe left the first stream
open forever; it now ends the stream it replaces. The Rust SDK already
refused a second subscribe, but only after sending it, so the refused
topics were delivered anyway; it now refuses before anything is sent.

Closes #86
…e a socket

A primitive that dies with its own socket takes the connection with it, so
the sweep on child exit finds nothing left to close and said nothing. The
sweep that matters is the other one: a descendant still holding a connection
the engine admitted under a name that is now dead. Log held and closed counts
at debug for both, so an operator can see the path ran.

Refs #24
The engine moves from 0.10.10 to 0.14.0 so the engine and the SDKs share
one version. 0.11.0 is skipped because a v0.11.0 tag already exists.
Version said 0.10.5 in Go and __version__ said 0.1.0 in Python while both
were released as 0.13.1. Each now has a test that fails when the constant
and the released version disagree.

Closes #87
…ctions

The docs still said the engine checks self-reported names and that issue #24
was open. Rewrite the enforcement sections around the resolver that ships:
where a connection's name comes from, why pid reuse is safe, what revocation
on child exit is for, what authenticate_connections does and does not buy, and
what degrades where process ancestry is unavailable.

Refs #24
# Conflicts:
#	CLAUDE.md
#	emergent-engine/src/config.rs
The fixes on this branch were described relative to the last published
versions, because the release number was not yet decided. It is now 0.14.0
for the engine and all four SDKs, so the text names it. Statements about
0.10.10 and 0.13.1 themselves stay as they are.
The steps still said the Go SDK has no version file and that the engine
and the SDKs were on different numbers.
An IPC connection is now named by the spawned child it descends from,
read from the peer credentials and /proc, not by the name the client
sends. A primitive started through uv or sh is named like a direct
spawn, so declaration enforcement applies to it and its tier is
released on its subscribe instead of at the 5000 ms deadline. A
connection the engine cannot tie to a child gets the protocol topics
and no primitive's name. Connections held for a child are revoked when
it exits. authenticate_connections = off, warn or strict governs only
whether a peer under another uid is admitted.

Closes #24
Closes #23
….14.0

Conflicts were the version wording only: the sweep side is kept and the
0.14.0 wording reapplied to it.
t.TempDir() embeds the test and subtest names. Go 1.23, which CI uses,
does not shorten them, so three tests built a socket path the kernel
refuses with 'bind: invalid argument'. Newer Go releases shorten the
name, which is why the suite passed locally.
GitHub creates no push event when more than three tags arrive in one
push, so the four SDK tags pushed together started no publish workflow.
@rrrodzilla
rrrodzilla merged commit 5d9b418 into main Sep 20, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant