tmp: emit and merge per-provider TMPX shape (tmpx_providers / tmpx_macros)#392
tmp: emit and merge per-provider TMPX shape (tmpx_providers / tmpx_macros)#392ohalushchak-exadel wants to merge 6 commits into
Conversation
…cros) Schema bundle pin moves from 3.1.0 to `latest` so adcp-go can pick up the TMPX surface changes that landed on adcp main but haven't been cut into a tagged release yet (adcontextprotocol/adcp #5689, #5729). This PR is a draft until a new spec bundle ships and we can re-pin to a signed release version. Wire surface: - New IdentityMatchResponse fields land in tmproto/types_gen.go: * TmpxMacros []TmpxMacro — provider-emitted ordered slot pairs * TmpxProviders map[string]TmpxProviderEntry — router-merged map * TmpxMacro struct (name + value) — generated from $defs - ProviderRegistration grows TmpxMacros []string for the registered macro names. - Generator schema dir moved from /tmp to /trusted-match to match the upstream rename. doc.go's go:generate line bumped. - The schema generator does not synthesize structs for inline object schemas, so tmpx_providers's value type is overridden via go-overlays.json and TmpxProviderEntry is hand-written in tmproto/tmpx_providers.go (mirrors how Attestation/IdentityToken are hand-written for the same reason). Identity agent (Scope3 side, producer): - TMPXConfig grows MacroNames []string, loaded from the new TMPX_MACRO_NAMES env var (comma-separated). Empty leaves the agent on the legacy single-`tmpx` emission shape — no behavior change until an operator declares the slot names. - TMPXSealer.MacroEntry pairs a sealed token with the first registered slot name (single-slot is the only shape emitted for now; multi-chunk encoding deferred until production deployments exceed the 255-char ad-server slot budget). - handler.go populates resp.TmpxMacros[0] alongside legacy resp.Tmpx when the sealer reports a configured macro entry. Legacy `tmpx` stays populated through the 3.x deprecation window for consumers that haven't migrated. Router merge: - mergeIdentityResponses folds each agent's TmpxMacros[] into the merged TmpxProviders map keyed by provider_id, preserving per-provider attribution across fan-out. Skips entries with empty providerID (defensive against parallel-slice misalignment) and legacy-only agents (those without TmpxMacros[] don't get synthesized into tmpx_providers — the router does not invent macro names from registration metadata in this version; their token flows only through the legacy carrier). - Legacy merged.Tmpx stays populated for back-compat — sourced from the first agent's first slot when TmpxMacros[] is present, falling back to legacy resp.Tmpx for transitional legacy-only agents. Test coverage: - TestMergeIdentityResponses_TmpxProvidersFromNewShape: two providers each emit TmpxMacros[]; assert the merged TmpxProviders map carries both entries with the right names + values, and legacy Tmpx mirrors the first provider's first slot. - TestMergeIdentityResponses_LegacyOnlyAgent: a legacy-only agent (Tmpx set, no TmpxMacros) leaves TmpxProviders empty and preserves the legacy carrier. - TestMacroEntry_EmitsWhenConfigured / TestMacroEntry_NotEmittedWhenDisabled: cover the three nil/empty/no-config paths that fall back to the legacy single-`tmpx` shape. The 4.0 cleanup (drop the legacy `tmpx` field, require agents to emit TmpxMacros[], require routers to populate TmpxProviders) is out of scope for this PR — those land when AdCP 4.0 is cut. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI drift check on PR #392 caught that adcp/types_gen.go (the top-level adcp-package types, generated by adcp/schemas/generate.py) was still pinned to the 3.1.0 docstrings while the schema bundle moved to latest. The diff is realigned whitespace on GetProductsRequest / GetProductsResponse fields — a few descriptions in the upstream schemas grew enough characters to push the struct-tag alignment one column wider. No type changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hand-written types.go was missing the `governance_aware` field that the latest schema bundle adds to MediaBuyCapabilities (the sync_governance + check_governance conformance declaration). Drift caught by adcp/schemas/lint.py on the schema-bundle bump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v3.2.0 (released 2026-06-30) contains the TMPX schema changes from #5689/#5729 — what this branch was pinned at `latest` to pick up before a tagged release existed. With the release out, swap to the signed pin so `download.sh`'s Sigstore verification runs and the freshness check passes. Diff is the VERSION string, the bundle SHA, and the `AdCP schema version:` comment header on adcp/types_gen.go. Generated content otherwise unchanged — `latest` was already at the v3.2.0 cut. `tmproto/types_gen.go` regenerates with no drift. Drops the only blocker on merging this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. Additive per-provider TMPX surface that keeps the legacy tmpx carrier alive through the 3.x window — the right shape: the new map is authoritative, the old string fails back gracefully, and the HPKE token stays opaque end-to-end.
Things I checked
- Schema-vs-types coherence. Bundle bumped 3.1.0 → 3.2.0 (
adcp/schemas/VERSION,.bundle-sha256) with both generated files regenerated.MediaBuyCapabilities.GovernanceAware *boolis hand-written inadcp/types.go:87;MediaBuyCapabilitiesis inKNOWN_TYPES(adcp/schemas/generate.py:95, mapped at line 1380),*boolsatisfies the lint optional-bool-pointer rule — correct escape hatch, no drift. - No duplicate type.
TmpxProviderEntryis defined exactly once, hand-written intmproto/tmpx_providers.go:8, wired viainternal/generate/go-overlays.json(IdentityMatchResponse.tmpx_providers → map[string]TmpxProviderEntry). The generator emitsTmpxMacrobut not a competingTmpxProviderEntry— no collision, package compiles. - Router merge (
router/router.go:636-757).TmpxMacros[]folds intotmpx_providers[provider_id]with a defensive empty-providerIDskip;mergedsets onlyTmpxandTmpxProviders, never carriestmpx_macrosat the outbound root; legacy-only agents are preserved in the legacy carrier but not synthesized intotmpx_providers(router has no registered names in that path).providerIDs[i]/responses[i]are built in one loop from the sameresultsslice, so the "mis-aligned slices" worry can't actually fire — the empty-ID skip is belt-and-suspenders.append([]tmproto.TmpxMacro(nil), ...)is a clean defensive copy. - Signing untouched. No diff to
tmproto/signing.go,verify_middleware.go, JCS canonicalization, replay window, orRemoteKeyStore.tmpx_macros/tmpx_providersare response-only — never enter signing canonical bytes or dedup/cache keys. No replay or attestation surface moved. (ad-tech-protocol-expertandsecurity-reviewerboth confirmed.) - Shape change sanctioned.
tmpx_providerswentMap<provider_id,string>(#5689) →Map<provider_id,{macros:[TmpxMacro]}>; the schema'sx-status: experimentalis the contract that permits the reshape without a major bump. Go type matches the v3.2.0 schema. - Test plan. All substantive boxes checked —
go test,go vet,go generateno-drift,download.sh 3.2.0Sigstore verify,lint.py --strict. Only "CI green on the PR" is open, which is the CI box itself, not a manual check of the changed path. No validation gap.
Follow-ups (non-blocking — file as issues)
- Version surface lags the bundle.
adcp/version.go:19SupportedADCPVersions()still returns{3.0, 3.1}while the bundle andtypes_gen.goheader are now 3.2.0 — a buyer pinningadcp_version:"3.2"gets downshifted to 3.1. Harmless since 3.2 is additive, but the advertised wire support now trails the schema pin. Confirm the decoupling is intentional or land a3.2entry. - Stale ref-path fallback.
doc.gorenamed the generator-schemadirtmp→trusted-match, butinternal/generate/schema.go:195still hardcodes/schemas/tmp/as the$reffallback prefix. Latent (TMP schemas carry$id, so the fallback never fires), but reconcile it on the nexttmprotoregen. - Router doesn't intersect returned macro names against registration (
router/router.go:659-662). A provider'sTmpxMacros[].Nameis copied verbatim intotmpx_providerswithout checking it against that provider's registeredProviderRegistration.TmpxMacros.security-reviewergraded this Low: theprovider_idkey is the router-assignedp.ID(not provider-controlled), so attribution stays correct and a provider can only name its own slots. Posture tightening, not a hole.
Minor nits (non-blocking)
- Legacy mirror is first-source-wins, not first-macro-wins.
router/router.go:663-674: the doc comment says "prefer the first provider's first macro value," but in a mixed fan-out where a legacy-only agent sorts ahead of a new-shape agent,legacyTmpxtakes the earlier legacy string and the later macro value never wins. Matches the "source order" the comment also states and is only the deprecated back-compat field, but the two code paths aren't covered by a mixed-shape test —TestMergeIdentityResponses_TmpxProvidersFromNewShapeis all-new,_LegacyOnlyAgentis all-legacy. A mixed-order case would pin the contract.
Schema-bump PR that caught its own drift twice on the way through CI — types_gen.go realignment, then governance_aware — which is the lint gate doing exactly its job. Safe to merge once CI confirms lint.py --strict against the real 3.2.0 bundle.
1. Advertised wire-version surface caught up to the schema pin. With
the bundle now on v3.2.0, adcp.SupportedADCPVersions() and the
default emitted by adcp_version negotiation were still returning
{3.0, 3.1} — so a buyer pinning adcp_version:"3.2" was downshifted
to 3.1 even though the types_gen.go header advertised 3.2. Adds
ADCPProtocolVersion32, lists it in SupportedADCPVersions, and
moves DefaultADCPVersion to 3.2. Updates the version negotiation
and seller-capability tests to expect 3.2 as the highest stable
release.
2. Stale `/schemas/tmp/` $ref-path fallback in
internal/generate/schema.go updated to `/schemas/trusted-match/`
to match the upstream rename. Latent (TMP schemas carry $id, so
the fallback never fired) but worth reconciling before the next
regen surfaces it.
3. New test TestMergeIdentityResponses_MixedShapeAgents covers the
first-source-wins contract on the deprecated `Tmpx` legacy mirror
when a fan-out mixes a legacy-only agent ahead of a new-shape
agent: the legacy string wins the legacy mirror (input order), the
new-shape value still populates TmpxProviders. Pins the source-
order semantics the prior tests didn't exercise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. Additive wire change done the right way: the new per-provider TMPX carrier lands without touching the seal, the 255-byte budget, or the priority-truncation rules, and the legacy tmpx string stays populated through the 3.x window so nothing downstream breaks before 4.0.
Things I checked
- Router merge is correct on every path (
router/router.go:636-757): nil responses skipped (:645), providerID alignment guarded byi < len(providerIDs)(:652), empty-providerID entries excluded fromtmpx_providers(:659), and macros deep-copied viaappend([]tmproto.TmpxMacro(nil), resp.TmpxMacros...)(:661) so a later mutation of the source response can't alias into the merged map. - Root
tmpx_macrosis not leaked outbound. The merged response literal (router/router.go:746-752) sets onlyTmpxandTmpxProviders, neverTmpxMacros— matches the spec's "agent→router carrier only" rule. - No cross-provider attribution mixing. Keys come from the router-trusted
p.ID, never from a provider-controlled response field; a malicious provider can't spoof another'stmpx_providerskey.security-reviewer: ship, no High/Medium. - Producer path is opaque-token-only (
targeting/identityagent/handler.go:196-208,tmpx.go:239):MacroEntrywraps the already-sealed string, nil-guards receiver/empty-token/empty-macroNames, andresp.Tmpxstays unconditionally populated. Legacy emission shape is preserved whenTMPX_MACRO_NAMESis unset — no behavior change until an operator opts in. - Schema-vs-types coherence.
VERSION→3.2.0 +.bundle-sha256bump with bothtmproto/types_gen.goandadcp/types_gen.goregenerated;TmpxProviderEntryhand-written (tmproto/tmpx_providers.go) and type-overridden viainternal/generate/go-overlays.jsonfor the inline-object map value;MediaBuyCapabilities.GovernanceAwareadded to the hand-written struct (adcp/types.go:79, listed inKNOWN_TYPES).ad-tech-protocol-expert: theMap<provider_id,string>→Map<provider_id,{macros:[TmpxMacro]}>shape change from #5689→#5729 is reflected correctly. - Version surface is internally consistent.
DefaultADCPVersion3.1→3.2,SupportedADCPVersionsadds 3.2 (adcp/version.go), and the seller/negotiation tests were updated to expect 3.2 as the highest stable release. No stale 3.1 default hardcodes remain. - No signing / verify / keystore / JWKS / HPKE code touched — confirmed.
Follow-ups (non-blocking — file as issues)
legacyTmpxgoes empty on one corner (router/router.go:663-673): a response withTmpxMacrospresent butproviderID == ""and its ownTmpxempty contributes nothing to the legacy mirror, even thoughTmpxMacros[0].Valueis recoverable. Only reachable under slice misalignment (itself a bug), but the back-compat carrier silently empties. A one-line fallback or comment would close it.tmpx_providersis conformance-Requiredwhen any provider emits TMPX, but the legacy-only-synthesis gap means a legacy-only agent that emits TMPX yields an emptytmpx_providerswithtmpxpopulated. Sanctioned only byx-status: experimentalon the schema, and your deployment runs the new-shape agent — but worth confirming the v3.2.0 bundle actually carries thatx-statusannotation, since the exemption rests on it. Tracked follow-up is the right call.- Multi-slot is silently single-slot.
parseTmpxMacroNamesacceptsA,BbutMacroEntryonly ever usesmacroNames[0](tmpx.go:243). Documented as deferred, but alogger.Warnwhenlen(macroNames) > 1would save an operator who set two slots expecting two. parseTmpxMacroNamesdoesn't charset-validate slot names (targeting/identityagent/config.go:669). Operator-controlled input, so hardening not a hole — but mirroring the registered-name charset at parse time catches typos at startup instead of as a downstream schema-validation failure.
Minor nits (non-blocking)
TmpxProviderEntry.Macroshas noomitempty(tmproto/tmpx_providers.go:9). Never hit in practice (the router only builds an entry fromlen(resp.TmpxMacros) > 0), but a bareTmpxProviderEntry{}would serialize"macros":null, diverging from the generator's usual convention.- Commit type.
tmp: emit and merge…isn't a conventional-commits type, andaddress PR #392 review follow-upshas no type at all — release-please will shrug at both. The third drift-cleanup commit in the series ("realigned whitespace," "missing governance_aware field") is an honest accounting of what a schema-bundle bump drags in; the prose is fine, only the prefixes need tightening.
Safe to merge once CI validates go test ./... and the lint/freshness checks — the one unchecked box in the test plan.
…e slot Multi-chunk encoding isn't implemented — MacroEntry only fills macroNames[0] today. The wire shape supports two slots per provider, but the splitter (and the matching reassembler on the receiver) is deferred until production deployments actually exceed the 255-char single-slot budget. parseTmpxMacroNames silently accepted any number of comma-separated names, so an operator could configure "S3_TMPX_1,S3_TMPX_2" expecting two-slot emission and discover at trafficking time that only the first slot ever fills. Pull the warning into a tiny helper (warnIfMultiSlotIgnored) so it's testable without spinning up a TLS JWKS server for NewTMPXSealer. Logged once at startup naming the active slot and the ignored ones — loud enough to catch on a config audit, quiet enough to not spam. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Clean implementation of the per-provider TMPX surface — additive on the wire, with the deprecated tmpx carrier kept load-bearing through the 3.x window. Right shape: provider→router carrier (tmpx_macros[]) and router→publisher map (tmpx_providers) stay distinct, and the merge refuses to leak the root-level carrier onto the outbound response, so the publisher gets one unambiguous schema signal.
Things I checked
- Schema/types coherence. Bundle pinned 3.1.0→3.2.0 (
VERSION+.bundle-sha256both moved),tmproto/types_gen.goregenerated with the matching header, andTmpxMacro/TmpxMacros/TmpxProviders/ProviderRegistration.TmpxMacrosall picked up.TmpxProviderEntryis hand-written intmproto/tmpx_providers.gowith theIdentityMatchResponse.tmpx_providersoverride ingo-overlays.json— correct, becausetmprotois built by theinternal/generateGo generator (notadcp/schemas/generate.py'sKNOWN_TYPES), which doesn't synthesize structs for inline object schemas. Mirrors the existingIdentityToken/Artifacthand-writes.ad-tech-protocol-expert: sound-with-caveats — wire shape matches #5689/#5729, theMap<provider_id,string>→{macros:[TmpxMacro]}reshape is sanctioned byx-status: experimental. GovernanceAware.MediaBuyCapabilitiesis inKNOWN_TYPES, so the hand-add toadcp/types.go:87is the only correct path;lint.pycaught the drift. No edit to generatedadcp/types_gen.gobeyond the regen whitespace/comment realignment.- Default version move.
DefaultADCPVersion3.1→3.2 (adcp/version.go:30) is additive, not a breaking wire change — new fields areomitempty,tmpxis deprecated not removed, negotiation still honors explicit 3.0/3.1 pins and rejects cross-major. It also closes a latent downshift:types_gen.goadvertised 3.2.0 whileSupportedADCPVersions()still returned{3.0, 3.1}, so a buyer pinning3.2was silently knocked to 3.1. No!/BREAKING CHANGEmarker required. - Router merge.
mergeIdentityResponses(router/router.go:652-665) keystmpx_providersbyproviderIDs[i], which is index-aligned withresponsesby construction (router.go:287-292) and sourced from the router's own trustedProviderConfig.ID(router.go:523) — not read from the response body, so a provider can't spoof another's map key.security-reviewer: no High/Medium — no cross-attribution, no plaintext leak, no TEE pinhole widening. Theappend([]tmproto.TmpxMacro(nil), ...)is a correct defensive copy; no aliasing of the per-provider backing array. - Emission guards.
MacroEntry(tmpx.go:570) short-circuits on nil receiver / empty token / emptymacroNames;handler.go:201only setsresp.TmpxMacrosonok. No nil-deref reachable.warnIfMultiSlotIgnoredlogs slot names only — confirmed the sealed token value never reaches the log. - Test-plan honesty. Every box checked except "CI green on the PR" (expected). New-shape, legacy-only, mixed-shape merge, multi-slot warn, and the three
MacroEntrydisable paths all have unit coverage.
Follow-ups (non-blocking — file as issues)
- Release-please won't bump on these commit types.
tmp:/adcp:/identityagent:are this repo's scope-prefix convention but not conventional-commit types, so release-please (release-type: go) classifies none of them asfeat/fix— thisfeat-level surface addition lands without a version bump or CHANGELOG entry. If a minor bump is intended, retitle one commitfeat(tmproto): …. Not a block (no breaking wire change), but worth deciding before merge. - Multi-chunk deferral.
MacroEntryfills onlymacroNames[0]; the schema capstmpx_macrosat 2. The startup warn is the right transitional posture, but track a hard follow-up before any 2-slot provider registers — otherwise a token exceeding one 255-char slot truncates at trafficking time. provider_idcharset guarantee. The schema doc promisestmpx_providerskeys are safe for logs/metrics/dashboards. Confirm registration enforces the^[A-Za-z0-9_]+$charset onprovider_idso that operational-surface guarantee stays honest (IDs are operator-assigned, so not exploitable — just keeping the doc true).
Minor nits (non-blocking)
- Stale
schemas/tmpreferences. The dir rename totrusted-matchupdated the load-bearing fallback (internal/generate/schema.go:195) and thego:generateline (tmproto/doc.go), but comment/test-data references linger:handler.go:131,tmproto/validate_ladder.go:9,internal/generate/schema.go:112, and the test fixture key ininternal/generate/generate_test.go:163. The third drift-cleanup commit in this PR; worth one more pass to retire the old name everywhere. - Duplicate
provider_idsilently overwrites.router.go:660— if the same non-emptyproviderIDappears twice in a fan-out (a registry config error), the second write wins with no warning, unlike thepackage_iddedup path right below it which logs. Provider IDs are expected unique per active set, so non-blocking — a one-line comment or matching warn would close the gap.
ad-tech-protocol-expert verified wire shape against the regenerated types_gen.go descriptions and the overlay, not the raw bundle JSON (fetched via Sigstore-verified download.sh, not in-tree) — exact maxItems/x-status should be confirmed against the v3.2.0 bundle or #5689/#5729 if anything looks off post-merge.
Approving on the strength of the clean three-way expert pass plus verified schema/types coherence. Follow-ups noted above.
Summary
The identity-match wire surface gains per-provider TMPX attribution. The legacy single-
tmpx-string field stays populated through the 3.x deprecation window so downstream consumers (rtdp, ads-tracking-endpoint, etc.) can stay on their current code paths until they migrate to the new map. The 4.0 cleanup (drop the legacy field, require agents to emittmpx_macros[], require routers to populatetmpx_providers) is out of scope for this PR.Schema bundle pinned to v3.2.0 (released 2026-06-30) which carries the merged TMPX changes (adcontextprotocol/adcp #5689, #5729).
Changes
Wire types (
tmproto)/tmp→/trusted-matchto match the upstream rename (tmproto/doc.go).types_gen.gopicks up:IdentityMatchResponse.TmpxMacros []TmpxMacro(provider-emitted ordered pairs)IdentityMatchResponse.TmpxProviders map[string]TmpxProviderEntry(router-merged)TmpxMacrostruct (name/value)ProviderRegistration.TmpxMacros []string(registered macro names)tmpx_providers.additionalPropertiesdoesn't synthesize a struct in the generator, soTmpxProviderEntryis hand-written intmproto/tmpx_providers.goand the field type is overridden viago-overlays.json.MediaBuyCapabilities.GovernanceAwarefield added (schema drift caught byadcp/schemas/lint.py).Identity agent (
targeting/identityagent)TMPXConfig.MacroNames []string, loaded from new env varTMPX_MACRO_NAMES(comma-separated). Empty preserves the legacy single-tmpxemission shape — no behavior change until operators set it.TMPXSealer.macroNames+MacroEntry()produce{name, value}pairs from the sealed token using the first registered slot. Single-slot only in v1; multi-chunk encoding deferred.handler.gopopulatesresp.TmpxMacrosalongsideresp.Tmpxwhen a macro entry is available.Router (
router)mergeIdentityResponsesfolds each agent'sTmpxMacros[]into the mergedTmpxProvidersmap keyed byprovider_id. Per-provider attribution survives fan-out.Tmpxfield on the merged response remains populated — sourced from the first agent's first slot whenTmpxMacros[]is available, falling back to that agent's legacyTmpxfor transitional legacy-only agents.tmpx_providersentries from a legacy-only agent'sTmpxfield — it doesn't have the provider's registered macro names in this code path and shouldn't invent them.Tests
TestMergeIdentityResponses_TmpxProvidersFromNewShape— two-provider new-shape fan-out, asserts per-provider attribution + legacy mirror.TestMergeIdentityResponses_LegacyOnlyAgent— legacy-only agent leavesTmpxProvidersempty.TestMacroEntry_EmitsWhenConfigured/TestMacroEntry_NotEmittedWhenDisabled— covers the nil-sealer / empty-token / no-config fallbacks.Operator-visible changes
TMPX_MACRO_NAMES=S3_TMPX(or comma-separated for multi-slot deployments, though only the first slot is emitted in this version).tmpxfield — identical behavior to today.Out of scope
specification.mdx, "Size budget") covers production today, so multi-chunk encoding is deferred until a real deployment needs it.tmpx_providerssynthesis. Router could look up the agent's registeredtmpx_macrosfrom registry metadata and synthesize an entry from the legacytmpxstring, but that complicates the merge and isn't necessary for our deployment (Scope3's agent emits the new shape). Tracked as a follow-up.tmpxfield and makingtmpx_providersrequired happens at the major-version cut.Test plan
go test ./...green for all sub-modules and the root modulego vet ./...cleango fix ./...no further diffgo generate ./tmproto/...reproduces the committedtypes_gen.go(no drift)adcp/schemas/download.sh 3.2.0verifies against Sigstore signatureadcp/schemas/lint.py --strictpasses (no hand-written/schema drift)🤖 Generated with Claude Code