Skip to content

feat: lawful basis builder, spec compliance check, conserver-direct delivery - #1

Merged
howethomas merged 7 commits into
mainfrom
thomashowe/con-1081-template-lawful-basis
Sep 25, 2026
Merged

howethomas merged 7 commits into
mainfrom
thomashowe/con-1081-template-lawful-basis

Conversation

@howethomas

@howethomas howethomas commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Adapters generated from this template now start with a lawful basis attachment, a schema compliance check they can copy, and the option to deliver straight to a conserver. Until now every generated adapter began non-compliant.

Changes

  • Lawful basis builder in vcon_builder.py. LawfulBasisConfig reads LAWFUL_BASIS, LAWFUL_BASIS_PURPOSE, LAWFUL_BASIS_JURISDICTION, LAWFUL_BASIS_EXPIRATION, LAWFUL_BASIS_PROOF_MECHANISM and LAWFUL_BASIS_PROOF_DESCRIPTION from the environment, a YAML vcon.lawful_basis: block, or both (resolve(), env wins per field). An invalid basis raises ValueError at load. add_lawful_basis() emits {purpose: "lawful_basis", start, party, dialog, mediatype: "application/json", encoding: "json", body: <JSON value>} and adds "lawful_basis" to extensions. Unset: one warning per process, nothing added. No basis is ever defaulted in code. vcon-lib's add_lawful_basis_attachment() is deliberately not used; 0.9.6's output is missing the required start field and mediatype.
  • finalize_vcon() removes the empty meta: {} / metadata: {} that vcon-lib 0.9.6 puts on every dialog. Both delivery classes call it before serialising, so generated adapters cannot forget it.
  • Spec compliance test tests/test_spec_compliance.py. assert_spec_compliant(vcon_dict, schema_path) validates against the official schema and checks the non-negotiables: no mimetype, every attachment has purpose/start/party/dialog, body is a string unless encoding: "json" (see the -04 section below), no empty meta/metadata/group/redacted. Written to be copied into other repos. The schema is vendored under tests/schema/ from ietf-wg-vcon/draft-ietf-vcon-vcon-core at fdcf2f5f420b726f7e70d16683ba0977d4610a1f, source recorded in tests/schema/SOURCE.md.
  • Conserver-direct delivery (delivery.mode: conserver): POST {CONSERVER_URL}/vcon with repeated ingress_lists query params (checked against the vcon-server API route) and a configurable token header, default x-conserver-api-token. Shares the webhook's retry, backoff and dead-letter code.
  • Docs. Stale build_new() docstring corrected. README marks TranscriptionProvider and JWS signing as not implemented. USAGE.md and config.example.yaml cover the lawful basis, finalize_vcon() and delivery mode.

-04

The template retargeted from draft-ietf-vcon-vcon-core-02 to -04 (published 2026-09-07); the vCon syntax parameter stays "0.4.0". The one rule that changed: with encoding: "json", body is the JSON value itself (object/array/number/bool/null), not a json.dumps() string (-04 §2.3.2, CDDL body: any). The vendored schema already permitted this and needed no change. add_lawful_basis() now writes the raw dict as body. Readers accept both shapes through a new json_body(attachment) helper, so vCons built under the old -02 reading (including this template's own earlier commits) still read back correctly. assert_spec_compliant() now also fails a body that's a string under encoding: "json", catching accidental double-encoding.

Verification

  • pytest -q in a fresh venv: 49 passed (14 on main).
  • The sample vCon, lawful basis included, validates against the vendored schema. With finalize_vcon() stubbed out, the compliance test fails, so it does not pass by construction.
  • ruff clean. mypy strict shows the same three missing-stub errors as main and nothing new.

Not in this PR

  • lawful_basis is not added to critical. Regulated adapters may want that.
  • The extension doc in vcon-docs still spells the attachment type and the top-level field must_understand; this template follows the core draft (purpose, critical).

Refs CON-1081

🤖 Generated with Claude Code

howethomas and others added 7 commits September 25, 2026 16:59
Add LawfulBasisConfig (env/YAML/merged resolution, validated at
construction) and add_lawful_basis() per draft-howe-vcon-lawful-basis.
Wire config.py to resolve vcon.lawful_basis YAML against LAWFUL_BASIS*
env vars (env wins per-field) into Config.lawful_basis. Also fix the
new_vcon() docstring, which claimed build_new() doesn't set the vcon
syntax param — vcon-lib >= 0.9.6 does.

Deliberately does not use vcon-lib's Vcon.add_lawful_basis_attachment(),
which emits a non-string attachment body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vendor the official JSON schema from ietf-wg-vcon/draft-ietf-vcon-vcon-core
(commit fdcf2f5, see tests/schema/SOURCE.md) and add
assert_spec_compliant(), a standalone, copy-pasteable validator that checks
a vCon dict against the schema plus the non-negotiables the schema alone
doesn't fully enforce (no `mimetype`, attachment purpose/start/party/dialog,
string bodies, no empty meta/metadata/group/redacted).

A sample vCon built with new_vcon()/add_lawful_basis() validates cleanly;
bare new_vcon() output also validates. Found that vcon-lib 0.9.6's
Dialog.to_dict() always emits empty meta/metadata dicts on dialogs added
via add_dialog() (a vcon-lib bug, out of scope here — worked around in the
test fixture, flagged in the CON-1081 report). Also found the vendored
schema requires `mediatype` on any attachment with a non-empty body, so
add_lawful_basis() now sets mediatype: application/json on its attachment.

Add jsonschema to dev dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add ConserverDelivery, which POSTs a vCon straight to a vcon-server
instance's /vcon endpoint (x-conserver-api-token header, configurable;
repeated ingress_lists query params). Extract the shared retry/backoff
loop and DLQ write out of WebhookDelivery into module-level helpers so
both delivery modes reuse the same machinery instead of duplicating it.

Query param name and auth header verified against vcon-server's
POST /vcon route in api/api.py (ingress_lists: Optional[List[str]],
plural/repeatable — distinct from the singular ingress_list used by the
separate /vcon/external-ingress and /vcon/ingress routes) and
CONSERVER_HEADER_NAME's default in common/settings.py.

Tested against a real aiohttp TestServer (no new dev dependency).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
config.example.yaml gets vcon.lawful_basis, delivery.mode and conserver
blocks. README drops the unimplemented TranscriptionProvider/JWS-signing
claims (marked "not implemented", not deleted outright) and lists the
new delivery modes and lawful-basis env vars. USAGE.md tells a new
adapter author to call add_lawful_basis() with LawfulBasisConfig, pick a
delivery mode, and run tests/test_spec_compliance.py.

Also add .omc/ and the scratch venv dir to .gitignore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spec-compliance test was stripping vcon-lib's empty dialog
meta/metadata inside the test fixture, which hid a real defect: every
adapter generated from this template would emit non-compliant vCons
while its own compliance test stayed green.

Add finalize_vcon(vcon_dict) as a public helper in vcon_builder.py: it
walks the whole vCon dict and removes any `meta`/`metadata` key whose
value is an empty `{}` (the shape vcon-lib 0.9.6's Dialog.__init__
always sets when neither is passed to add_dialog(), and Dialog.to_dict()
never omits). WebhookDelivery.deliver() and ConserverDelivery.deliver()
both call it before serializing, so adapters following the documented
delivery path get this for free without having to remember it.

The compliance test's sample fixture now calls finalize_vcon() directly
instead of hand-stripping keys, so it exercises the real helper. Verified
by temporarily reducing finalize_vcon() to a no-op: with the fix
disabled, test_sample_vcon_with_lawful_basis_is_spec_compliant and the
new finalize_vcon unit tests fail as expected; restored and confirmed
green again. Added unit tests for finalize_vcon() (real vcon-lib dialog
bug reproduction, nested structures, idempotence, delivery integration)
and documented it in USAGE.md next to add_lawful_basis().

`critical` stays unpopulated on the lawful-basis extension — left as a
noted gap in the CON-1081 report, not addressed here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…N body encoding

The project retargeted from -02 to -04 (published 2026-09-07). Syntax
stays "0.4.0". The one rule change: with encoding: "json", body is the
raw JSON value (object/array/number/bool/null) per -04 §2.3.2 ("The
value of the body parameter is a JSON value"; CDDL body: any) — not a
json.dumps() string. The vendored schema already allowed this ("Any type
for encoding=json, otherwise it must be a string"), so only the code and
tests that stringified JSON bodies under the old -02 reading needed to
change; the schema itself (fdcf2f5) is unchanged and matches -04.

- add_lawful_basis(): body is the dict itself, no json.dumps(). Still
  sets mediatype: "application/json" (required by the vendored schema
  whenever body is non-empty) and encoding: "json".
- New json_body(attachment) helper: returns an encoding: "json"
  attachment's/analysis's body as a Python value, accepting both the -04
  raw value and a legacy json.dumps() string (json.loads() if str), for
  compatibility with vCons built under -02 rules.
- assert_spec_compliant() (tests/test_spec_compliance.py): body must be a
  str unless encoding == "json", in which case it must NOT be a str
  (catches accidental double-encoding). Updated its docstring, the sample
  fixture's analysis body, and vcon-lib's add_lawful_basis_attachment note
  (its body being a dict is fine under -04; it's still missing the
  required `start` field and `mediatype`).
- Updated docstrings/README/USAGE/tests/schema/SOURCE.md to name -04 and
  describe the JSON-body rule; added tests for json_body() (raw value,
  legacy string, list/scalar/null bodies, wrong-encoding ValueError, a
  real add_lawful_basis() round trip) and for the double-encoding catch
  in assert_spec_compliant().

Test count: 49 passed (was 42), `pytest -q` in the repo's scratch venv.
ruff check/format clean; mypy strict on src shows the same 3 pre-existing
stub-only errors as before (no new ones).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The install step used `uv pip install --system`, which the runner's
externally managed Python rejects, so CI failed before running anything,
on main as well. Install into a venv instead. Add the yaml and aiofiles
stubs to the dev extra and ignore missing imports for vcon-lib, which
ships no types, so `mypy src/` passes under strict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@howethomas
howethomas merged commit 5f99443 into main Sep 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant