Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,9 +264,10 @@ not a starting point to improve on.**
rely on manual verification alone.
- **Coverage is uneven, intentionally not backfilled wholesale**: DID/
identity/credential/certification/certify (the newest code) have real unit
tests. `template/render.clj`, `model/graph.clj`, `app/trigger.clj`,
`app/contracts.clj` do not — they're exercised by the `dev/*_demo.clj`
scripts and manual curl instead. Don't assume test presence maps to code
tests. `app/trigger.clj` has a handler test for its 409 refusals only.
`template/render.clj`, `model/graph.clj` and `app/contracts.clj` have
none — they're exercised by the `dev/*_demo.clj` scripts and manual curl
instead. Don't assume test presence maps to code
age or importance.
- **One PR per coherent increment**, each independently mergeable, each with
its own "what/why/verification" description — this repo's entire history
Expand Down Expand Up @@ -354,13 +355,19 @@ not a starting point to improve on.**
Display code (`attestations-for-contract`) doesn't walk the chain at all,
so this is inert today, but a future feature that *does* walk evidence
chains should treat this as unverified territory.
- **`current-state`'s temporal semantics** (`execute.clj`) were not fully
traced this session — it's unclear from a quick read alone whether it
reads state as of real wall-clock "now" or some other point when a clause
consults `state`. Every real scenario tested so far only needed "state
after a permanent, never-revoked approval," which is insensitive to this
distinction; don't assume it's been verified for a case where state
changes and un-changes over time.
- **Late-arriving state changes are refused, not replayed.** `trigger!`
hands a clause the state as of the request's own `now` (valid time), and
refuses with `:out-of-order` (HTTP 409) any trigger whose *new state* would
land before a state change already recorded — writing it would overwrite
later state in valid time (it had been doing exactly that before the fix).
Answer-only triggers (attestations) may still be back-dated. Accepting a
late report properly means replaying every later execution in valid-time
order — possible because clauses are pure — but a replay can change answers
already acted on (payouts), so it is a design decision still open, not a
missing guard. Concurrent triggers on one contract are optimistic: the
transaction asserts the state the clause read is still the latest, and
`trigger!` re-runs the (pure) clause if it is not. See
`execute/assert-state-unchanged`.
- **`docs/walkthrough.md` promises every output came from a real run.**
Part 5 (identity, DID/VC, attestation) was checked by executing its
Clojure blocks verbatim. If you change what `certify!`, `grant-office!`,
Expand Down Expand Up @@ -419,7 +426,7 @@ identity for this email".
```sh
bin/clj -M:run test
```
Currently 45 tests / 151 assertions, 0 failures. Focus a namespace or var
Currently 53 tests / 182 assertions, 0 failures. Focus a namespace or var
with `--focus <fully.qualified.name>`.

**Lint + format + test together** (what to run before considering any change
Expand Down Expand Up @@ -462,10 +469,12 @@ bin/template-conformance
REPL path now exists in `docs/operator-runbook.md`; onboarding no longer
requires reverse-engineering the seed script, but it is still manual and
deliberately has no browser write surface.
3. **Trace `current-state`'s exact temporal read semantics** (see §6) and
add a test that actually exercises a contract whose state changes more
than once, so the "as of now vs. as of the clause's own `now`" question
has a checked answer instead of an assumed one.
3. **Design contract state beyond one snapshot** — replay of late-arriving
executions (see §6), and a data-driven statechart for multi-step contracts
(e.g. construction: inspections certified by an engineer or planning
director, progress payments, retainage). Align with Accord's proposed
`runtime@1.0.0.AgreementState` and `obligation@1.0.0.ObligationTransition`
(accordproject/models PR #200, open) rather than inventing new shapes.
4. **If a second contract template is ever added, use it to pressure-test
`MunicipalAgreement` reuse and the attestation-clause pattern** — both
are currently proven against exactly one real contract type each. A
Expand Down
1 change: 1 addition & 0 deletions docs/walkthrough.md
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,7 @@ Failures are a code to branch on and a message to read:
| `400 invalid-request` | the request is not a valid instance of its `$class` |
| `404 no-such-contract` | |
| `409 duplicate-request` | that `requestId` has already run on this contract |
| `409 out-of-order` | the request would change state before a state change already recorded (`at` is too early) |
| `422 clause-refused` | the contract declined — *Already approved by the Board* is an answer, not a failure |
| `422 no-clause` | no logic is stored for this contract's type |
| `501 runtime-unavailable` | the clause's runtime is not implemented here (today: `:typescript`) |
Expand Down
2 changes: 2 additions & 0 deletions src/com/trustblocks/app/trigger.clj
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@
:invalid-request 400
:no-such-contract 404
:duplicate-request 409
:out-of-order 409
:state-conflict 409
:no-clause 422
:clause-refused 422
:runtime-unavailable 501
Expand Down
242 changes: 183 additions & 59 deletions src/com/trustblocks/xtdb/execute.clj
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@
other way."
[dt]
(cond
(inst? dt) dt
(instance? java.time.Instant dt) dt
(inst? dt) (.toInstant ^java.util.Date dt)
(string? dt) (-> (try (java.time.OffsetDateTime/parse dt)
(catch Exception _
(java.time.OffsetDateTime/parse (str dt "Z"))))
Expand Down Expand Up @@ -135,9 +136,14 @@
;; ---------------------------------------------------------------- state

(defn current-state
"The contract's State instance as Concerto JSON, or nil before any execution."
[node contract-id & {:keys [reg]}]
(store/get-instance node contract-id :table state-table :reg reg))
"The contract's State instance as Concerto JSON, or nil before any execution.

As of wall-clock now by default. `:at` (an Instant) reads it as it stood at
that point in valid time instead -- which is what trigger! passes, so a
clause sees the state its own request took effect in, not whatever later
executions have made of it since."
[node contract-id & {:keys [reg at]}]
(store/get-instance node contract-id :table state-table :reg reg :at at))

(defn state-as-of
"The State as it stood at `instant` in valid time -- the question a lawyer
Expand Down Expand Up @@ -201,6 +207,70 @@
"WHERE contract = ? AND request_id = ?)")
[contract-id request-id]])

(defn- later-state-change
"The valid-from of this contract's earliest state change strictly after
`at`, or nil if there is none."
[node contract-id at]
(->> (xt/q node (list 'from state-table
{:for-valid-time :all-time
:bind [{:xt/id contract-id} 'xt/valid-from]}))
(map #(.toInstant ^java.time.ZonedDateTime (:xt/valid-from %)))
(filter #(.isAfter ^java.time.Instant % at))
sort
first))

(defn- out-of-order! [contract-id at later]
(throw (ex-info (str "This request takes effect before a state change already "
"recorded for this contract")
{:trustblocks/error :out-of-order
:contract contract-id
:at (str at)
:later-state-from (str later)})))

(defn- state-version
"The newest system-from among this contract's state rows, or nil before any
execution has written state. Any state write after it -- at whatever valid
time -- produces a row newer than this, so it identifies the state a clause
was handed."
[node contract-id]
(->> (xt/q node (list 'from state-table
{:for-valid-time :all-time
:bind [{:xt/id contract-id} 'xt/system-from]}))
(map #(.toInstant ^java.time.ZonedDateTime (:xt/system-from %)))
sort
last))

(defn- assert-state-unchanged
"A transaction op that aborts the whole transaction if any state has been
written for this contract since `version` (see state-version) -- the clause
ran against a state that is no longer the one on record, whether the new
write is dated before, at, or after this request. Optimistic concurrency:
trigger! runs the clause again against the new state, which is safe because
nothing was written and the clause is pure.

The order check (later-state-change, before the write) is separate and is
a refusal rather than a retry. A new state is written valid from `at`
onward with no end, so writing it behind a later change would overwrite
that change in valid time. Accepting a late report properly means
re-running every later execution in valid-time order -- possible, since
clauses are pure, but a replay can change answers already acted on (a
payout already made), which is a decision for a person, not a silent side
effect of a trigger."
[contract-id version]
(if version
[:sql (str "ASSERT NOT EXISTS (SELECT 1 FROM contract_state FOR ALL VALID_TIME "
"WHERE _id = ? AND _system_from > ?)")
[contract-id version]]
[:sql (str "ASSERT NOT EXISTS (SELECT 1 FROM contract_state FOR ALL VALID_TIME "
"WHERE _id = ?)")
[contract-id]]))

(def ^:private max-attempts
"How many times trigger! re-runs a clause whose state changed underneath it
before giving up. Each retry means another writer committed in between, so
exhausting this takes sustained contention on one contract."
5)

(defn- assert-failed? [e]
(= :xtdb/assert-failed (:xtdb.error/code (ex-data e))))

Expand Down Expand Up @@ -236,6 +306,90 @@
{:trustblocks/error :runtime-unavailable
:runtime runtime}))))

(defn- run-and-record!
"One attempt at trigger!'s clause run and write. Returns trigger!'s result,
or ::stale if another execution wrote this contract's state in between --
see assert-state-unchanged."
[node reg ctx contract-id contract src runtime request now at request-id by]
;; Asked again on every attempt, ahead of the order check: after a retry, a
;; concurrent copy of this same request may be the write that moved the
;; state, and its caller should hear "already executed", not "out of order".
(when-let [prior (execution-for-request node contract-id request-id)]
(duplicate! contract-id request-id prior))
(let [version (state-version node contract-id)
state (current-state node contract-id :reg reg :at at)
result (run-clause runtime ctx src {:data contract :request request
:state state :now now})
response (:response result)]

(when-not response
(throw (ex-info "Clause returned no :response"
{:trustblocks/error :invalid-clause-output
:contract contract-id
:returned (keys result)})))
(doseq [[what instance] [[:response response] [:state (:state result)]]
:when instance]
(when-let [errs (store/validate reg instance)]
(throw (ex-info (str "Clause produced an invalid " (name what))
{:trustblocks/error :invalid-clause-output
:$class (get instance "$class")
:errors (restate-errors
reg (get instance "$class") errs)}))))

;; A clause that only answers -- an attestation, say -- may be dated
;; before later state changes: it read the state as of its own time and
;; writes none. Only a new state has to land after every earlier one.
(when (:state result)
(when-let [later (later-state-change node contract-id at)]
(out-of-order! contract-id at later)))

(let [written
(try
(xt/execute-tx
node
(cond-> []
request-id (conj (assert-unused contract-id request-id))

:always (conj (assert-state-unchanged contract-id version))

:always
(conj [:put-docs execution-table
(cond-> {:xt/id (random-uuid)
:contract contract-id
:at now
:request (k/->xtdb request)
:response (k/->xtdb response)
:clause src}
request-id (assoc :request-id request-id)
by (assoc :by by))])

;; The new state is valid from the moment the request took effect,
;; not from when the database learned of it. That separation is the
;; whole point: a back-dated execution lands in valid time while
;; system time still records when we believed it -- provided it is
;; not behind a later state change, see assert-state-unchanged.
(:state result)
(conj [:put-docs {:into state-table :valid-from at}
(k/instance->doc (:state result)
:identity-field (mm/identity-field
reg (get (:state result) "$class")))])))
(catch Exception e
(if (assert-failed? e)
;; Two assertions can fail. A used request id is final; otherwise
;; the state moved underneath the clause, and trigger! runs it
;; again -- which also re-checks the order against the new state.
(if-let [prior (execution-for-request node contract-id request-id)]
(duplicate! contract-id request-id prior)
::stale)
(throw e))))]

(if (= ::stale written)
::stale
{:response response
:state (:state result)
:at now
:request-id request-id}))))

(defn trigger!
"Run a contract's clause against `request` and record what happened.

Expand All @@ -256,6 +410,12 @@
landed. Re-running is not harmless -- a second snow-removal visit report
would quietly pay the contractor twice.

The clause sees the state as it stood at `now`, not as it stands today, and
a trigger whose new state would land behind a state change already recorded
is refused as :out-of-order rather than overwriting it. If another execution
writes this contract's state while the clause runs, the clause is run again
against the new state -- see assert-state-unchanged.

`:by` records who submitted it. Everything else here answers what happened
and when; for a record that may be read in a dispute, who is the third
question, and the one nothing else in the row answers.
Expand All @@ -264,7 +424,12 @@
a bad request from a refusal from a broken clause without reading messages.
See com.trustblocks.app.trigger, which turns those tags into status codes."
[node contract-id request now & {:keys [reg ctx request-id by]}]
(let [reg (or reg (store/registry-from-xtdb node))
(let [;; A clause receives DateTimes as wire strings, never a platform date
;; (see com.trustblocks.clause), so an Instant or Date from a REPL is
;; normalised here. A string passes through untouched, keeping the
;; caller's own spelling of it.
now (if (string? now) now (str (->instant now)))
reg (or reg (store/registry-from-xtdb node))
ctx (or ctx (clause/context))
contract (or (store/get-instance node contract-id :reg reg)
(throw (ex-info "No such contract"
Expand Down Expand Up @@ -292,58 +457,17 @@
:errors (restate-errors
reg (get request "$class") errs)})))

(let [state (current-state node contract-id :reg reg)
result (run-clause runtime ctx src {:data contract :request request
:state state :now now})
response (:response result)]
(let [at (->instant now)]
(loop [attempt 1]
(let [outcome (run-and-record! node reg ctx contract-id contract src runtime
request now at request-id by)]
(cond
(not= ::stale outcome) outcome

(when-not response
(throw (ex-info "Clause returned no :response"
{:trustblocks/error :invalid-clause-output
:contract contract-id
:returned (keys result)})))
(doseq [[what instance] [[:response response] [:state (:state result)]]
:when instance]
(when-let [errs (store/validate reg instance)]
(throw (ex-info (str "Clause produced an invalid " (name what))
{:trustblocks/error :invalid-clause-output
:$class (get instance "$class")
:errors (restate-errors
reg (get instance "$class") errs)}))))

(try
(xt/execute-tx
node
(cond-> []
request-id (conj (assert-unused contract-id request-id))

:always
(conj [:put-docs execution-table
(cond-> {:xt/id (random-uuid)
:contract contract-id
:at now
:request (k/->xtdb request)
:response (k/->xtdb response)
:clause src}
request-id (assoc :request-id request-id)
by (assoc :by by))])

;; The new state is valid from the moment the request took effect,
;; not from when the database learned of it. That separation is the
;; whole point: a back-dated correction lands in valid time while
;; system time still records when we believed it.
(:state result)
(conj [:put-docs {:into state-table :valid-from (->instant now)}
(k/instance->doc (:state result)
:identity-field (mm/identity-field
reg (get (:state result) "$class")))])))
(catch Exception e
(if (assert-failed? e)
(duplicate! contract-id request-id
(execution-for-request node contract-id request-id))
(throw e))))

{:response response
:state (:state result)
:at now
:request-id request-id})))
(< attempt max-attempts) (recur (inc attempt))

:else
(throw (ex-info "The contract's state kept changing while this request ran"
{:trustblocks/error :state-conflict
:contract contract-id
:attempts attempt}))))))))
12 changes: 9 additions & 3 deletions src/com/trustblocks/xtdb/store.clj
Original file line number Diff line number Diff line change
Expand Up @@ -135,12 +135,18 @@

(defn get-instance
"Read an instance back out as Concerto JSON, restoring $class and the original
camelCase property names."
[node id & {:keys [table reg] :or {table contract-table}}]
camelCase property names.

`:at`, an Instant, reads the instance as it stood at that point in valid
time rather than as it stands now."
[node id & {:keys [table reg at] :or {table contract-table}}]
;; The id must be matched as a literal in the binding pattern. Writing
;; {:xt/id $id} with an :args param does NOT filter -- XTDB treats $id as
;; another column to project and returns the entire table.
(let [q (list 'from table [{:xt/id id} (symbol "*")])]
(let [bind [{:xt/id id} (symbol "*")]
q (list 'from table (if at
{:for-valid-time (list 'at at) :bind bind}
bind))]
(when-let [record (first (xt/q node q))]
;; Falls back to the stored schema rather than to ->camelCase: inferring
;; the name is lossy for anything not plain camelCase (vechileID,
Expand Down
Loading