Execution and recovery contract

This document specifies the first-milestone execution boundary for configuration control on an existing cluster, as required by Specify execution and recovery. It refines the current design §§7.7, 7.8, 9.1, 12.1–12.7, 13.7, 14.4, 14.6 and 15.2 for the PoC profile (PostgreSQL with OpenBao KV v2 and Transit, design §7.7), under the owner's retention and recovery policy (§7.8) and identity and approval policy (§13.7). Each section names the design text it refines. It is a PoC contract, not evidence that any mechanism has passed design experiment E4 or E6.

The evidence it rests on, abbreviated below:

Short name	Report
DS	Approval and dispatch safety
DB	Database semantics
KL	Key loss and restoration
E3	Talos compatibility
E1	Secret ingress: extraction before persistence
FR	Feasibility evidence review
Fx	Investigation fixtures

"DS row 005" is row 005 of the dispatch-safety capture (DS §3). Every dispatch result holds for one apply-config --mode=no-reboot on one container worker running Talos v1.13.6, at one assignment revision, against one PostgreSQL 17.11 server at its default isolation, with a rollout limit of one (DS §7). This contract inherits those limits; §9 lists them with the gaps.

Where neither the design nor the evidence decides a question, this contract makes a choice consistent with the design, for a stated reason, and marks it in place as (choice §10.n); §10 lists each with its reason and alternative, and says where the choice is not the most conservative option. "Inferred" marks a claim that no row or test measured.

Two sibling contracts share the boundary. The secret ingress and compilation contract (docs/spec/compilation.md) owns ingestion, including drift adoption's extraction, the baseline, redaction and the release record. The persistence and API contracts (docs/spec/persistence-api.md) own entities, revisions, transactions, publication, the API, authentication and authorization, idempotency, the API handle of an operation, migrations and the restore epoch as a persistence mechanism. Where this contract needs something from persistence it states the requirement and says "see persistence-api.md"; it specifies no schema. The durable records it requires are: plans and their states (§2), approvals, approval and identity revocations and cancellations, operations and their owners (§3.4), the machine timeline with its revision order (§4.1), observations, attempts, responses, accounting decisions (§5.2), drift records, freezes and adoption records (§6), and recovery entries, epochs, scope states and scope releases (§7).

1. Supported operation and state values

Design: §12.1, §12.2, §18.2.

The PoC supports one operation class, the one E4 exercised: applying a published, encrypted per-machine artifact to an existing worker with Talos apply-config in no-reboot mode, over direct Talos access (DS §2.1, §6.3). The artifact's contract minor is the node's running minor (compilation contract §10.2); a v1.13 worker running v1.13.6 is the only evidenced target (E3 §4.3). Publication, observation, adoption and recovery are separate capabilities that send nothing to a machine.

Out of scope until their operation-specific safety evidence exists: control plane targets, bootstrap, reset, wipe, upgrade, reboot, auto, try and staged modes, remote transports and managed-cluster etcd recovery (design §18.2). The Upgrade/LifecycleClient transition is deferred and design experiment E3 has not passed (E3 §6.3); this contract claims no upgrade support, no lifecycle execution and no full E3. The design's bounded Ignore drift policy is not supported in the PoC (choice §10.1); §6 specifies freeze, adopt and revert.

The manager stores three distinct values per machine:

Value	Meaning
Desired	The release the application database records as selected for the machine. Selection is not authorization; approval belongs to a plan (§2).
Applied	The last verified configuration: a release, the configuration digest verified on the machine, and its source, either a completed operation (§4) or an adoption record (§6.3).
Observed	The latest machine-reported identity, assignment evidence, running Talos version, configuration digest, machine-configuration resource version and health results, with the observation's revision, time and purpose (§4.1).

The configuration digest is SHA-256 over the machine configuration read back from the node (the v1alpha1 machine-config resource's spec) with its trailing newlines replaced by exactly one, and the same function over the artifact's plaintext. This is the normalization the fixtures and E4 used; the read path adds one newline (Fx §4 check 3; DS §2.3). It is one fixed function for every comparison in this contract (choice §10.2). Planning cannot decrypt an artifact, and no runtime identity can decrypt a baseline, so compilation records the digest: over each artifact's plaintext at publication, and over the exact input it encrypts as a baseline at import and drift adoption, which ingestion reads from the node as the same v1alpha1 resource spec (compilation contract §2.3 step 8, §11). Its guessability, as a persisted digest of a secret-bearing document, was not assessed (compilation contract §4.1, choice §16.26 there).

Applied changes only when an operation enters completed, to that operation's bound release and artifact digest, or by an adoption record, to the adopted release and the digest verified by observation (§6.3). An adoption record is the approved, observation-verified baseline that design §12.1 allows as the second source of the applied release; it also establishes the first Applied of a machine that has none. An RPC response alone never updates Applied.

Two differences are kept apart:

• Pending convergence: the Desired release's artifact digest for the machine differs from the Applied digest. A plan and approval resolve it; an offline machine in this state is simply not yet converged.
• Drift: the observed digest differs from the Applied digest while no operation holds the machine's scope (§6).

A newer observation does not invalidate a plan unless it contradicts one of the plan's bound preconditions; observation freshness at dispatch is governed by §3.

2. Immutable plan and approval binding

Design: §12.2, §12.7, §13.7.

A publisher creates every plan from a published release, the adoption plan of §6.3 included (design §13.7 item 2). Planning resolves a published artifact. Dispatch never re-renders a draft or silently substitutes a newer artifact. The immutable plan binds:

• plan, release and exact per-machine artifact identities, with the artifact's configuration digest (§1);
• the release's renderer and contract record, its secret and encryption dependency records, and the artifact's Transit key by an identity the provider does not reissue on the supported topology, together with its version, not by name and version alone (compilation contract §9, §10.2; KL §7 item 1, inferred; residual risk accepted in design §7.7);
• machine identity and assignment revision. The machine identity is the machine record's identity key, its SMBIOS UUID or, for a machine inventoried without one, its Talos node ID, together with the Talos cluster ID of the cluster it records the machine as a member of (see persistence-api.md); it is a precondition as well as a postcondition (below, §3.2 comparison 3) (choice §10.26);
• the machine's Desired release at plan creation. Plan creation refuses an apply-config plan, a revert included (§6.4), whose release is not the machine's Desired, so only the selected release is ever planned; an adopt plan binds it as §6.3 states (choice §10.24);
• the machine's baseline revision, a per-machine counter advanced by every change of Applied, whether by a completed operation or by an adoption record (§6.3). A machine with no Applied has no baseline revision, so no apply-config plan can be made for it before its baseline is accepted; the adopt plan that accepts it binds none (§6.3);
• the expected pre-dispatch configuration digest. This is mandatory for every apply-config plan, because the request replaces the whole configuration. Plan creation refuses a plan that binds no drift record unless this digest equals the digest of Applied at the bound baseline revision; a revert expects the drifted digest its drift record names instead (§6.4). Only a revert therefore expects a configuration that Applied does not describe, and any other such configuration that dispatch finds is drift (§3.2) (choice §10.23);
• the open drift record, if the machine has one; such an apply-config plan is a revert (§6.4), and such an adopt plan adopts the drifted configuration (§6.3, choice §10.38);
• operation apply-config, mode no-reboot, the transport route (§3.5) and every other parameter value that reaches the Talos request;
• expected preconditions: the machine identity, the node's running Talos minor, the maximum age of the execution-time observation, the validity window of the use-time dependency check (§3.1), and any health or capacity check;
• expected postconditions: the machine identity, the assignment revision and the artifact's configuration digest, plus any bound health check;
• the transport deadline and the verification deadline, as durations from the attempt, the first no later than the second, and the maximum number of attempts;
• rollout scope and limit, expiry, idempotency key and the approval policy (design §13.7: one approval by an approver);
• the creating identity and role.

The plan also carries plan-time evidence: the redacted diff, the upstream validation result and any available dry-run evidence, all derived and redacted under the compilation contract (§8.3 there). A diff is of whole configurations, not only of the intended patch: E3 saw a one-label patch applied as a full re-encode, the worker's configuration read shrinking from 428 lines to 55, with what else changed unrecorded (E3 §7). Plan-time evidence informs review and approval. It never substitutes for the execution-time checks against live state in §3, and the timeline records the two separately.

The plan is invalid if any bound value changes, including the artifact, assignment, Desired release, baseline revision, operation, mode, route, a parameter value, a precondition, expiry or approval policy. The controller must replan and obtain approval again rather than repair the plan in place.

Plan states. Until dispatch commitment the plan is the handle for its change, and no operation exists for it. The commitment transaction (§3.2) creates the plan's one operation and links the plan to it; from then on the operation's states (§4) carry the outcome (choice §10.3).

State	Meaning	Leaves by
proposed	The immutable plan exists but is not authorized.	an approval, to approved; a cancellation, to cancelled; its expiry, to expired
approved	An approval matches the complete binding.	the §3.2 transaction, to committed; revocation of the approval or of the approving identity, to revoked; a cancellation, to cancelled; its expiry, to expired
committed	Terminal for the plan. Its operation exists and holds the outcome.	none
revoked, cancelled, expired	Terminal. No operation was created and nothing was sent (DS row 002, where the prototype reports the outcome as cancelled).	none

An approval whose epoch is not the current recovery epoch authorizes nothing (§7.1); an unexpired approved plan whose approval carries another epoch cannot commit until an approver approves it again in the current epoch. Revocation, cancellation and expiry after commitment are recorded against the plan and reach its operation through §3.3.

Approval. One approval by an identity holding approver authorizes a plan; automation never holds approver, and recovery-admin carries no approval (design §13.7 items 2 and 3). The approval names the plan. A plan is immutable, so the plan it names is the whole binding and no revision number enters it (persistence and API §4.1); across a restore, the epoch (§7.1) is what keeps an earlier approval from authorizing anything. It records the approving identity, the role it acted under and the recovery epoch in which it was recorded (§7.1), and is marked self-approval where design §13.7 item 3 requires. Of the two recording cases §13.7 leaves to this contract, both are marked self-approval (choice §10.4):

• the approving identity authored any revision the release contains, whether of a fragment, a profile, an assignment or an import base, however long ago and whether or not this release changed it;
• the approval is by the human whom the token-issuing record names as responsible for an automation token that authored or published content in the release, or created the plan. The token record is persistence's, which requires a responsible human for every token (see persistence-api.md). As a guard only, an approval for which no such record can be found is recorded as "self-approval undetermined", never as not self-approval.

Approval authorizes exactly this binding. Publication, a retained dependency verdict and access to a Transit key never authorize dispatch (design §7.4 step 5, §7.8). The trusted controller enforces authorization; the provider does not (design §12.7, §13.2).

Revocation and cancellation. Any approver or recovery-admin may revoke an approval; recovery-admin records an identity revocation, which invalidates every approval that identity gave and that no attempt has used yet, including approvals already used by a commitment (design §13.7 item 4). This contract also lets it refuse every later attempt of an operation that identity approved, retries included (§3.3, choice §10.6). A plan may be cancelled by its creator, any approver or recovery-admin (item 6). What each reaches is §3.3 and §8.

An approval revocation locks the approval it revokes before recording the revocation (on PostgreSQL, SELECT … FOR UPDATE), so it waits for a commitment or attempt transaction that read the approval FOR SHARE (§3.2 comparison 1; DS row 003). An identity revocation likewise locks the revoked identity's principal record, which those transactions read FOR SHARE. The records are persistence's; see persistence-api.md.

3. Dispatch commitment

Design: §12.7, §12.5, §12.3, §7.4.

No role executes. The controller dispatches an approved plan through the transactions below (design §13.7 item 2). The database and the external systems do not share a transaction, so commitment has two phases with different guarantees, and a residual window that this contract states rather than hides.

E4 placed the dispatch commitment boundary at the COMMIT of the commitment transaction, and showed each comparison that its controls removed letting through the failure it exists to prevent (DS §6.1, §8).

3.1 Execution-time evidence, gathered before the transaction

Before touching any protected resource, the controller checks that the plan is currently approved, unrevoked and unexpired and that the scope gate is open (comparisons 1 and 6 of §3.2). Artifact decryption and the client that sends mutating requests are gated by that application check; a plan whose approval was revoked while it was queued gets neither. Observations, this one included, use the read-only observation client (§3.5), which needs no approval. The check is not atomic with what follows, which is why §3.2 repeats it.

The controller then gathers and durably records, on the machine's timeline (§4.1) as dispatch evidence for this plan, naming the plan and the controller instance (the operation does not exist yet; §2):

1. a fresh machine observation taken for this dispatch (purpose evidence): identity, assignment, running Talos version, configuration digest, machine-configuration resource version, and the health and capacity evidence the plan's preconditions name; and
2. a use-time check, with its time, of the dependencies dispatch itself needs, under the executor identity: decryption of the bound artifact under the bound key identity and version, and the operation credentials for the target: its cluster's Talos access configuration, read once at its latest version before item 1's observation, which uses it as the attempt made on that evidence does (a retry's new evidence reads it again, §3.3), whose path, version and created_time the evidence records, never the value (persistence and API §3.3).

Item 2 is deliberately narrow. The executor holds scoped artifact decryption and operation credentials, not compiler-level secret access (compilation contract §1). Applying a retained artifact needs its release record, the key version it names at or above the decryption floor, an executor credential the provider recognises and an unsealed provider; losing a source secret version blocks regeneration, not application (design §7.8, "Guarantees kept apart"; KL §5.2). A retained monitor verdict is not this check (design §7.6; dependency monitor §8). A sealed OpenBao fails it, and nothing is dispatched until the operator unseals (design §7.8 item 4).

A stored observation older than the plan's maximum age never satisfies item 1, whatever it says. A machine that cannot be observed cannot be dispatched to. A node whose running minor differs from the artifact's contract minor fails the precondition: it is outside the proven path (§1).

3.2 The commitment transaction

The transaction that creates the durable dispatch intent compares database facts only, atomically, each read under a lock that a concurrent change must wait for or precede. On PostgreSQL a plain read does not stop a concurrent change before commit (DB §4.2 rows 010 and 011); design §7.7's consequences require such locks, each with a test that can fail. The transaction first locks the machine's row, the lock under which §4.1 allocates timeline revisions (on PostgreSQL, FOR UPDATE), and reads the other records named below FOR SHARE; the lock order is persistence's (see persistence-api.md).

1. no operation exists for this plan yet: the transaction creates it, so a second executor for the same plan is refused (DS row 009; §2, choice §10.3);
2. the plan is approved (in an attempt transaction, committed to this operation), unexpired and has no recorded cancellation; the approval is not revoked; the approving identity has no identity revocation; and the approval was recorded in the current recovery epoch (§7.1). The approval, the approving identity's principal record and the installation state that holds the current epoch are each read so that a concurrent revocation or recovery-mode entry waits for this transaction or precedes it (on PostgreSQL, FOR SHARE; DS row 003 for the approval);
3. the artifact, assignment, operation, mode, route and parameter bindings are unchanged, with the machine's assignment head read FOR SHARE, and the machine's baseline revision equals the bound one, so a plan made before an adoption or another operation's completion, or for a machine whose baseline was never accepted, cannot commit (DS row 010). The machine's Desired release is the bound one, read under the machine row's lock, which a publication's selection also takes (see persistence-api.md), so a plan approved before another release was published cannot apply the release the machine no longer desires (choice §10.24; no DS row exercised it). This term is compared at commitment only, not by attempt transactions (§3.3);
4. the evidence recorded under §3.1 was recorded for this plan by this controller instance, satisfies the plan's preconditions, is inside its bound maximum age or validity window, and no newer observation of the machine contradicts it (choice §10.28). The observed identity must match the bound one under the machine's identity key: the bound SMBIOS UUID, or, for a machine recorded by Talos node ID, that node ID and no SMBIOS UUID; and the node's Talos cluster ID must equal the bound cluster's. A node reached at the target's address that is not the planned machine is thus refused before any send (choice §10.26). The observed configuration digest must equal the bound pre-dispatch digest; on a retry, where the evidence is recorded for the operation, it may instead equal the bound artifact's digest (DS row 023, TestEvidenceMayShowArtifactOnlyOnRetry). A commitment whose evidence shows another digest is refused, and §6.1 is applied to that evidence (below). The latest use-time check must have read the Talos access version the evidence read (path, version and created_time, §3.1 item 2): an attempt made again after its attempt transaction failed to commit reads the access again, and one that moved since the evidence is refused, so the operation is observed and classified again (§5) with the new version and never sends on evidence another version read;
5. this operation takes the machine's coordination scope, and no other operation on that scope is committed, sending, verifying or unresolved (on PostgreSQL, a partial unique index; DS row 010, control row 011; DB §4.3);
6. this operation takes a slot in the plan's rollout scope, counting every operation of that scope in committed, sending, verifying or unresolved against the bound rollout limit. The machine row's lock does not cover other machines' operations, so at the PoC limit of one the slot is a partial unique index on the rollout scope over those states, as comparison 4's is on the machine scope, and the commitment waits for the scope's lock before it reads its time (choice §10.29); a limit above one needs a counted lock, which is open (choice §10.5); and
7. the scope gate is open: the machine scope is not frozen (§6.2), no drift record is open on the machine other than one the plan binds, a drift record the plan binds is still open (§6.4; a revert whose record has closed is stale), and either recovery mode is not in effect or the scope was explicitly released in the current recovery epoch (§7.3 step 7). The freeze, the open drift record and the scope's recovery state are read under the machine row's lock, and recovery mode with the installation state of comparison 1.

The commitment transaction also creates the operation in committed, links the plan and its §3.1 evidence to it, and makes the committing controller instance the operation's owner at generation 1 in the current epoch (§3.4).

If any comparison fails, nothing is committed and nothing is sent; the refusal is recorded afterwards in its own transaction (§4.1). When comparison 3 refuses a commitment because its evidence reports a configuration digest other than the bound pre-dispatch digest (a plan expects Applied's digest, a revert its drift record's; §2), the transaction that records the refusal also applies §6.1 and §6.5 to that evidence observation as to a drift observation, under the same basis and scope rules. The refusal does not by itself establish drift: a digest that differs from Applied opens a drift record, or is recorded on the one already open; a digest equal to Applied with a record open closes it as returned (§6.5), which is how a revert refused after a manual return ends; and an observation those rules exclude, such as one read while another operation held the scope, changes no drift record (choice §10.23). Freeze, recovery mode and scope release are durable database facts precisely so that they can be compared here; checking them only before the transaction would leave a race in which dispatch proceeds while mutation is meant to be paused.

The machine scope also protects the binding it was taken for. Any transaction that changes a machine's assignment checks that machine's coordination scope and is refused while an operation on it is committed, sending, verifying or unresolved, and, in recovery mode, while the scope has not been released in the current recovery epoch (§7.3 step 7): an attempt the restored state does not hold may still be in flight for the old assignment. Detecting a changed assignment revision only during verification would be too late: the artifact for the old revision would already have reached the machine.

The PoC rollout limit is one (choice §10.5). E4 exercised it only on a rollout scope of one machine, where the machine scope stood for comparison 5 (DS §2.1); comparison 5's index on the rollout scope is not evidenced. Drain is not required for a no-reboot apply at the limit of one (inferred: design §12.3 names drain with worker concurrency, and no E4 row drained); the cluster health and capacity gate is a bound precondition checked under comparison 3.

3.3 After commitment

Every attempt, the first included, is recorded by an attempt transaction before its request is sent; the timeline therefore shows the commitment and the attempt before any Talos request. The attempt transaction takes the same locks as §3.2, repeats its comparisons 1–3 and 6, against newly gathered §3.1 evidence recorded for the operation when it is a retry, and adds, in the prototype's numbering (DS §2.1), comparisons 7 to 9 below. It does not repeat comparison 2's Desired term: a publication after the commitment does not stop the committed operation, which applies its bound release and leaves the machine pending convergence towards the new one (§1). The added comparisons:

1. the recording controller is the operation's current owner at the current generation and epoch, and the operation is committed, or unresolved with a safe-to-retry classification made by that owner (§5). The check is made inside the UPDATE that records the attempt (design §7.7, consequences), not by a read followed by an insert: a read-then-insert recorded a stale attempt on PostgreSQL (DB §4.4 row 018; DS rows 007 and 008);
2. on a retry, no attempt response, ownership transition or reclassification was recorded on the timeline after the revision at which the retry was classified, so a late acceptance of the earlier attempt forces reclassification instead of a duplicate request (DS row 023, TestRetryRefusedAfterNewerTimelineFact); and
3. the operation has attempts left under the bound maximum.

The transaction records the attempt identity, the owner token, the route, and the absolute transport and verification deadlines derived from the plan. An owner that lost ownership before its attempt transaction records no attempt and sends nothing (DS row 007). The first attempt transaction may be the commitment transaction itself. Reading the approval or the ownership and recording the attempt later is not sufficient. The attempt's time is read after comparison 7's write (choice §10.30).

The controller sends only after its attempt transaction has committed. E4 established this for its own executor only: its one send follows a successful attempt transaction on every path (DS §6.2). An implementation must show the same property for itself (§9.2).

Every request carries a transport deadline no later than its recorded verification deadline. A transport deadline ends the client's wait; it does not bound when the request can land. A request sent through the control plane's API proxy landed about 11 s after the partition healed, after its client had been killed at its deadline (DS §4.4, capture 1 row 012), and late landings occurred in three of six control-plane partitions across both captures and the reproduction (FR §7). §5.2 states what accounts for such an attempt.

Revocation, cancellation and a closed gate after commitment. Before the commitment transaction commits, a revocation prevents dispatch (DS row 002), and so does a cancellation or expiry, which fail the same comparison (no row cancelled a plan). After it, the revocation is recorded on the timeline and fails comparison 1 of every attempt transaction that has not yet committed, so it also permits no retry (DS rows 003 and 004). An operation with no recorded attempt then becomes unresolved, with its scope held, and then cancelled (§4). An attempt already recorded is sent anyway and runs to its own classification (DS row 005); after it, revocation undoes nothing, and only a new approved plan, such as a revert (§6.4), or recovery changes the machine (design §13.7 item 4). Freezing the scope or entering recovery mode closes the scope gate through comparison 6, which likewise permits no further attempt, and an operation with no recorded attempt becomes unresolved with its scope held. A closed gate can reopen, so it is not grounds for cancelled: the operation stays unresolved until the gate reopens and it is classified under §5, or recovery-admin resolves it (§4).

An identity revocation has the same effect. Design §13.7 item 4 makes it invalidate every approval the identity gave that no attempt has used; this contract also makes comparison 1 refuse every later attempt transaction of an operation that identity approved, retries included, as an approval revocation does (choice §10.6). An attempt already recorded is sent anyway; if its outcome is then unknown, the operation is unresolved and ends only through accounting (§5.2) and a completion observation, as completed or failed (§5). Identity revocation was not measured (design §13.7, evidence and its limit).

The residual window. The machine or a dependency can change after §3.1 and before the request arrives. Some such changes surface: as a failure before any attempt is recorded, as a definitive rejection, or as a postcondition mismatch, each classified under §5. Not all do. apply-config sends a full configuration, so an out-of-band change that lands inside the window, such as an emergency talosctl edit, can be overwritten by a request that then verifies and completes normally, leaving no trace in this operation's evidence. The bound maximum observation age narrows this window; it does not close it, and this contract claims no detection of such a change. Closing it needs a compare-and-apply precondition on the Talos side; whether apply-config offers one was not checked (DS §8), and none is assumed.

3.4 Ownership: the takeover fence

Design: §12.5.

The ownership mechanism is the takeover fence that E4 recommends (DS §8). An operation's owner is a controller instance with a generation and the epoch it was issued in. A takeover, in one conditional UPDATE on the owner's generation, sets a new owner, increments the generation and moves the operation from committed, sending or verifying to unresolved; an unresolved operation stays unresolved under its new owner. The attempt transaction's comparison 7 checks owner, generation, epoch and state together. E4's control row 008, without those comparisons, let a stale executor record an attempt and send; the rows do not show that either half alone suffices (DS §4.2), so both are required. Concurrent takeovers of one generation elect exactly one winner (DB §4.4 row 015, PostgreSQL).

The controller takes over, at its start, every non-terminal operation owned by another instance, and recovery-admin may request a takeover explicitly; no timer or lease takes an operation over (choice §10.7). A takeover sends nothing and never classifies an outcome: it only stops the old owner from recording further attempts.

The fence bounds only work not yet recorded. It cannot stop a request already sent, or one sent by anything that bypasses the database, direct talosctl included. A database fence is not a Talos fence (DS §7; design §12.5). A restore rewinds generations, so the fence alone does not survive one (§7.1).

Alternatives E4 names and neither built nor measured: an expiring lease, an advisory lock held for the executor's lifetime, and a Talos-side compare-and-apply (DS §8).

3.5 Talos client and route

The executor's Talos client is selected here (compilation contract §10.1). It is the Talos Go machinery client inside the controller process, so that the decrypted artifact stays in the memory of the process that decrypted it (choice §10.8). E3 saw the same RPC outcomes through the machinery and through talosctl for the version read, configuration read, no-reboot dry run and label-patch apply (E3 §4.3, §6.2). Every E4 dispatch row used talosctl with --file <artifact> (DS §2.1), so the selection holds on a condition: DS rows 001–005, 012–017 and 022 are re-run through the machinery client and reach the same outcomes before this contract's dispatch is accepted. They run over the worker's own endpoint, the only route allowed (below). DS sent every row but 015 through the control plane (DS §2.1), so rows 012–014, whose faults held a request there, run as direct-route variants of the same fault. Rows 012 and 014 must reach row 015's outcome: unknown at the transport deadline, the operation unresolved until accounted (§5.2), then classified from its observation. Row 013 keeps its control: its accounting is attempted at the executor's exit, as DS recorded it, and must be refused (§5.2; DS §4.4), then succeeds after the settle floor as row 015's. The proxy's own late landings are §9.3 item 2's measurement, not these rows. The fallback is the pinned talosctl subprocess with the artifact passed through an inherited descriptor, never a named file or an argument, as for the compiler's fallback. That is not E4's channel either, so the fallback carries the same condition: the same rows re-run through it, and compilation §15's leak scan applied to its descriptor channel over successful, rejected and interrupted dispatches. If either fails there too, dispatch is not accepted and the selection returns to the owner. The machinery client met the condition (AP S4's precondition), so the fallback is not built. Two of the condition's rows do not run through the controller, as choice §10.32 records: row 003's revocation refuses the attempt before any mutating Talos request, so its outcome, no apply sent, does not depend on the dispatch client. When the attempt takes the machine row first (choice §10.31), the outcome is row 005's, which the controller runs. Row 003's Go check uses a Talos stand-in, for the evidence observation's reads too (the live rows take those reads from the worker); row 022's invalid configuration is sent by the machinery client to the worker's own endpoint directly, since no plan carries one.

Both clients are built from the cluster's one talosconfig, which carries os:admin (persistence and API §3.3). Every observation (§4.1), whether taken for a plan or not, goes through an observation client whose source is held to an allowlist of read calls by test, as ingestion's is; drift, restoration and recovery observations have no plan and need no approval. Only dispatch uses the client that sends a mutating request, and only after §3.1's approval check (design §13.1; owner decision, 2 October 2026).

The plan binds the route, and in the PoC the only route allowed is the target machine's own endpoint, the Talos endpoint on its machine record at the plan's creation (persistence and API §3.3); a plan naming the control plane's endpoint with the worker as target node is refused at plan creation (choice §10.9). E4 exercised both (DS §2.1, row 015), and every late landing came through the control plane's proxy; the one row sent to the worker's endpoint did not land late. The proxied route stays refused until the §9.3 item 2 measurement states its late-landing distribution, and it is re-admitted only by an amendment of this choice. One row is thin evidence, and whether a dial failure at the worker's endpoint proves no send was not tested (DS §4.4), so the route changes no accounting rule (§5.2).

A Talos response or error can embed the configuration or its diff (E3 §5.4, §7). The controller records response text on the timeline only through the compilation contract's redaction (§8.3 there), which withholds configuration-bearing output it cannot positively recognise, failing closed. The response's class (gRPC code, or transport outcome) is always recorded.

4. Operation timeline and states

Design: §12.5, §15.2.

An operation exists from its plan's dispatch commitment (§2, §3.2), except an adopt operation: the adoption record of an adopt plan creates it directly in completed (§6.3). It holds no scope or rollout slot and has no attempt, and its completed rests on the adoption record's observation, not on a completion observation of a sent artifact; none of the rows below after the first two applies to it. An operation's one durable timeline links, before its own entries, the plan and approval, plan-time evidence and the execution-time evidence the commitment relied on; it then holds ownership transitions, dispatch commitment, request attempts, responses, reconnects, revocations and cancellations, accounting, postconditions and final classification. State changes are append-only facts with a current projection. §4.1 specifies the entries.

State	Meaning
committed	The commitment transaction created the operation; the machine scope and a rollout slot are held.
sending	An attempt is recorded; the request is in flight or its outcome is unknown.
verifying	The request was accepted; the manager is collecting identity, digest and health evidence.
completed	Terminal. For an apply-config operation: at least one attempt was recorded, and postconditions prove the bound artifact is applied and healthy; Applied is updated. For an adopt operation, which has no attempt: the adoption record is committed (§6.3).
rejected	Terminal. Every recorded attempt has a recorded, definitive Talos response of a class proven to precede any mutation (§5.1).
failed	Terminal. At least one attempt was recorded, every attempt is accounted for, and a completion observation contradicts the postconditions, whether a request changed the machine wrongly or no request took effect. Applied is not updated.
cancelled	Terminal. No attempt is recorded for the operation; after a restore, none in the restored journal, which can omit an attempt recorded and sent after the snapshot (§7.3 step 5). This is never an undo of remote work, and never proof that nothing was sent.
unresolved	Evidence cannot yet establish a terminal state or safe retry. Scope and slot stay held.

Before commitment there is no operation: a plan that is revoked, cancelled or expires first ends in the plan state of that name (§2; DS row 002).

From	To	Condition	Evidence (§9.1)
(none)	committed	The §3.2 transaction succeeds for an approved plan.	DS rows 001, 009, 010
(none)	completed	The adoption record of an adopt plan (§6.3); such an operation takes no other state.	none
committed	sending	The §3.3 attempt transaction succeeds; the controller then sends.	DS rows 001, 007
committed	unresolved	Takeover (§3.4); or, after commitment, a revocation, cancellation, the plan's expiry or a closed scope gate; or any other refusal of the attempt transaction by comparison 1, 2, 3 or 6.	DS rows 003, 004, 007, 019
sending	verifying	An acceptance response is recorded.	DS rows 001, 016
sending	rejected	A definitive pre-mutation rejection is recorded for this attempt, and every earlier attempt has one too.	DS row 022
sending	unresolved	Lost response, transport failure or timeout, or takeover; a recorded response of any other class; or a definitive rejection while an earlier attempt has none, which accounts for this attempt only.	DS rows 012–015, 020
verifying	completed	Postconditions established by a completion observation, and every recorded attempt accounted for.	DS rows 001, 017
verifying	failed	A completion observation contradicts the postconditions (a bound health check only at or after the verification deadline, below), and every recorded attempt is accounted for.	none
verifying	unresolved	Postconditions neither established nor contradicted by the recorded verification deadline, an attempt is not accounted for, or takeover.	DS row 021 (takeover)
unresolved	completed	Postconditions later established by a completion observation, at least one attempt recorded, and every recorded attempt accounted for.	DS rows 020, 021; capture 1 row 012
unresolved	failed	A completion observation contradicts the postconditions (a bound health check only at or after the verification deadline, below), at least one attempt is recorded, every recorded attempt is accounted for, and the operation is not classified safe to retry (§5).	DS row 013, on untrue accounting
unresolved	rejected	A delayed definitive pre-mutation rejection is recorded, and with it every recorded attempt has one. An attempt that was accepted, or whose outcome is unknown, rules rejected out.	none
unresolved	sending	Classified safe to retry (§5) and the §3.3 attempt transaction succeeds.	DS rows 012, 014, 015, 019
unresolved	cancelled	No attempt is recorded for this operation (after a restore, in the restored journal; §7.3 step 5), and either none can (the approval or its identity is revoked, the approval's epoch is not the current one, the plan expired or is cancelled) or recovery-admin resolves it so on a recorded reason.	DS rows 003, 004

No other transition is valid. A takeover of an unresolved operation changes its owner and leaves its state. Recovery-mode entry takes over every non-terminal operation, each of which is owned, so it reaches unresolved through the takeover rows (§7.2). The machine scope and rollout slot are released only on entering a terminal state.

The controller makes each transition that evidence determines. Where evidence cannot, only recovery-admin decides: it records that an attempt is accounted for (§5.2) and resolves an unresolved operation, only into one of the listed targets, recorded with the deciding identity, its role and the evidence relied on (design §13.7 item 5 and its derived note). No other role does either.

A completion observation (purpose completion) is taken after every recorded attempt of the operation is accounted for, and recorded on its timeline. It records the machine identity, assignment revision, running Talos version, configuration digest, machine-configuration resource version and the bound health results. Values matching the bound postconditions establish them and yield completed; a value that contradicts them, such as another digest, yields failed; an observation that could not read a value does neither. Nor does a completion observation yield either when a recorded observation of the machine with a higher basis (§4.1), of any purpose, reports for any bound postcondition (the machine identity, the assignment revision, the configuration digest or a bound health check) a value that differs from the completion observation's: the machine may have changed after the completion read began, and a later completion observation decides. A higher-basis read that agrees with the completion observation blocks nothing, so repeated reads confirming a failed postcondition let the operation fail. Only a recorded observation blocks: a higher-basis read that has started but not yet recorded its result does not, and its result, recorded after the Applied change, opens no drift record (§6.1). A digest change that persists is detected by the next drift observation, not prevented; a late contradiction of the identity, the assignment revision or a bound health check with an unchanged digest reopens nothing (§9.3).

A bound health check has a convergence window: a failed health result in a completion observation taken before the recorded verification deadline is recorded and yields neither, the controller takes further completion observations, and the first that establishes every postcondition yields completed; a completion observation taken at or after the deadline that still reports it failed yields failed (choice §10.25). The machine identity, assignment revision and configuration digest have no window: once every attempt is accounted for, no request of the operation can still change them. The transaction that records failed also opens a drift record when the completion observation's digest differs from the Applied digest, or records the observation on the one already open, because releasing the scope makes that difference drift (§6.1). An observation ordered before the last accounting, including one ordered after the attempt record but before the attempt's request settled, or one not tied to this operation, never completes or fails it: a stalled executor may send after it, and the apply may change or degrade the machine. "Ordered" is by timeline revision under §4.1's ordering rule. A timeout is not proof of failure, completion or retry permission.

An observation ordered after an attempt does not show that the attempt's request has executed or can no longer execute. The scope is therefore released only when every recorded attempt is accounted for, meaning one of:

• a response from the target is recorded on the timeline (DS rows 016, 022); a recorded transport outcome is not one (§5.1); or
• recovery-admin has recorded an accounting decision under §5.2.

An executor's exit, a killed client or a transport timeout is not accounting (DS §4.4, §8). A matching digest is not accounting either, even when the digest differed before the attempt: direct Talos access can apply the same artifact independently while the attempt's request is merely held. An operation with an unaccounted attempt stays unresolved with its scope held, however well the machine's state matches, because releasing the scope would let a newer operation be overwritten by the late request (DS row 011). An apply-config operation with no recorded attempt never completes, even when a completion observation shows its artifact: it sent nothing that could have put the artifact there. It stays unresolved until it is classified safe to retry (§5) or ends cancelled (the unresolved → cancelled row above). A drift or evidence observation recorded while the scope was held opens no drift record (§6.1), and cancellation does not re-evaluate it: if the artifact's digest differs from Applied, the first observation after the scope is released opens the drift record. After a restore, the scope's marking evaluates its restoration observation instead (§6.1, §7.3 step 6).

4.1 Timeline content

Design: §15.2, §13.6.

Every entry records its kind, time (database clock), the acting identity and role, or the controller instance, owner generation and epoch, and a revision. Required entries:

Entry	Content
Plan	the binding of §2, the plan-time evidence reference, creator and role
Approval	the plan, approver, role, epoch, self-approval mark (§2)
Revocation, identity revocation, cancellation	what it names, who, role
Observation started	purpose, the plan or operation it is taken for, if any, and the Talos endpoint it dials: that plan's route when taken for a plan or its operation, the machine's current endpoint otherwise (persistence and API §3.3); recorded before the remote read, its revision is the observation's basis (below)
Observation	purpose (evidence, completion, recovery, drift, restoration, publication), basis, the Talos access version it read (path, version, created_time; persistence and API §3.3), identity, assignment evidence, running Talos version, configuration digest, machine-configuration resource version, health results, or which values could not be read and why: a failed access read or connection reads none, and records its talos-access-unavailable cause
Use-time check	each dependency checked, its result, under which identity
Commitment	the operation created, the §3.1 evidence it links, owner, generation, epoch, comparisons passed
Refusal	the transaction and the comparison that failed, by number; recorded after the refused transaction rolls back, by a separate transaction that allocates its revision like any entry (below) and, for a commitment refused on its evidence's configuration digest, applies §6.1 and §6.5 to that evidence (§3.2)
Attempt	attempt id, owner token, route, transport and verification deadlines; for a retry, the classification revision it is bound to, which admits that one attempt and whose observation the retry rests on
Response	attempt id, class (acceptance, gRPC code, transport outcome), redacted text or the withheld notice (§3.5)
Ownership transition	the operation, from and to owner, the generation and epoch it moved from and to, reason
Accounting	attempt id, basis (a response, or a §5.2 decision with its start and end observations and its recorded facts), decider and role
Classification	the operation, the outcome of §5 that ends the operation or admits a retry (completed, rejected, failed, safe to retry), the observation it rests on (none for a rejection, which rests on responses) and the entries relied on, by revision (the observation's entry and every response and accounting entry of the operation's attempts); an unresolved or cancelled operation names its cause on its state entry
Applied change	from, to, digest, source (operation or adoption record)
Endpoint change	previous and new Talos endpoint, who, role (persistence and API §3.3)

Plan entries, from creation to commitment or a terminal plan state (§2), and machine-scope facts are recorded on the machine's timeline beside its operations' entries: drift records and their resolution, freezes and unfreezes, adoption records, recovery-mode entry, scope states and scope release (§6, §7), and Talos endpoint changes.

A publication start names a publish operation and its publicationGeneration, with no plan, and dials the current inventory endpoint. Its ownership fences, required identity/version evidence and immutable release linkage are defined in persistence and API §6.1.1. It never authorizes mutation and does not update Applied or perform drift classification. The observation API returns publicationGeneration as null for the other purposes.

Ordering. Every entry's revision, for a plan, an operation or a machine-scope fact alike, is allocated from one per-machine counter under the machine row's lock (on PostgreSQL, the row locked FOR UPDATE), so that revision order within a scope is commit order. "After" in this contract means a higher revision in that order. E4's prototype took an observation's basis as the highest committed revision read without the lock, which is unsound under concurrent writers: an accounting could commit after an observation that read before it (DS §7). An observation is ordered by when its remote read began, not by when it is recorded: before the read starts, the controller records an "observation started" entry, whose revision is the observation's basis, and the observation is recorded with that basis. For an observation, "after X" means a basis higher than X's revision, so an observation whose read began before an accounting and was recorded after it counts as before that accounting. Per-operation allocation would not order a drift observation against an adoption approval or a drift record, which §6.3 needs. The counter and its schema are persistence's; see persistence-api.md.

Linking. From an operation, a reader reaches its plan, approval, every observation it relied on, every attempt and response, every accounting and the outcome; from a machine, every operation, drift record, adoption record and recovery epoch that affected it. A drift whose observed digest equals the artifact digest of an operation that was accounted by decision (§5.2) is linked to that operation as a possible late landing (§6.1).

Content limits. The timeline holds no plaintext configuration and no secret value; response text follows §3.5, and diffs are the compilation contract's redacted review data. The PoC deletes no timeline entry: application-controlled deletion of database records needs the reference graph of design §7.4, which the PoC does not build (design §7.8 item 1, derived). A restore erases every entry after its snapshot; §7 records the restore itself on a recovery timeline that the restored database cannot contain.

Nothing measured the timeline as a user surface: no investigation tested the authenticated API, scoped authorization or an operation timeline (FR §9 item 5). The API that exposes it is persistence's (design §11.2 Operation).

5. Interruption and retry classification

Design: §12.5, §12.6.

Recovery examines the target and cluster rather than replaying the journal. For each attempt, operation, mode, assignment revision and intervening state, the controller classifies the outcome:

Classification	Required action
Completed	Record completion, persist the bound release and digest as Applied, and release the scope and slot.
Safe to retry	Create a bounded retry within the original plan, expiry and unrevoked approval, admitted only by the §3.3 attempt transaction.
Unresolved	Preserve the assignment and stop dependent or conflicting mutations; observe further or request a specific recovery-admin decision.
Rejected	Record the response as proof of non-mutation, release the scope and slot, and require a corrected plan; the unchanged plan is not retried.
Failed	Record the contradicting completion observation, leave Applied unchanged, release the scope and slot, and require a corrective plan; if the observed digest differs from Applied's, the same transaction opens a drift record (§4, §6.1), so the corrective plan is a revert or an adoption.
Cancelled	Record what ended it (§5.1) and release the scope and slot; no attempt is recorded, and nothing is undone.

Completed, safe to retry and unresolved are the design's classifications for an outcome made uncertain by restart, ownership loss or transport failure. Rejected and Failed cover an outcome that is known, and Cancelled an operation that never recorded an attempt, under the §4 condition. Rejected needs a definitive response received and recorded for every recorded attempt, never when an earlier attempt was accepted or is unknown, and only of a class proven to precede mutation (§5.1). Failed needs a completion observation that contradicts the postconditions, not evidence that is merely missing, and every attempt accounted for (§4). Every other error or gap is unresolved.

A definitive rejection of any attempt rules out safe to retry: the retry would send the same artifact the node refused. With every attempt rejected the operation is rejected; otherwise, once every attempt is accounted for, a completion observation at the pre-dispatch digest makes it failed.

One case fits two rows: at least one recorded attempt, every attempt accounted for, none accepted or definitively rejected, and an observation at the pre-dispatch digest, which both contradicts the postconditions and meets the safe-to-retry evidence. The controller classifies it safe to retry only while the operation has attempts left under the bound, its approval passes comparison 1 and its scope gate is open; otherwise it classifies it failed (choice §10.10). E4 left this to a harness flag: row 013 ended failed, and rows 012, 014 and 015 retried (DS §2.1). A restored operation therefore never retries: its approval's epoch is not the current one (§7.1).

The controller observes after a lost response, restart, ownership loss or reconnect. Before a retry or dependent mutation it rechecks identity, assignment, current digest, running version and relevant health. An old operation that may have been accepted remotely blocks conflicting work until its outcome is resolved. Idempotency keys and journals prevent duplicate application intent in the manager, but do not provide exactly-once remote execution (design §12.5).

5.1 Evidence per outcome

For apply-config/no-reboot on one worker at one assignment revision, E4 found this evidence sufficient (DS §6.3); this contract adopts it and adds nothing it did not show:

Outcome	Evidence required
Completed	at least one attempt recorded and every attempt accounted for; then a completion observation of the artifact's digest and the other postconditions
Rejected	a recorded InvalidArgument response of the validation-error class to every attempt; any other InvalidArgument, such as the immediate-mode refusal below, accounts for its attempt and leaves the outcome to a completion observation. E4 showed it before any mutation for a validation error only, with the resource version unchanged (DS §4.6); no other code or error class is proven pre-mutation
Failed	at least one attempt recorded and every attempt accounted for, and a completion observation contradicting a postcondition, a bound health check only at or after the verification deadline (§4). The accounting must be true, not merely recorded (DS row 013)
Safe to retry	every attempt accounted for, or none recorded; no accepted response and no definitive rejection of the Rejected row's class (§5); a completion or recovery observation at the pre-dispatch digest; attempts left, an approval passing comparison 1 and an open scope gate (§5); a new attempt transaction bound to the classification's revision
Unresolved	any attempt with neither a recorded response from the target nor an accounting decision; or no completion observation by the verification deadline that establishes or contradicts the postconditions
Cancelled	after the commitment, with no attempt recorded: a revocation of the approval or of its identity, a cancellation, the plan's expiry, an approval whose epoch is not the current one, or a recovery-admin resolution. Before the commitment there is no operation; the plan ends revoked, cancelled or expired (§2)

Two response classes are deliberately not rejections. A dial failure (Unavailable) is an unknown outcome: whether it proves no send was not tested (DS row 015). A node refusing a configuration as not applicable in immediate mode was seen only on dry runs (E3 §4.3); a recorded response of that kind accounts for its attempt, but the outcome is decided by a completion observation, not by the response.

Only a response from the target (an acceptance or a gRPC code the node returned) accounts for its attempt. A transport outcome (a dial failure, a timeout, a reset connection) is recorded as the response class of §4.1 but accounts for nothing: the attempt stays unaccounted until a §5.2 decision.

A retry re-applies the same artifact. If an earlier, abandoned request landed on top of it, it would apply the same configuration again, which should change no configuration; an identical re-apply was not captured (DS §4.4, §5). A retry is safe only as long as the accounting behind it is true (§5.2).

5.2 Accounting a lost response

No evidence establishes when an abandoned request can no longer land. The executor's exit is not enough (DS §4.4, capture 1 row 012), and E4's 30 s settle is a choice, not a bound: nothing shows how long the control plane's proxy can hold a request (DS §6.3; FR §9 item 3). Until there is such evidence, an attempt whose response was lost is accounted only by a recovery-admin decision (DS §8), which may be recorded only when all of these hold and are recorded with it:

1. the attempt's executor is stopped or taken over, so it can record no further attempt (§3.4). Evidence of a stop is the takeover, which recovery-admin may request explicitly; this contract keeps no separate stop record;
2. at least the settle floor has elapsed since the latest of the attempt's transport deadline, the executor's stop or takeover, and the first successful read of the machine after that deadline. The settle floor is a static deployment setting of at least 30 s, E4's settle, held outside the application database so that a restore cannot change it (choice §10.11). It is a floor, not a bound: the captured landing came about 11 s after the heal and within it, and nothing shows that every landing does; and
3. a recovery observation taken after the floor, with the machine-configuration resource version read at the start and end of the settle. The settle starts at the first successful read of condition 2, which is the start observation, so a change between that read and the end cannot be hidden by a later start. Only recorded observations count: a read begun and not yet recorded is no evidence, and no entry marks a begun read as abandoned, so the decision rests on the observations recorded when it is taken. That version moved on every landed apply in E4 (DS §2.3); a change during the settle is recorded, and the decision states how it is explained.

The owner of an operation with an unaccounted attempt takes these recovery observations: none before the attempts' latest transport deadline, then one, and one more each time a settle floor has passed since its last was recorded (or, while it is still being read, began), as condition 2 measures it. They settle nothing, since the attempt is unaccounted (§4); the decision names two of them.

The decision records that it is a judgement, not evidence that the request can no longer land. The residual is explicit: if the request lands after the decision, it can overwrite work that the released scope then admitted, which is the stale A-after-B case reached through accounting that was recorded but not true (DS §4.4). That landing is not prevented. It is detected only as drift at the next observation, when the observed digest differs from Applied, and is linked to the operation (§4.1, §6.1).

5.3 Interruption points

Required interruption tests cover failure before commitment, after commitment but before send, after remote acceptance, before response recording, and before local completion recording. Each test must show the resulting classification and blocked scope; a generic "mark successful" recovery action is not valid. E4 killed its executor at each point and found the state this contract names, with no generic resolution (DS §4.5):

Point	E4 row	State left	Resolution under this contract
before commitment	018	no operation; the plan still approved	another controller runs the plan from §3.1
after commitment, before any attempt	019	committed, no attempt	takeover; no attempt to account; a recovery observation at the pre-dispatch digest classifies safe to retry
accepted, response not recorded	020	sending, no response	takeover; refused while unaccounted; accounting decision (§5.2); completion observation
matching observation, completion not recorded	021	verifying, response recorded	takeover; a new completion observation; completed

6. Drift handling

Design: §12.4, §7.1, §9.1, §13.7.

No feasibility evidence covers this section. No investigation exercised drift freeze, adoption policy or revert; E1's adoption flow covers only extraction ordering, on the same input and function as import (FR §6 SP02, SP22; FR §9 item 1). Every transition below is specified, not evidenced; §9.3 states what would close the gap.

6.1 Detection

Drift is an observed configuration digest that differs from the Applied digest while no operation on that machine is committed, sending, verifying or unresolved. Two look-alikes are not drift:

• Desired differing from Applied is pending convergence (§1).
• A digest mismatch while an operation holds the machine scope is evidence for that operation's classification (§5).

The controller observes each managed machine periodically (purpose drift), every configured drift interval (choice §10.36). A mismatch in a drift observation, in a restoration observation after a restore (§7.3 step 4), or in the evidence observation of a commitment that comparison 3 refused on its digest (§3.2), opens a drift record on the machine's timeline, naming the observation, the Applied digest and the observed digest, and raises the design §15.3 digest-mismatch alert, as its opening entry (choice §10.35). So does the completion observation that makes an operation failed, in the transaction that releases the scope (§4). The definition above holds for a restoration observation too: while a restored operation on the scope is still unresolved, the mismatch is evidence for its classification (§7.3 step 5), and the drift record opens only once that operation is terminal: with its failure, or otherwise at the scope's marking (§7.3 step 6). A scope whose restored operation stays unresolved is marked unresolved and opens no drift record yet. A machine has at most one open drift record. A drift whose observed digest equals the artifact of an operation accounted by decision is linked to it as a possible late landing (§5.2).

An observation is compared with the Applied that held at its basis (§4.1), not with a later one. If the machine's baseline revision (§2) changed after that basis, the observation opens and closes no drift record, and a later observation decides. It likewise opens and closes none when a recorded observation of the machine with a higher basis read the configuration digest: the higher-basis one decides. The completion observation that makes an operation failed is the exception: any such observation read the same digest (§4) and, recorded while the operation held the scope, opened nothing. A drift or evidence observation also opens none if an operation held or took the machine scope at or after its basis. A read that began before an operation applied a new release therefore cannot report the old digest as drift, and a read that began before a newer recorded one cannot open or close a record against it.

Drift is never an automatic apply trigger, and detection does not freeze the scope by itself (choice §10.12). It needs no freeze to stop stale work: while the record is open, comparison 6 refuses every plan that does not bind it (§3.2), including an older plan whose bound pre-dispatch digest happens to equal the drifted one, and a plan that binds it is a revert (§6.4) or an adopt plan (§6.3). Design §12.4 has the operator choose freeze, adopt or revert.

External Talos access, break-glass included, remains independent of application approval and is reconciled as drift afterward (design §12.7, §13.5).

6.2 Freeze

Any author, publisher, approver or recovery-admin may freeze a machine scope; only an approver unfreezes it (design §13.7 item 6). A freeze is a durable fact on the machine scope that closes the scope gate: an approved plan fails comparison 6 of §3.2, and no attempt transaction succeeds until an approver lifts it (§3.3). A freeze does not stop an attempt already recorded, and it is independent of drift: it may be placed without a drift record and outlives the record's resolution until lifted.

POST /machines/{id}/freezes and POST /machines/{id}/unfreezes each take a reason (1 to 1024 bytes, not blank) and record one immutable freeze or unfreeze fact with its principal, role, reason, epoch and act, as one freeze or unfreeze entry at the machine's next revision (§4.1), under the machine row's lock that comparison 6 reads under. Freezing a frozen scope, or unfreezing one that is not frozen, is 409 conflict and writes nothing (choice §10.34).

6.3 Adopt

Adoption accepts the configuration the machine now runs. It sends nothing to the machine: dispatching the adopted release would mutate a machine whose state is being accepted (design §12.1).

1. Ingest. The drifted configuration is read from the node and ingested through the compilation contract's pipeline as a drift adoption (§2.3 there): known and marked secrets are extracted before any backup-visible write, and the exact input is encrypted as the baseline under the baseline key, which no runtime identity can decrypt. Adoption success, failure or interruption leaves no plaintext in drafts, indexes, staging, database writes that a base backup, log archive or storage snapshot can capture, responses or logs; sanitizing a record afterwards does not satisfy this (design §7.1; E1 §6). This ingestion is a human author's, as the owner decided the question design §13.7 left open (persistence contract choice §17.22) (choice §10.14). Ingestion records the baseline's configuration digest (§1) over the same input it encrypts (compilation contract §2.3 step 8). An existing-cluster import records it the same way, so the handover below compares against the same digest.
2. Publish. The sanitized document becomes the machine's new import base, and a release compiled from it is published (compilation contract §6, §11). Publication resolves nothing and authorizes nothing.
3. Plan and approve. An adoption is requested as a plan with operation adopt (choice §10.13), created by a publisher like every plan (§2; design §13.7 item 2) and approved by an approver, as design §13.7 item 3 allows, with the self-approval rule. The adopt plan binds the machine and its identity (§2), its assignment revision, its baseline revision (none for a machine with no Applied), the machine's Desired release at plan creation (or none), the open drift record (none for a handover), the adopted release, the baseline's configuration digest, a maximum observation age, its expiry, the approval policy, and its creator (choice §10.38). It has the plan states of §2, and any change to a bound value needs a new plan.
4. Record. The adopt plan's commitment is the adoption record transaction. It locks the machine row as §3.2 does, so it is ordered against every observation, commitment and drift record of the machine (§4.1), and it requires:
5. no operation exists for the plan (§3.2 comparison 0);
6. the plan's approval passes §3.2 comparison 1, under the same locks: unexpired, not revoked, its identity not revoked, and recorded in the current recovery epoch;
7. the machine's assignment revision, baseline revision and Desired release equal the bound ones, and the bound drift record is still open (for a handover, the machine still has no Applied);
8. the machine's recorded observation with the highest basis (§4.1), of any purpose, began its read after the approval, is no older than the age the plan binds, and shows the machine identity the plan binds, the machine's configuration digest equal to the baseline's and its assignment revision unchanged; a started read with no recorded result does not count (§9.3); and
9. no operation holds the machine scope, and the scope gate is open apart from a freeze: adoption changes nothing on the machine, so a freeze does not block it, but recovery mode without a release of the scope in the current epoch does (§7.5).

Instead of an operation that sends, the transaction creates the plan's operation of kind adopt directly in completed, with the adoption record as its outcome. It is never committed, sending, verifying or unresolved, so it never counts against comparison 4.

The controller records it unasked: for each adopt plan whose approval reads approved and whose machine's scope admits it now (requirement 4.5), it takes an observation for the plan (purpose evidence, at the plan's route) and then attempts the record, which relies on whichever recorded observation has the highest basis. It does so after each approval and periodically. Requirement 4 fails in two ways. Missing evidence: no recorded observation, or the one the record relies on (whoever took it) began before the approval, is older than the plan allows, or could not read the identity, the configuration or the assignment evidence. Then the transaction rolls back without a refusal entry and the plan is observed again later. Contradicting evidence, an observation showing another value, is refused. A refused record ends the attempts for that plan, so a mismatch is recorded once and not on every period: a new plan is the retry (choice §10.27). The refusal entry's own transaction (§4.1), under the machine lock, appends nothing when the plan was adopted or refused since the refused attempt rolled back, so racing attempts record one outcome; likewise the record finds a refusal entry naming the plan under the machine lock and writes nothing, so a refused plan is never adopted afterwards. An adopt plan's only operation is its adoption, so an attempt that finds one (requirement 1), such as a second controller instance's racing for the same plan, finds the plan adopted: it rolls back without a refusal entry, which would follow the success on the timeline.

The record compares the observed digest with the baseline's digest, the bytes read from the node, not with the recompiled release's artifact (choice §10.15). Whether an artifact compiled from an unchanged import base reproduces the node's bytes is not evidenced, and E3 saw the renderer re-encode a whole configuration (E3 §7); requiring equality with the artifact could make adoption impossible.

The record sets Applied to the adopted release with the baseline's digest, marked adopted by observation rather than dispatched, and confirms the adopted release as Desired, selecting it if publication has not already (the selection is persistence's), so that the machine is not at once pending convergence towards the release it drifted from and a later plan cannot silently undo the adoption. It advances the baseline revision (§2), so every plan made before the adoption, such as an approved revert to the previous release, fails comparison 2 and must be planned and approved again. It closes the drift record. If a newer observation shows the machine has changed again, it has the highest basis, requirement 4.4 fails and the machine stays drifted, even when an older read is recorded after it. If a revert closed the drift record first, requirement 4.3 fails, even when a later edit restored the adopted bytes. If another release was selected as Desired after the plan was created, requirement 4.3 fails too: the record would overwrite that newer selection unseen, so a new adopt plan, whose approval sees the selection it supersedes, is needed. Publishing and approving an adopted release does not by itself resolve drift.

If the adopted release's artifact digest differs from the baseline's, the machine is then pending convergence (§1), not drifted. Converging it is an ordinary approved plan whose redacted whole-configuration diff shows what the re-encoding changes (§2).

Existing-cluster handover. The same transaction establishes the first Applied for a machine that has none, which is how existing-cluster adoption (design §9.1) hands a machine over. Such a machine is not drifted, since there is no Applied to differ from; the adopt plan is the handover authorization, binds no drift record and no baseline revision, and compares the baseline digest recorded by import (step 1); requirements 4.1 to 4.5 apply otherwise unchanged. The rest of that handover, from publishing the import draft to recording the adoption, belongs to the approval work; the import that precedes it belongs to the adoption work.

6.4 Revert

A revert is an ordinary plan (§2) for a machine with an open drift record, from the selected applicable release, after checking assignment, current state and dependencies (design §12.4). It binds the drift record and, as its pre-dispatch digest, the drifted digest, so if the machine drifts again before commitment, comparison 3 fails and the plan must be made again. Its approval is recorded as approving the overwrite of configuration that no reviewer has seen: the drifted configuration has no sanitized form unless adopted (choice §10.16). The approval and its timeline entry carry that mark as unseenOverwrite (persistence and API §9.3), derived from the immutable plan. Normal approval applies, and a freeze must be lifted by an approver before it can commit. Comparison 6 admits the record the revert binds and no other (§3.2). A revert ends through the normal path: it advances Applied when it reaches completed, and the same transaction closes the drift record reverted with a drift entry after the Applied change, resting on the completion observation.

6.5 Drift states

From	To	Condition
none	open	A drift or restoration observation's digest, or the evidence observation's of a commitment refused on its digest (§3.2), differs from the Applied at its basis, with the scope free and no recorded higher-basis observation of the digest (§6.1); or the completion observation that makes an operation failed differs from Applied (§4).
open	open	Freeze or unfreeze; a further differing observation is recorded on the same record, under the same basis rules (§6.1).
open	closed (adopted)	An adoption record (§6.3).
open	closed (reverted)	The revert operation reaches completed (§6.4).
open	closed (returned)	A later observation equals the Applied at its basis again with no Bronzeward action, such as a manual revert, and no recorded observation with a higher basis read the digest (§6.1); recorded as such.

A revert that ends failed, rejected or cancelled leaves the record open. An operation's own digest mismatch while it holds the scope opens no record, except the one that makes it failed, as the scope is released (§4).

7. Recovery after management-state restoration

Design: §14.6, §14.4, §7.7, §7.8, §13.7.

After restoring the database, OpenBao or the credential store, the operator enters recovery mode before normal startup (design §7.7 duty 6, §14.6). In the PoC, entering and leaving recovery mode, releasing scopes and resolving unresolved operations belong to recovery-admin (design §13.7 item 5). OpenBao repairs such as unsealing, restoring, and key or policy changes stay with the OpenBao administrator, and break-glass access with neither.

What is evidenced and what is not. A restore rewinds identifiers and fence generations on both backends, and a pre-restore ownership token passes the fence again (observed negative, DB §4.7 rows 027 and 055; reproduced, FR §7 R4). No investigation modelled recovery-mode entry, executor quiescence or a restore epoch (DS §7; KL §6; FR §9 item 4). The procedure below is specified, not evidenced; §9.3 states what would close it.

7.1 What a restore does to the fences

A restore erases everything after its snapshot: plans, operations, attempts, takeovers, approvals, revocations, identity revocations, drift and adoption records, and releases. It rewinds the counters that issued them: identifiers drawn from database sequences are reissued, as in the prototypes, and so are revision numbers and fence generations (DB §4.7). Persistence makes every identifier that leaves the database random and never reissued (see persistence-api.md), so under it only revision numbers and generations are reissued; this contract does not rely on that (item 3 below). Anything outside the database that still holds a pre-restore token or reissued number (a running executor, a provider object, a log, a machine running an erased release's artifact) can collide with a newly issued one. A recovery epoch kept as a counter in the same database is rewound by the same restore (DB §9, inferred), so "greater than any restored value" does not make it unequal to an epoch issued after the snapshot.

This contract therefore requires from persistence, see persistence-api.md:

1. a recovery epoch identifier, issued at installation and again at each recovery-mode entry, that is never issued again, including after a restore to any earlier snapshot. This contract compares epochs for equality only and never orders them. A random identifier of at least 128 bits meets this without state outside the database (choice §10.17);
2. every ownership token, approval and scope release carries the epoch in which it was issued, and the comparisons of §3.2 and §3.3 compare it; and
3. no rule of this contract matches a release, operation or approval across a restore by identifier. A machine is matched to a release by configuration digest only.

Approvals and ownership are then safe across a restore in this sense: an approval or token issued before the current entry, including one the restore erased or revived, authorizes nothing, because its epoch is not the current one. Requests already sent are not covered by any fence (§3.4); §7.3 step 1 handles them.

7.2 Entry

The controller supports a recovery start, a startup option under which it keeps every dispatch gate closed: it runs no executor and attempts no commitment or attempt transaction. Until recovery-mode entry commits it serves only a liveness probe and the entry act itself: the restored epoch is still current until then, so a token revoked after the snapshot still authenticates, and any other act served would be taken on that token's authority (see persistence-api.md). Once entry has committed, it serves observation, the checks of §7.3, the recovery API, the acts §7.5 always allows (approval and identity revocation, plan cancellation, freeze) and the installation-wide acts of §7.5 (drafts, ingestion, compilation and publication). It runs no job worker, so a request whose work runs as a job, such as publishing the release that clears a blocked scope (§7.4), is accepted but completes only after the normal restart below, which entry allows before any scope is released. For a restore of any of the three backup families, the operator stops, or establishes as stopped, every controller instance before the restore of any family begins, records the time (§7.3 step 1), and keeps every instance stopped until recovery-mode entry commits; the only controller started meanwhile is the one with recovery start, after the restore. Until entry commits, the epoch the instances hold is still current, whichever family was restored, so an instance left running would pass the comparisons of §3.2 and §3.3 on the approvals and ownership it reads: only this stop prevents that. It is an operator step that Bronzeward neither enforces nor detects, like the restore itself (design §14.6). After the restore, the operator starts one controller with recovery start, and recovery-admin records recovery-mode entry through the API (design §13.7 item 5, §14.6). The start option only keeps the gates closed; no server-side command enters recovery mode by itself (choice §10.18). Bronzeward does not claim to detect a rollback (design §14.6): starting normally after a restore is an operator error that the epoch check does not catch, because no entry has issued a new epoch.

The entry record states which backup families were restored, the identity of each backup and its age, so that the pairing duties of design §7.7 (duties 3 and 4) can be checked (§7.3 step 3). Entry is one transaction that:

1. issues the new epoch;
2. closes the scope gate of every machine scope: comparison 6 fails and no attempt transaction succeeds until the scope is released (§7.3 step 7);
3. takes over every non-terminal operation into the new epoch, so each in committed, sending or verifying becomes unresolved, and each unresolved one stays unresolved under the new owner (§3.4);
4. marks every machine scope pre-restore unaccounted (§7.4): the restored journal can omit operations dispatched after the snapshot, so no scope is presumed quiet because its restored journal shows nothing (choice §10.19); and
5. abandons every ingestion claim whose epoch is not the new one (compilation contract §3.5).

Approvals carrying another epoch need no action: they fail comparison 1 from the moment the epoch changes (§3.2).

Once entry has committed, the operator may restart the controller normally, as persistence also allows: recovery mode and the scope gates are then database facts, comparison 6 still refuses every scope not released in the current epoch, and recovery mode stays in effect until §7.6. The restart is what runs the job worker that publishes the release clearing a blocked scope (§7.4), which the recovery start does not run (see persistence-api.md).

7.3 Procedure

1. Quiesce. Every controller instance was stopped, or established as stopped, before the restore of any backup family began, and stayed stopped until entry committed, apart from the one started with recovery start after the restore (§7.2); the time of the stop is recorded. Stopping is what prevents a further attempt; after entry, the epoch refuses one from any instance that was missed (§7.1). Neither prevents a request already sent from landing. Every scope stays pre-restore unaccounted until recovery-admin records the §5.2 accounting decision for it. In §5.2 condition 2 the stop of every pre-restoration instance takes the place of the executor's stop, and the settle floor runs from the latest of the three times that condition names, not from the stop alone. For a scope whose attempt the restored state does not hold, the attempt's transport deadline is unknown, and the latest time it could have been takes its place: the stop of the pre-restoration instances plus the maximum transport deadline, a static deployment setting held outside the application database like the settle floor. Plan creation refuses a transport deadline above it. The setting is a high-water mark: it may be raised but never lowered, because a plan made under a higher value may be the one whose attempt the restore erased. One decision may cover many scopes, each with its own recovery observations, and refuses all of them when one fails a condition. The entry records the stop. The controller takes those observations itself (purpose recovery, naming no plan or operation), the recovery start after entry and every start while recovery mode is in effect: one of each pre-restore unaccounted scope once that scope's transport deadline has passed, then one per settle floor after the last was recorded, a machine inventoried later included. The floor is the machine's: any recovery observation of it paces the next, whichever loop or controller took it, and the observation's start rechecks it under the machine's lock. The decision accounts every restored unaccounted attempt on the scope under its one act, then moves the scope to unresolved where a restored operation holds it, and to blocked otherwise, until its restoration observation and the dependency check re-mark it (step 6).
2. Verify schema, revisions, releases, operation journals and provider and key references from the restored state. Re-record, from the operator's records, every identity revocation made after the restored backup, which the restore erased (design §13.7 item 4). A token revoked after the backup must not authenticate again; persistence meets this by refusing every automation token whose epoch is not the current one, so every token is reissued with the server-side tool, and by a denied-subject list in deployment configuration, which a restore does not rewind: for each identity revocation, of a human or a service identity, the operator adds its subject to the list before the revocation is recorded, so no failure between the two can lose both; the tool issues no token to a listed identity, so a reissue that precedes the re-recording cannot revive a revoked service identity (design §13.7 item 1; see persistence-api.md).
3. Check dependencies. Check retained dependencies (design §7.6) and the backup pairing of design §7.7, and test the decryption and credentials each scope needs (§7.4) under the actual recovery identities, stopping and naming what is missing (KL §5.3, §7 item 4). The check runs under identities that cannot create a Transit key: Transit creates a missing key on encrypt for an identity holding create, and a key recreated under a lost key's name is not a recovery path (KL §5.3, §5.4). Applying a retained artifact and regenerating a release need different dependencies (design §7.8, §14.4).
4. Observe machine identity (its SMBIOS UUID, Talos node ID and Talos cluster ID), assignment, running version, configuration digest and health for reachable targets, after step 1 (purpose restoration).
5. Reclassify restored pending operations under §5, never replaying a journal entry because it is pending. A restored operation never retries: its approval's epoch is not the current one (§5). One with no attempt ends cancelled, which claims only that the restored journal records none: an attempt the restore erased is covered by step 1's decision, and a landing of it is drift at step 6. The recovery start ends such an operation cancelled itself once entry has committed and step 1's decision has accounted its scope, sending nothing, so that its scope can be marked and released before the normal restart; the executor of that restart classifies one with an attempt. One with an attempt is accounted, by its recorded response or by a §5.2 decision such as step 1's, and is classified by the §5.1 table: completed or failed by a completion observation taken after that accounting, and otherwise unresolved, which holds the scope until a later classification or a recovery-admin resolution. It is never rejected, even when every restored attempt has a definitive rejection: the restored journal can omit a later attempt (§7.2 item 4), which step 1's decision accounts for but no response shows rejected.
6. Mark each scope ready, blocked or unresolved, recording the missing dependency or evidence (§7.4). A restoration observation marks a scope ready only when the identity it shows matches the machine record as comparison 3 of §3.2 requires, the identity every plan binds (§2); one that shows another identity leaves the scope blocked on that mismatch (choice §10.26). An observed digest that the restored Applied does not explain is drift (§6), not grounds for an apply, even when it matches a restored release's artifact (§7.1 item 3): the restoration observation's own record applies §6.1's drift rules, as a drift observation's does. A recovery-admin's mark request takes the scope's restoration observation and its step 3 check itself, then marks the scope from that evidence; it names no state.
7. Release. recovery-admin releases eligible scopes explicitly. The release is recorded in the current epoch and opens the scope gate for that scope only. It is not approval: a plan for the scope still needs an approval recorded in the current epoch (design §14.6, §13.7 item 5). A release needs the scope's Desired release to be the one its mark checked the dependencies for; a Desired moved since needs a new mark.
8. Leave recovery mode (§7.6).

A restored desired state is never proof of current machine state, and a scope release is not blanket approval for pending mutations.

7.4 Scope states in recovery

State	Meaning	Leaves by
pre-restore unaccounted	set at entry; a request sent before the restore may still land	the step 1 accounting decision: to unresolved where a restored operation holds the scope, otherwise to blocked until step 6 re-marks it
unresolved	a restored operation on the scope is unresolved	its resolution under §4 and §5, then re-marking
blocked	the restoration observation failed or showed a machine identity other than the machine record's (choice §10.26), or a dependency to apply the machine's Desired release is missing: its release record, its artifact's key version at or above the decryption floor and decryptable by the executor identity, or the executor's operation credentials (§3.1 item 2)	a later successful restoration observation whose identity matches, for a failed or mismatched one; the dependency restored or repaired, or a newly published release selected as Desired whose dependencies are present; then re-marking
ready	accounted, no operation holds the scope, the dependencies above present, a restoration observation whose basis (§4.1) follows step 1 and whose machine identity matches the machine record (choice §10.26)	release
released	released in the current epoch	recovery-mode exit, or a new entry

A machine inventoried after entry starts pre-restore unaccounted, like every scope at entry: the restore may have erased an earlier inventory of the same machine together with an attempt sent to it. It takes the §7.3 step 1 decision after the same settle floor, then is marked and released like any other before recovery mode can be left (§7.6).

The dependency set is the one dispatch checks at use time (§3.1), for the release a plan would target, not the dependencies of regenerating it (choice §10.20). A scope whose Desired release names a key version lost for good leaves blocked through a new release: ingestion, compilation and publication stay allowed during recovery mode (§7.5), and a regenerated release needs the source secret versions it pins, or new ones ingested (design §14.4; KL §5.3). Every plan still repeats the check at use time (§3.1).

Only ready scopes may be released. A drifted ready scope may be released and then handled under §6; release does not resolve drift. A freeze is independent of these states and survives release.

7.5 What recovery mode allows

Recovery mode is enforced per scope, not by refusing the whole installation (choice §10.21):

Acts	While recovery mode is in effect
Acts that only remove authority: approval and identity revocation, plan cancellation, freeze	always allowed, to the roles design §13.7 gives them
Acts that record recovery: entry, accounting decisions (§5.2), resolutions of unresolved operations, scope marking and release, leaving	always allowed, to recovery-admin
Drafts, ingestion, compilation and publication	allowed installation-wide: writes are confined to the database and provider; ingestion/publication may perform their read-only node observations
Plan creation and approval, unfreeze, adoption, and anything that could send to a machine	refused on a scope still pre-restore unaccounted
Commitment and attempt transactions, and adoption records	refused on every scope not released in the current epoch (comparison 6; §6.3 requirement 4.5)
Every act on a released scope	allowed, as outside recovery mode

Plan creation is refused with approval on a pre-restore unaccounted scope, since the plan's preconditions would be judged against a scope whose state is not yet accounted. Approval on a scope that is accounted but not yet released is allowed: it authorizes nothing until release, because comparison 6 still refuses the commitment.

7.6 Leaving recovery mode

recovery-admin leaves recovery mode only when every machine scope is released (choice §10.22). A scope that cannot yet be released keeps the installation in recovery mode, but blocks only itself and, while a restored operation on it is unresolved, its rollout scope: that operation keeps its rollout slot (§3.2 comparison 5), so at the PoC limit of one no plan for another machine of that rollout scope commits until the operation is terminal. Released scopes outside that rollout scope are fully usable (§7.5), and a blocked scope has an exit through a successful observation, a repair or a new release (§7.4). One case has no exit: a machine that can never be observed again (destroyed or permanently unreachable) cannot supply the restoration, recovery or completion observation that ready, the §5.2 decision and a restored operation with an attempt need, so its scope keeps the installation in recovery mode, and such an operation keeps its rollout slot. Nor has a machine whose node now reports another identity than its record's: a successful restoration observation leaves its scope blocked, and no repair or release clears it (§7.4, choice §10.26). The PoC specifies no decommission or exclusion act for either; that is a gap (§9.3), and until one exists the scopes outside its rollout scope stay fully usable because they are released individually. Leaving never reopens a scope that was not checked. Leaving returns every released scope to normal, each with a scope state entry naming the exit, and ends the recovery timeline; the epoch stays current until the next entry. Leaving reads the installation state for update, so an inventory request that holds it in recovery mode commits first and its new scope, pre-restore-unaccounted, refuses the exit (persistence and API §5, T11).

8. Invariants and worked interleavings

An implementation and its reviewer can check these directly:

1. One uncertain operation per machine. At most one operation per machine coordination scope is committed, sending, verifying or unresolved.
2. Commitment and attempt precede send. No Talos request is sent without a durably recorded commitment and attempt earlier on the same timeline, and every attempt was admitted by an attempt transaction.
3. Applied follows evidence. Applied changes only on entering completed, to that operation's bound release and digest, or by an adoption record, to the baseline's digest. Both need an observation taken for that purpose: after at least one attempt is recorded and every recorded attempt is accounted for, or after the adoption approval and inside the age it binds.
4. Plans do not change. Any change to a binding is a new plan that needs a new approval.
5. Release only on a terminal state. The machine scope and rollout slot are released only by completed, rejected, failed or cancelled, and never while a recorded attempt is unaccounted for.
6. Absence of evidence classifies nothing. A timeout, lost response, executor exit or pending journal entry alone never yields completed, rejected, failed, cancelled or safe to retry, and never accounts for an attempt.
7. No plaintext before extraction. Known and marked secrets never reach a backup-visible write in plaintext, adoption included.
8. Nothing from another epoch authorizes. An approval, ownership token or scope release whose epoch is not the current recovery epoch never satisfies §3.2 or §3.3.
9. A held scope freezes the assignment. A machine's assignment does not change while an operation on it is committed, sending, verifying or unresolved.
10. No mutation through a closed gate. No commitment or attempt transaction succeeds for a scope that is frozen, except the adoption record, the adopt plan's commitment, which changes nothing on the machine (§6.3 requirement 4.5). None of them, the adoption record included, succeeds for a scope under recovery mode without a release in the current epoch.
11. Only a stated identity decides. Every accounting decision and every operator resolution is recovery-admin's, recorded with its basis; the controller decides only what evidence determines.
12. One operation per plan, from commitment. A plan has no operation before its commitment transaction and exactly one after it.
13. Recovery mode holds each scope, not the installation. While recovery mode is in effect, nothing that could send to a machine, approve or adopt succeeds for a pre-restore unaccounted scope, and a released scope is governed as outside recovery mode (§7.5).

8.1 Revocation racing commitment

Plan P for machine M is approved; the controller has recorded its §3.1 evidence for P. A revocation R of P's approval and P's commitment transaction C, which would create operation O, race.

• R commits first. C fails comparison 1. No operation is created, nothing is sent, and P is revoked (DS row 002).
• R starts while C holds the approval. R locks M's row FOR UPDATE before the approval (persistence-api.md §5, T5b), so it waits for C, which holds M's row, and commits after it (§2). O is committed. C updated M's row, so once C commits, R locks the row's new version afresh. If R takes it before O's attempt transaction, R then locks the approval and commits, and the attempt fails comparison 1: committed → unresolved → cancelled, nothing sent (DS row 003; that R waited is inferred from the harness's record, DS §4.1). If the attempt transaction takes M's row first, R waits for that transaction too, and the outcome is DS row 005's (choice §10.31).
• R commits after C, before the attempt. The same outcome (DS row 004).
• R commits after the attempt transaction. The attempt is sent, lands and completes. R prevents any retry and undoes nothing (DS row 005). This is the stated residual.
• Without the in-transaction comparison (control row 006), a revocation after a single early check is not seen and the artifact is sent.

Each conclusion is only as strong as invariant 2, which E4 showed for its own executor only (§3.3).

8.2 Uncertain outcome blocks a newer plan (stale A-after-B)

Operation A (artifact a) for M is sending under owner X when the network partitions. The controller restarts as Y and takes A over; A becomes unresolved.

• A newer plan B for M cannot commit: comparison 4 fails while A is unresolved (invariant 1; DS row 010). Nothing newer than a can be dispatched while A is uncertain, so a late delivery of X's request cannot overwrite newer configuration. Without the scope index, E4's control row 011 recorded both A and B completed with A's stale send overwriting B.
• Y refuses to resolve A while its attempt is unaccounted (DS rows 012–015). If X's response is recorded, A is accounted; otherwise only a §5.2 decision accounts for it.
• After accounting, a completion observation of a completes A, Applied becomes a, the scope is released, and B's plan, made against the old baseline, now fails comparison 2; a plan made afresh, B2, can proceed (DS row 010).
• If M still reports the pre-dispatch digest, A is safe to retry only while attempts remain, its approval still passes comparison 1 and the scope gate is open; then one bounded retry runs (DS rows 012, 014, 015). Otherwise A is failed (§5, §8.4, choice §10.10).

8.3 Crash between commitment and send

O is committed; the controller X dies before its attempt transaction (DS row 019), or loses ownership there (row 007).

• Y takes O over: the generation rises and O becomes unresolved. If X is still alive, its attempt transaction fails comparison 7 and it sends nothing (row 007; control row 008 shows the send without the fence).
• No attempt was ever recorded, so nothing needs accounting. Y takes a recovery observation; at the pre-dispatch digest, O is safe to retry, and Y's first attempt transaction admits one attempt. One landing (rows 007, 019).
• Had the approval been revoked meanwhile, the attempt transaction would fail comparison 1 and O would become cancelled.

8.4 A late landing and the accounting decision

A's request goes through the control plane's proxy; the worker is partitioned; X's client reaches its transport deadline and is killed; the partition heals.

• A is unresolved with an unaccounted attempt. recovery-admin may not account it on X's exit. In E4, accounting on exit alone classified row 013 failed 11 s after the heal, the moment another capture's identical request landed; nothing landed there, so failed happened to be right, on evidence that could not tell the cases apart (DS §4.4). In the reproduction a request landed inside row 013's window, just before the observation, which was therefore right again (FR §7).
• After the settle floor (§5.2) and a recovery observation: if the observation shows a, the landing happened and A completes (capture 1 row 012). If it shows the pre-dispatch digest, A is retried once, provided attempts remain, its approval still passes comparison 1 and its gate is open; otherwise A is failed (§5).
• If the request lands after the decision, it re-applies a. That is harmless on top of A's own retry (inferred, §5.1), but if the scope has since passed to B and B completed, the landing reverts B. The next drift observation sees a where Applied is B's, opens a drift record linked to A, and the operator chooses freeze, adopt or revert (§6). The landing is detected, not prevented.

8.5 Identity revocation

Approver P approved plans Q1 and Q2. Q1's operation is committed with no attempt; Q2's has an attempt recorded. recovery-admin records an identity revocation of P.

• Q1's attempt transaction fails comparison 1: unresolved, then cancelled. An uncommitted plan approved by P cannot commit and ends revoked. The revocation locks P's principal record, so it waits for any commitment or attempt transaction that read it FOR SHARE (§2).
• Q2's recorded attempt is sent and runs to its classification. No further attempt of Q2 is admitted: comparison 1 refuses it, retries included (choice §10.6). If the attempt was accepted and a completion observation shows its artifact, Q2 completes; if its outcome is unknown, Q2 is unresolved and ends through a §5.2 accounting decision and a completion observation, completed or failed.
• Identity revocation was not measured (design §13.7); this walk-through rests on the comparisons, not on a row.

8.6 Drift: freeze, then adopt or revert

M's Applied is release r3 with digest d3, and plan P (r4, pre-dispatch d3) is approved. During an incident an operator edits M with talosctl; a drift observation reads d9. No operation holds M's scope, so drift record D opens.

• P cannot commit: comparison 3 fails, d9 ≠ d3, and comparison 6 fails because D is open and P does not bind it. An author freezes M's scope while the incident continues.
• Adopt. The drifted configuration is ingested as a drift adoption; its secrets are extracted and its exact bytes encrypted as the baseline with digest d9. Release r5 is published from the new import base. A publisher creates an adopt plan binding M, its assignment and baseline revisions, its current Desired release, D, r5 and d9, and an approver approves it. A drift observation after the approval still reads d9 and D is still open, so the adoption record commits (the freeze does not block it, §6.3 requirement 4.5, invariant 10): Applied is (r5, d9, adopted), Desired is r5, the baseline revision advances, D closes. P now also fails comparison 2 and must be planned again. The freeze stays until an approver lifts it. If r5's artifact digest is not d9, M is pending convergence towards r5.
• Revert instead. A publisher plans r3 for M; the plan binds D and pre-dispatch digest d9, and an approver approves the overwrite of the unseen change, then lifts the freeze. The operation commits, sends and completes: Applied is (r3, d3), the baseline revision advances, D closes. Had M changed again to d10 before commitment, comparison 3 would have refused it.
• Neither path is evidenced (§6).

8.7 Restore to an older backup

Backups of all three families were taken at T0, paired as design §7.7 requires. At T0, operation S on M3 was sending: its attempt was recorded and its request sent. After T0, S's request was accepted and S completed, Applied becoming s; release r4 was published and operation A applied it to M1, Applied becoming a; operation C on M2 recorded an attempt whose request is held by the control plane's proxy; approver P was identity-revoked. The database is then restored to T0: S is sending again with no response, and r4, A, C and P's revocation do not exist.

• Entry. Before the restore begins, the operator stops every controller instance; after it, the operator starts one in recovery start, and recovery-admin records entry. A new epoch E is issued; every scope's gate closes; every restored non-terminal operation is taken over into E, so S becomes unresolved; every scope is pre-restore unaccounted; ingestion claims are abandoned. Every approval, P's revived ones included, carries an epoch other than E and authorizes nothing.
• A stale instance that was missed reconnects after entry with a pre-restore token. The restore reissued its generation (DB §4.7), but its token's epoch is not E, so comparison 7 refuses it; comparison 6 would too. Before entry nothing would have refused it (§7.2).
• Step 2. recovery-admin re-records P's identity revocation from the operator's records. P has been in the deployment's denied subjects since the revocation, so P was refused before entry too; automation tokens are reissued (§7.3 step 2).
• Steps 1 and 4. After the settle floor from the stop, recovery-admin records the accounting decision for M1, M2 and M3, each with its recovery observation; for M3 it accounts S's attempt too, whose response the restore erased (§5.2). restoration observations follow. M1 reads a; M1's restored Applied is the pre-A release, so a drift record opens on M1 (§6.1). r4 does not exist in the restored database, and M1 is matched by digest only (§7.1 item 3). M2 reads its pre-dispatch digest; C does not exist in the restored journal.
• Step 5. A completion observation of M3, taken after that accounting, reads s: S completes and Applied becomes s. Had it read S's pre-dispatch digest, S would end failed rather than retry, since its approval's epoch is not E (§5).
• Steps 3, 6 and 7. M2's artifact key version is present and decryptable under the executor identity: M2 is ready and released. M1 and M3 are ready and released, and M1 is then handled under §6. Had the provider been restored from a backup older than the database's, against §7.7's pairing duty, a machine whose Desired release names a key version that provider lacks would be blocked, naming that version (KL §3.1 case H, §5.3), until the version is restored or a newly published release is selected for it (§7.4).
• During recovery mode. Drafts, ingestion and publication continue for the whole installation, and released scopes are fully usable (§7.5). A plan for M2 needs an approval recorded in E. If C's held request lands after that plan completes, the next drift observation of M2 shows it (§8.4).
• Leave. Once every scope is released, recovery-admin leaves recovery mode (§7.6).

9. Verification and evidence limits

Design: §18.1 (E1, E3, E4, E6), §18.2.

9.1 Transition evidence

The issue's verification walks each transition against the dispatch mechanism E4 proved and the selected policies:

Transition or rule	Status	Basis
Commitment boundary; revocation before, during and after it	evidenced	DS rows 002–006
Scope index; A-after-B stopped while A is uncertain	evidenced	DS rows 010, 011
Takeover fence against a stale owner	evidenced, database only	DS rows 007, 008; DB §4.4
Second executor for the same plan	evidenced	DS row 009
Uncertain send held unresolved; refusal while unaccounted	evidenced	DS rows 012–015, 020
Accounting of a lost response	unsupported: no bound on late landing; operator decision (§5.2)	DS §6.3; FR §9 item 3
Retry after accounting	evidenced for the accounting E4 made	DS rows 012, 014, 015, 019
rejected on InvalidArgument	evidenced for a validation error only	DS row 022
failed	only on untrue accounting	DS row 013
Reconnects during recording and verification	evidenced	DS rows 016, 017
Interruption at each gate	evidenced	DS rows 018–021
Plan expiry, observation age, contradicting observation, attempt bound	unsupported: not exercised	DS §7
Plan cancellation, before and after commitment	unsupported: no row cancelled a plan	DS §4.1
Assignment change refused while the scope is held (invariant 9)	unsupported: not exercised	none
Scope gate (comparison 6) under a freeze or recovery mode	unsupported: in the prototype's code, exercised by no row	DS §7
Takeover at controller start	unsupported: E4's takeovers were harness commands	DS §2.1
Safe retry against failed on a pre-dispatch observation (§5)	unsupported: a harness flag in E4	DS §2.1
Identity revocation, including its refusal of retries	unsupported: not measured	design §13.7 item 4
Observation ordering under concurrent writers	unsupported: prototype basis unsound; this contract's read-start basis (§4.1) is not yet evidenced	DS §7
Machinery Talos client	evidenced: E4 used talosctl; the §3.5 re-run used the machinery client	§3.5; AP S4 precondition
Drift freeze, adoption record, revert	unsupported: no investigation	FR §9 item 1
Recovery-mode entry, quiescence, epoch, per-scope gating	unsupported: not modelled; fences rewind	DB §4.7; FR §9 item 4
Recovery dependency check	evidenced for eligibility, not application	KL §5.3, §6
Timeline as a usable surface	unsupported: not tested	FR §9 item 5

Everything evidenced holds within the limits stated at the top: one operation, mode, worker, assignment revision and Talos version; PostgreSQL only; rollout limit one; late landing observed through the control plane's proxy.

9.2 Required verification

An implementation of this contract must show, each with a control that can fail:

• DS's rows 001–023 re-run through the implementation's own controller and Talos client over the worker's own endpoint, rows 012–014 as the §3.5 direct-route variants, including its controls 006, 008 and 011, with the rows choice §10.32 names run as the Go checks it describes; if the fallback is taken, the same rows through it and the §3.5 scan of its channel;
• that it sends only after its attempt transaction commits, on every path;
• each lock of §3.2 and §3.3, with the unlocked control showing the race (design §7.7 consequences); comparison 1's approval and principal locks have none, and their race is shown alone (choice §10.31);
• plan expiry, observation age, a contradicting newer observation and the attempt bound, each refusing;
• plan cancellation before commitment and after it with no attempt;
• an apply-config operation with no recorded attempt not completed by a completion observation that shows its artifact (§4);
• an assignment change refused while an operation holds the scope;
• comparison 6 refusing under a freeze and under recovery mode without a release;
• takeover at controller start of every operation another instance owns;
• the §5 precedence: a retry admitted only with attempts left, a passing approval, an open gate and no definitive rejection, and failed otherwise;
• identity revocation before commitment, after it with no attempt, and after an attempt, with a retry refused;
• observation ordering under a concurrent accounting transaction (§4.1); a completion read that yields neither completed nor failed when a recorded higher-basis read differs from it on a bound postcondition, for each of the identity, the assignment revision, the digest and a bound health check, a completion read confirmed by repeated agreeing higher-basis reads that still yields failed, a revert refused after a manual return whose refusal closes the drift record as returned (§6.5), and an adoption read refused when a higher-basis read contradicts it (§4, §6.3); and, as the residual's evidence, a completion and an adoption committed while a contradicting higher-basis read is still unrecorded, whose late result opens no drift record while the next drift observation does;
• plan creation refusing a plan that binds no drift record and expects a digest other than Applied's, and a commitment whose evidence shows an undetected out-of-band change refused, with a drift record opened by the transaction that records the refusal (§2, §3.2);
• a bound health check failing in a completion observation before the verification deadline and passing in a later one: completed; still failing at or after the deadline: failed; unreadable at the deadline: unresolved; and a contradicting digest before the deadline: failed at once (§4); the health cases wait for an operation class that binds a check (choice §10.33);
• plan creation refusing an apply-config plan whose release is not the machine's Desired, a commitment refused when another release was published after plan creation, a publication racing a commitment waiting for it or preceding it, with the unlocked control, and a publication after commitment leaving the operation's attempt admitted (§2, §3.2 comparison 2, §3.3);
• a commitment and a retry's attempt each refused, with nothing sent, when the §3.1 evidence shows an SMBIOS UUID other than the bound one, none for a machine recorded by SMBIOS UUID, one for a machine recorded by Talos node ID, another Talos node ID for such a machine or another Talos cluster ID; nothing committed or sent when its identity read fails, a failed systeminformation read included, which leaves the identity unread and so is missing evidence (choice §10.28); with the controls that a machine recorded by node ID whose systeminformation read returns no resource is admitted, and that a machine recorded by SMBIOS UUID whose node shows a matching UUID and cluster ID but another node ID is admitted; each SMBIOS case against a stubbed Talos response, since the fixture's nodes report none; and a scope left blocked, not ready, by a restoration observation showing another identity, then marked ready after a matching one (§3.2 comparison 3, §7.4, choice §10.26);
• drift detection (including no record from a read begun before an Applied change, none opened or closed by a read that a recorded higher-basis read supersedes, and a record opened with a failed operation), freeze, adoption record (success, stale observation, changed-again machine, another machine identity, revoked approval, changed baseline revision, changed Desired selection, drift record closed by a revert) and revert (success, re-drift before commitment), with E1's leak screen over adoption's backup-visible surfaces;
• recovery-mode entry after a database restore to a snapshot older than a takeover, an approval, a revocation and an attempt: every request but liveness and entry refused under recovery start before entry (§7.2), a stale instance refused by the epoch, every scope pre-restore unaccounted, pre-entry approvals refused, a restored operation not retried, approval refused on a pre-restore unaccounted scope, release refused for a scope not ready, a released scope usable while another is not, and a blocked scope cleared by a new release; and
• the complete existing-cluster slice of design experiment E6.

9.3 Gaps carried, and what would close them

These are the evidence review's gaps that this contract must address (FR §9 items 1, 3, 4 and 5). Their implementation evidence is mapped in the PoC acceptance record, including the separate live and Go checks and the dated delegated acceptance assessment. The independent timeline review and the residuals below require their own evidence; passing CI or that mapping alone does not close them.

1. Drift freeze, sanitized adoption and approved revert (item 1). Closed by an E6-style run on the fixture: an out-of-band talosctl change detected and frozen; a drift adoption through the compilation pipeline with E1's screen over every backup-visible surface on success, rejection and interruption; an adoption record refused on a stale or changed observation; a revert that completes and one refused by re-drift; and a measurement of whether an artifact compiled from an unchanged import base reproduces the baseline digest (§6.3).
2. A bound on when an abandoned request can no longer land (item 3). Closed by evidence of the proxied request's lifetime: the Talos control plane proxy's forwarding and cancellation behaviour, established from its source and then measured across partition lengths, pauses and connection reuse, with enough captures to state a distribution rather than one landing; and whether a dial failure proves no send. Until then accounting is an operator decision (§5.2), and a late landing is detected as drift, not prevented. The measurement is recorded in the late-landing report: on that fixture no request landed after a partition of 45 s or more, consistent with transport defaults Talos does not configure and with kernel behaviour the report assumes; a pause of the worker keeps the request waiting with no such cut-off, though none landed; and a failed dial sent nothing. It supplies no bound for §5.2, so this item stays open and accounting stays as above.
3. Recovery-mode entry and executor quiescence after restoration (item 4). Closed by a restoration run on the fixture: a database restored to a snapshot older than a takeover, an approval, a revocation and a held attempt, with a missed stale instance; showing the epoch refusing it, entry gating every scope, the held request's landing detected, and the scope states, per-scope rules and release of §7.4 and §7.5. DB §4.7 closes only the negative half: fences alone do not survive a restore.
4. The operation timeline (item 5's timeline). Closed by the E6 slice exercising the timeline as an operator reads it: approval, commitment, attempts, observations, accounting and outcome linked for one change, one interruption, one drift and one restoration, with the timeline's response text passed through the compilation contract's redaction.

Narrower gaps: plan expiry, observation age and the attempt bound (DS §7); plan cancellation, the assignment-change refusal, comparison 6 and takeover at controller start (§9.1); observation ordering under concurrency (DS §7); InvalidArgument beyond a validation error, and every other response class (DS §4.6); identity revocation (design §13.7); the machinery client (§3.5); the backup age combinations g1/g2/g1 and g2/g1/g2, a restore onto a new OpenBao cluster and token expiry across a restore (KL §6). A specification gap, not an evidence one: no act decommissions or excludes a machine that can never be observed again, so its scope keeps the installation in recovery mode, and a restored operation on it with an attempt keeps its rollout slot (§7.6). A machine whose node now reports another identity likewise keeps the installation in recovery mode, its scope blocked (§7.4), though a restored operation on it with an attempt fails on that contradicting completion observation and releases its slot (§4, §5). A second specification gap: completion and adoption consult recorded observations only (§4, §6.3), so a machine change read by a started but unrecorded higher-basis observation is neither refused nor recorded as drift; Applied names a digest the node no longer runs until the next drift observation opens a drift record, and a late contradiction of the identity, the assignment revision or a bound health check alone neither reopens the completed operation nor opens a drift record. No later apply-config plan overwrites that change unseen: with no drift record open, every such plan expects Applied's digest (§2), so its commitment is refused when its §3.1 evidence reports the running one, and that difference opens a drift record (§3.2). Closing it needs a rule that retires an interrupted read, so that completion can wait for outstanding reads without waiting forever.

Until these close, an implementation must expose unresolved outcomes and stop conflicting work rather than claim safe retry beyond §5.1, exactly-once execution, universal stale-worker prevention, a bound on late landing, or detection of an out-of-band change overwritten inside the §3.3 residual window, or refusal of a completion or adoption that an unrecorded read contradicts.

The Upgrade/LifecycleClient compatibility deferral remains explicit: this contract does not claim full E3, upgrade support or lifecycle execution.

10. Choices for owner review

Each settles, for the stated reason, a question that neither the design nor the evidence settles, consistently with the design. Most take the most conservative option; those that do not say so. Each is marked in place as (choice §10.n).

1. Ignore is not a PoC drift policy (§1, §6). Alternative: support design §12.4's bounded Ignore. Which role may ignore is an owner decision the design leaves open (design §13.7, "Limits and what stays open"), and design §18.2 does not require Ignore.
2. Configuration digest is SHA-256 over the normalized read-back (§1). E4 and the fixtures measured it; compilation records it for artifacts and baselines (compilation contract choice §16.26). Owner decision, 2026-09-27 (ginsys/bronzeward#20), with that choice. Alternative: a keyed digest under the compilation contract's HMAC key, which removes any guessing oracle at the cost of a provider call per observation and an unevidenced primitive.
3. The operation is created by the dispatch commitment; before it the plan is the handle (§2, §3.2). The design (§11.1, §12.5) does not settle when. It keeps comparison 0 as E4 ran it (DS row 009) and gives no operation to a change that was never dispatched. Alternative: create the operation with the plan, and make the commitment its conditional approved → committed transition.
4. Self-approval marked for any contained authored revision, however old, and for a token's recorded responsible human; "undetermined" only as a guard (§2). Alternative: mark only content changed in this release, and ignore automation tokens; fewer marks, less visible.
5. Rollout limit fixed at one (§3.2). Only limit one was exercised. Alternative: design §12.3's configurable worker concurrency.
6. Identity revocation refuses every later attempt, retries included (§3.3). Design §13.7 item 4 invalidates only approvals no attempt has used; a retry is a new send under the revoked identity's authority. Owner decision, 2026-09-27 (ginsys/bronzeward#20), with persistence contract choice §17.19. Alternative: let an approval already used by an attempt stay valid for that operation's bounded retries, unless the approval itself is revoked.
7. Takeover at controller start and on recovery-admin request only; no timer (§3.4). Alternative: a lease that expires into takeover, which E4 did not build and which cannot stop a sent request either.
8. Machinery Talos client in process, conditional on re-running DS rows; the fallback under the same condition (§3.5). Chosen to keep the decrypted artifact in one process, not as the most conservative option: neither the client nor the fallback's descriptor channel is what E4 measured. Alternative: the pinned talosctl subprocess with --file, as E4 ran it, which puts the plaintext artifact in a named file. Owner decision, 2026-09-27 (ginsys/bronzeward#20): in process; a failed DS re-run selects the fallback without reopening this choice; only a failure of the fallback too returns it to the owner.
9. The route is bound in the plan; only the target machine's own endpoint is allowed in the PoC (§3.5). Owner decision, 2026-09-27 (ginsys/bronzeward#20): it removes the control plane's proxy, through which every observed late landing came, from the PoC's dispatch, until §9.3 item 2 measures it. The cost is that a worker unreachable on its own endpoint cannot be configured. Alternative: both E4 routes, for route flexibility, which keeps that proxy in the dispatch path with no bound on its late landings.
10. On a pre-dispatch observation after full accounting, retry only while attempts remain, the approval passes and the gate is open; otherwise failed (§5). E4 left it to a harness flag. Alternative: always failed, requiring a new plan; or stay unresolved until the gate reopens, which keeps the scope held.
11. Accounting a lost response needs a recovery-admin decision after a settle floor of at least 30 s, held outside the database (§5.2). Owner decision, 2026-09-27 (ginsys/bronzeward#20): 30 s, with choice §10.9 keeping the proxy, the only path seen landing late, out of dispatch; the §9.3 item 2 measurement is the input for any change. Alternatives: no floor, leaving it to judgement; a longer floor; or a supervisor time bound, which DS §8 says would need its own evidence.
12. Drift detection does not freeze by itself (§6.1). Design §12.4 reports and lets the operator choose. Alternative: freeze on detection.
13. An adoption is requested as a plan with operation adopt, whose commitment is the adoption record (§6.3). It reuses the plan binding, approval, revocation and epoch rules. Alternative: a separate adoption approval resource with its own rules.
14. author performs the ingestion that feeds an adoption (§6.3). Design §13.7 left the ingestion role to the owner, who has decided it (below). The adopt plan is a publisher's like every plan (design §13.7 item 2), and its approval an approver's. Owner decision, 2026-09-27 (ginsys/bronzeward#20): final, with persistence contract choice §17.22; automation does not ingest. Alternative: publisher, or a new privileged-ingestion role.
15. The adoption record compares with the baseline's digest, not the recompiled artifact's (§6.3). Alternative: require the artifact to reproduce the node's bytes, as the prior draft did; unevidenced, and E3's re-encoding suggests it may never hold.
16. A revert's approval is recorded as approving an unseen overwrite (§6.4). Alternative: an ordinary plan with no such mark.
17. The recovery epoch is a never-reissued random identifier compared for equality (§7.1). Alternative: a counter held outside the database, which another restore could also rewind.
18. Recovery start keeps the gates closed and serves nothing but liveness and entry until entry commits; entry is always the recovery-admin API act; no rollback detection (§7.2). Alternatives: entry by a server-side command before the controller starts; or detection through a high-water mark held outside the database, which this contract does not specify.
19. Every scope is pre-restore unaccounted at entry (§7.2). Alternative: only scopes with restored non-terminal operations, which misses operations dispatched after the snapshot.
20. A scope is blocked on the dependencies to apply its Desired release, not those to regenerate it (§7.4). A new release clears it. Alternative: release a scope with the missing dependency recorded, relying on the use-time check of each plan.
21. Recovery mode is enforced per scope (§7.5): authority-removing and recovery acts always; drafts, ingestion and publication installation-wide; approval, adoption and dispatch refused on a pre-restore unaccounted scope; released scopes usable as outside recovery mode. Alternative: refuse every mutation installation-wide except those acts until exit, which with choice §10.22 lets one scope hold the whole installation.
22. Leaving recovery mode needs every scope released (§7.6). Alternative: leave and freeze the unreleased scopes, which would hand their unfreezing to approver, outside the recovery role.
23. An ordinary plan cannot overwrite undetected drift (§2, §3.2). A plan that binds no drift record expects Applied's digest, and a commitment whose evidence shows another is refused and opens a drift record, so the operator makes design §12.4's freeze, adopt or revert choice before the change is overwritten. Alternative: let a plan expect the running digest and rely on the approver's review of the diff against it, which lets an unseen out-of-band change be overwritten without that choice.
24. Every plan binds the machine's Desired release; an apply-config plan only the current one, and commitment refuses once it changed (§2, §3.2). Publication selects Desired (persistence contract choice §17.6), so without the binding a plan approved before a later publication could still commit and apply a release the machine no longer desires; adopt plans already worked this way (§6.3). Owner decision, 2026-09-27 (ginsys/bronzeward#20). The cost is that every publication covering a machine invalidates its approved, uncommitted plans, which must be planned and approved again; a committed operation is not stopped (§3.3). Alternative: bind only the artifact and let the approver's plan stand until it is committed or expires.
25. A bound health check has a convergence window up to the verification deadline; the identity, assignment revision and digest have none (§4). A health check can be briefly red after a no-reboot apply that landed; failing at once would leave Applied behind the node and open a drift record for a change that succeeded. Once every attempt is accounted for, waiting cannot change the other postconditions. Owner decision, 2026-09-27 (ginsys/bronzeward#20). E4 bound no health check, so the window is unevidenced. Alternatives: fail on the first contradiction of any postcondition, as the prior text did; or a window for every postcondition, which only delays outcomes already known.
26. Machine identity is a precondition: the node's identity key and Talos cluster ID must match the machine record before any send and before a scope is ready (§2, §3.2 comparison 3, §7.3 step 6, §7.4). As a postcondition only, a route that reaches another node, for instance after address reuse, would configure it and detect the mismatch afterwards, and any successful restoration read would make a scope ready whichever node answered. Owner decision, 2026-09-27 (ginsys/bronzeward#20), for the precondition; the identity it compares, owner decision, 2 October 2026 (ginsys/bronzeward#22):
    • The identity key is fixed at inventory, per machine. It is the machine's SMBIOS UUID when the machine reports one, and otherwise its Talos node ID (see persistence-api.md). There is no fallback at read time: a machine recorded by SMBIOS UUID whose node reports none, or whose identity read fails, is refused, never compared by node ID, so a failing read cannot weaken the check; and a machine recorded by node ID whose node reports an SMBIOS UUID is refused as well, so a machine that reports one is held to it. A node reports none when its systeminformation read succeeds and returns no resource, as on the fixture; an error any identity read returns on a connection the node has accepted is a failed read, and refuses whichever key the machine is recorded by. A credential the node refuses, or an endpoint that does not answer, fails before any read, as the access failure persistence-api.md defines. At a commitment, a failed read leaves the identity unread, which choice §10.28 treats as missing evidence: nothing is committed, sent or recorded as a refusal, and fresh evidence decides; a failing read still never admits a commitment.
    • The cluster is compared by its Talos cluster ID, the value the cluster record holds (see persistence-api.md), against the node's Infos.cluster.talos.dev resource. Talos describes it as the cluster's globally unique identifier, distinct from the shared cluster.secret (machinery v1.13.6, config/types/v1alpha1/v1alpha1_types.go).
    • What each reports. The node ID is Talos's Identities.cluster.talos.dev value, random, kept across reboots and reset on a wipe (machinery v1.13.6, resources/cluster/identity.go); the SMBIOS UUID is the systeminformation resource. On the Docker fixture (Talos v1.13.6, platform container) talosctl get systeminformation returns no resource on either node, get identity a distinct node ID per node, and get info one cluster ID on both. On one bare-metal node of the operator's office cluster (Talos v1.13.8) all three are present. Each was read with talosctl under os:admin, 2 October 2026, not yet through the application's read-only client, which #22 and #26 show.
    • Limits. The fixture exercises only the node-ID key; the SMBIOS-UUID key is checked against a stubbed Talos response until a scenario runs on hardware. A wipe gives a node-ID machine a new node ID, so it is refused until it is inventoried again; the PoC excludes reset and reuse. SMBIOS's nil and all-ones values count as a reported UUID: the first machine reporting one is inventoried under it, and a second machine reporting the same value, like a clone sharing a UUID, cannot be inventoried under it; recorded under its node ID instead, it is refused at every comparison, since its node reports a UUID; and reached at the first's endpoint it would pass the first's comparison. Two clusters generated from one Talos secrets bundle share a cluster ID and a certificate authority, so only the first can be recorded; the second is refused at inventory. A node swapped between the evidence read and the send stays a residual.
    • Alternatives. The SMBIOS UUID only, which the fixture cannot report, leaving S1's inventory and this check unrunnable there; the node ID only, which is readable everywhere but drops the hardware key a reinstall keeps; a fixture on the QEMU provisioner, which needs root and KVM; a Bronzeward-issued per-machine marker written into the configuration, stronger against clones but a new mechanism with no evidence; or disclosure only.
27. The controller records an adoption unasked, once per plan (§6.3 step 4). The record needs an observation begun after the approval, so the controller takes one for each approved adopt plan, after each approval and periodically, and attempts the record on it. It waits while an operation holds the scope or recovery mode closes it, since those clear without a new plan, and observes again when the evidence is missing: no observation, or the one the record relies on (another reader's included) began before the approval, is too old or left a value unread, so missing evidence never records a refusal. An attempt that finds the plan already adopted or refused by another records nothing. A refusal ends the attempts: a digest, identity or assignment mismatch does not heal by repeating the comparison, and repeating it would append a refusal entry every period until the plan expires. Agent decision, 2026-10-08 (ginsys/bronzeward#25). Alternatives: an operator route that requests the record (one more route and role decision, and the operator would still need a fresh observation first); retrying refused plans until they expire.
28. A commitment reads its evidence as an adoption does, and its refusal is not the plan's outcome (§3.2). The evidence is the latest observation recorded with purpose evidence for the plan by the committing controller. None, one begun before the approval, one older than the plan allows, or one that left the identity, the configuration or the running version unread is missing evidence, not a comparison-3 failure: nothing is committed or refused, and fresh evidence decides. A newer observation contradicts the evidence only on a value it read. The observed assignment evidence is not compared, since comparison 2 compares the assignment head itself. A refusal entry names the observation that showed a mismatch, never the digest (persistence and API choice §17.38). Each refused commitment records its own refusal entry: a freeze lifted, a scope released or another operation finished lets the same plan commit, so a refusal does not end the plan as an adoption's does (choice §10.27). Until drift records exist (ginsys/bronzeward#27), a digest refusal applies no §6.1 step. Agent decision, 2026-10-09 (ginsys/bronzeward#25). Alternatives: refusing on missing evidence, which appends a refusal on every attempt that precedes its observation; one refusal per plan, which leaves a plan refused by a freeze unable to commit once the freeze is lifted.
29. A commitment waits for the rollout scope before it reads its time (§3.2 comparison 5). Each commitment takes a transaction-level lock on its cluster's rollout scope after its other locks and before it reads the time (persistence and API §5 rules 4 and 5). Another machine's commitment that holds the slot uncommitted is waited for there, so expiry and evidence age are judged at a time after the wait, and the slot it took is seen by comparison 5. Only a commitment enters the rollout scope, so the partial unique index is never waited on. Agent decision, 2026-10-09 (ginsys/bronzeward#25). Alternatives: waiting at the index after the time is read, which commits a plan that expired during the wait; reading the time again after the insert, which the operation's fixed creation time cannot follow.
30. An attempt reads its time after comparison 7's write, and its refusal settles the operation separately (§3.3, §4). The attempt transaction takes §3.2's locks except the rollout scope, then makes comparison 7's conditional UPDATE on the operation's row, the last lock it takes, and only then reads its time (persistence and API §5 rule 4). The comparisons are still judged in their numbered order, and any refusal rolls the write back. A first attempt compares the observation its commitment recorded, at the attempt's time, and any newer observation that contradicts it on a value it read. A refused attempt is recorded by a separate transaction under the same epoch check and machine lock, followed, after a refusal it may settle, by the operation's lock, all before it reads its time; the entry names the controller and the owner generation it submitted. After a refusal by 1, 2, 3 or 6, that transaction moves a committed operation its controller still owns to unresolved, and an unresolved one it owns, so moved now or by an earlier refusal, on to cancelled when no attempt exists and comparison 1, re-read there, still fails. A refusal by 7 or 9, or by a controller that no longer owns the operation, records the refusal only. Agent decision, 2026-10-09 (ginsys/bronzeward#25). Alternatives: reading the time before the operation's lock, which judges expiry and evidence age before a takeover holding that row has committed; settling inside the refused transaction, which its rollback would undo.
31. The approval and principal locks of comparison 1 have no unlocked control (§3.2, §9.2; DS row 003). A commitment or attempt also holds the machine row FOR UPDATE, and a commitment the plan state FOR UPDATE. An approval revocation takes the machine row before the approval and the plan state after it, and an identity revocation takes every machine row before the principal. A revocation that finds the approval's or principal's row unlocked therefore still waits, and §9.2's unlocked control is not constructible for these two locks. The verification is the race itself: a revocation started while the commitment holds its locks waits for it. Then either the attempt is refused, or the attempt takes the machine row first, is recorded, and the revocation waits for it too (DS row 003). Either way no attempt is recorded after a committed revocation; an attempt recorded before it is sent, as in DS row 005. DS row 006's control places the revocation after the commitment, before the attempt: before the commitment, the plan's state, revoked, is terminal (§2) and the database refuses moving it to committed, so no controller commits it with or without comparison 1. The FOR SHARE reads stay as the contract's stated order for a writer that does not take the machine row. Agent decision, 2026-10-14 (ginsys/bronzeward#26). Alternatives: a control dropping the machine and plan-state locks as well, which would no longer test the approval's lock; dropping the FOR SHARE reads as redundant, which couples comparison 1's ordering to every revocation's lock order. Amended 2026-10-28, lead decision at 19:02 CET (ginsys/bronzeward#431): the outcome had been stated as the refusal alone. A waiting locker does not keep its place across an update on PostgreSQL. The commitment updates the machine row (its timeline entry's revision), so once it commits, the revocation that waited on the old row version has to lock the new one (READ COMMITTED's update-chain recheck, EvalPlanQual). An attempt transaction started after the commit can lock that version first. TestExecutorRevocationHeld/attempt-first forces this order. Rejected alternative: making the refusal certain by keeping the revocation ahead of any later attempt. Every way of doing that works against PostgreSQL's row-lock semantics, and safety does not depend on it.
32. DS rows that need an interleaving inside a transaction run as Go checks (§3.5, §9.2). Rows 003, 009, 010 and 023 and the controls of rows 006 and 008 act inside a commitment or attempt transaction, need two executors ordered at a point inside one, or remove a guard the production path keeps: a revocation waiting on the commitment's locks (003), a second executor inside the commitment (009), comparison 4 on a plan the executor skips while the scope is held (010; live, it needs two controllers that selected different plans, one held between its evidence and its commitment while the other commits), a retry's classification (023), and the unchecked or unlocked variants (006, 008). This is the selected test route, not a claim that no fault could reach them: a Go check fixes the interleaving that a live run would have to arrange across processes. Row 003 is also one of §3.5's condition rows; when its revocation takes the machine row first, the attempt is refused before any mutating Talos request, so a stand-in meets it (§3.5). The other order's outcome is row 005's, which runs live (choice §10.31). Each is a Go check driving the implementation's transactions, or its executor with a Talos stand-in, against PostgreSQL. Row 011's control is the schema's, not a transaction's: with both scope indexes dropped, a second plan's operation rows insert directly; it shows the indexes are the database's guard, while the commitment transaction's own comparison 4 refuses that plan independently (row 010's check). Row 022 needs an artifact the node refuses as invalid, and compilation validates every artifact before publication, so no plan carries one: the machinery client sends such a configuration to the fixture's worker directly, and a check records its classification as rejected. The other rows run live through the controller. Agent decision, 2026-10-14 (ginsys/bronzeward#26). Alternatives: seams inside the store's transactions reachable from a running controller, which ship test-only code on the production path; a compilation bypass to publish an invalid artifact, which removes the boundary row 022 stays behind.
33. The PoC's apply-config plans bind no health check (§3.1, §4; choice §10.25). Design §18.2's working configuration-control PoC verifies the result by identity, assignment revision and configuration digest; §3.1 binds a health check only where an operation class names one, and the no-reboot apply names none. Every observation therefore records health as none-bound, and §9.2's bound-health cases (a failing check inside the convergence window, at or after the verification deadline, unreadable at it) apply to the first operation class that binds one, such as the upgrades §18.2 excludes, and are tracked by ginsys/bronzeward#188. The contradicting-digest case, its immediate failed and the drift record that outcome opens hold now. Agent decision, 2026-10-14 (ginsys/bronzeward#26). Alternatives: binding a Talos health check to the PoC's apply, a new precondition and fixture surface no PoC acceptance row asks for; leaving the cases open against #26, which would make an unbound check a closure blocker.
34. A freeze and an unfreeze record a required reason and refuse a no-op (§6.2). Design §13.7 asks for recorded operator intent; the routes had no body, and §9.1 refuses unknown fields, so the contract names one: a reason, as an approval revocation takes. A freeze on a frozen scope, or an unfreeze on one that is not frozen, is 409 conflict, so the timeline holds only acts that changed the scope. Agent decision, 2026-10-15 (ginsys/bronzeward#27). Alternatives: an empty body, which records who but not why; an idempotent no-op answering 201, which writes timeline entries that change nothing.
35. A drift record is a projection of drift timeline entries, and its opening entry is the digest-mismatch alert (§6.1, §6.5). Each change of the record is one drift entry naming the record (by its opening entry's revision), the change (opened, observed, closed) and the observation it rests on; an opening or a later differing observation names a possible late landing, a closing its resolution. The record keeps the opening observation, Applied's digest, the latest differing observation, its digest and possible late landing, and the closing. Design §15.3 lists the digest-mismatch alert but no delivery channel, so the PoC raises it as the opening entry on the machine's timeline, and serves the open record as the machine resource's openDrift (persistence-api.md §9.3): its opening revision, observation, latest observation and late landing. Agent decision, 2026-10-15 (ginsys/bronzeward#27). Alternatives: a separate alert table or notifier, a delivery surface no PoC acceptance row asks for; one entry kind per change, which triples the kinds a timeline reader must know.
36. The drift interval is one configured period (§6.1). Every controller with an executor runs a drift loop: one interval after it starts and every execution.driftInterval after that (default 5 minutes, at least 1 second), it takes a plan-less drift observation of each machine that has an Applied, whose scope no operation holds and that recovery mode leaves open, as the adoption loop reads that scope. The transaction recording the observation applies §6.1 and §6.5 to it. The loop skips the machines whose observation the basis rules would exclude anyway, and a pass does not run at startup, so a restart reads no machine sooner than an interval later. A failed read is logged and retried at the next pass. Agent decision, 2026-10-15 (ginsys/bronzeward#27). Alternatives: a per-machine schedule spreading the reads, which the PoC's machine count does not need; a pass at startup, which would read every machine on each restart of a fixture run.
37. A drift adoption is an ingestion of kind drift-adoption that needs an open drift record (§6.3 step 1). POST /ingestions takes "kind": "drift-adoption" with "source": "machine" only, since the configuration adopted is the one the node runs; a request for a machine with no open drift record when the ingestion starts is 409 conflict naming the machine. Otherwise it runs as an import does, and its claim and ingestion resource carry the kind. The ingestion binds no drift record and compares no digest: the adopt plan binds the record, and the adoption record requires it still open and the observed digest equal to the baseline's (requirements 4.3 and 4.4). Agent decision, 2026-10-15 (ginsys/bronzeward#27). Alternatives: no kind, an import of a drifted machine, which leaves the ingestion's purpose unrecorded; binding the record at ingestion, which the record re-checks anyway.
38. An adopt plan of a machine with an Applied needs an open drift record, which it binds with the baseline revision (§6.3 steps 3 and 4). Plan creation for such a machine with no open record is 409 conflict; with one, the plan binds the record by its opening entry's revision and the machine's baseline revision, and still no expected digest, since the adoption record compares with the baseline's. The record checks the bound record still open under the machine lock, advances the baseline revision, names the previous Applied release in its applied-change entry, and then appends a drift entry closing the record as adopted, resting on the observation the record relied on. Agent decision, 2026-10-15 (ginsys/bronzeward#27). Alternatives: binding no record and closing whichever is open at the record, which could adopt over a later drift a revert had already resolved; refusing the record instead of the plan when no record is open, which admits plans that can never be recorded.

11. Traceability

Clause	Design	Evidence
§1 operation, exclusions	§12.2, §18.2	DS §6.3; E3 §4.3, E3 §6.3
§1 state values, digest	§12.1	Fx §4 check 3; DS §2.3
§2 plan binding	§12.2, §12.7	KL §7 item 1; E3 §7
§2 plan states	§11.1, §12.5	DS §4.1 row 002, DS §4.2 row 009
§2 approval, revocation, cancellation	§13.7 items 2–4, 6	none: policy (FR §9 item 5); DS row 003 for the approval lock
§3.1 execution-time evidence	§7.6, §7.8, §12.7	KL §5.2
§3.2 commitment	§12.7, §7.7	DS §4.1, DS §4.3; DB §4.2, DB §4.3
§3.3 attempt, residual	§12.7, §13.7 item 4	DS §4.1, DS §4.4, DS §6.2; DB §4.4; FR §7
§3.4 takeover fence	§12.5	DS §4.2, DS §8; DB §4.4
§3.5 client, route, response text	§7.1, §12.2	E3 §4.3, E3 §6.2, E3 §7; DS §2.1, §4.4
§4 states, accounting	§12.5, §13.7 item 5	DS §4.4, DS §4.5, DS §6.3
§4.1 timeline	§13.6, §15.2, §7.8	DS §7; FR §9 item 5
§5 classification	§12.5, §12.6	DS §6.3, DS §4.6; E3 §4.3
§5.2 accounting a lost response	§12.5	DS §4.4, §6.3, §8; FR §7; FR §9 item 3
§6 drift, adopt plans	§12.4, §7.1, §9.1, §12.1, §13.7 items 3 and 6	none: FR §6 SP22, FR §9 item 1; extraction only, E1 §6; E3 §7
§7 recovery, per-scope rules	§7.7, §7.8, §13.7 items 1, 4 and 5, §14.4, §14.6	DB §4.7, DB §9; KL §5.3, KL §6; FR §9 item 4
§8 interleavings	§12.5, §12.7, §13.7	DS rows cited in each walk-through
§9 gaps	§18.1, §18.2	FR §9, FR §10