Skip to content

E5 provider-capability experiment

This is Phase-0 evidence code for issue 8. It is not the v1 implementation and it will not be maintained past the research report it produces.

What it is for

Design §7.3 asks for an inventory before any provider is selected: secret creation, read and versioning, artifact encryption, any signing needs, key custody, startup unlock, rotation, metadata-only checks, backup/restore and migration. These scripts exercise that inventory, plus the permissions each operation needs, against three candidates:

  • OpenBao KV v2 and Transit, as the investigation fixtures run it
  • a local age-backed store, which the experiment builds itself under fixtures/.state/data
  • SOPS/age, as an alternative store and as an import/export mechanism

Every operation is one cell: one command, run under one identity, with its expected outcome written down before it runs. run/lib.sh records each cell as a row of cells.tsv with the observed outcome, derived from the exit status and the output alone, and keeps the whole output as a transcript. A cell whose observation differs from the expectation is recorded as a mismatch and the run continues; mismatches are findings for the report, not failures to retry past.

Each row also carries a kind, the distinction acceptance criterion 2 asks for:

kind meaning
primitive an encryption or storage primitive, with no version identity of its own
versioned complete versioned secret or provider behaviour
metadata an observation made from metadata alone, without the value
unsupported the candidate has no such capability; named, never left blank
- a procedural step (a seal, a crash, a snapshot) that the cells after it depend on; not a capability claim

versioned is reserved for behaviour the provider itself enforces. The local age store's generations are file names that only the writer's convention keeps immutable, and SOPS keeps no versions, so their cells are primitives.

Why shell, not Go

The issue's limits say the pinned age and sops are command-line versions and that a library integration is not evidenced by these runs. Driving the pinned CLIs is therefore exactly the claim being measured, and a program linking the libraries would measure something else.

Running it

A capture needs a fresh fixture, and E5_OUT must name an empty absolute path on disk, outside this checkout:

fixtures/bin/up
E5_OUT=<somewhere with ~100 MiB> experiments/e5-provider-capabilities/run/test-lib
E5_OUT=<the same directory>      experiments/e5-provider-capabilities/run/all
E5_OUT=<the same directory>      experiments/e5-provider-capabilities/run/collect-evidence
fixtures/bin/down

run/test-lib checks the cell recorder and e5_sops; it needs the sops and age binaries that fixtures/bin/up caches, not a running fixture. run/all refuses an E5_OUT that already holds a capture, and a fixture that still holds the local stores of an earlier one. run/collect-evidence needs the fixture still up, because it rebuilds its refusal patterns from this run's credentials.

E5_OUT has no default, for the same reasons as E1's: an evidence bundle carries a PostgreSQL data directory, a default under /tmp would put it in RAM on most systems, and one inside the checkout would be removed by fixtures/bin/down. Deleting it afterwards is part of finishing the run.

No secret value is ever an argument. Values go to the CLIs over stdin, as the fixtures' own canary does, because argv is readable by any local user.

What it deliberately does not do

It does not classify retained, blocked, lost or unknown dependencies (issue 9), and it does not restore backups of different ages in combination (issue 10). It shows that each capability exists, what permission it needs, and one snapshot/restore round trip per candidate.

It selects nothing: not a provider, not a physical layout, not a deployment profile. That decision belongs to issue 13.