Compliance hardening: enforce measured behavior, real gates, retained evidence

Addresses the attached findings as blocking compliance failures. Each fix
removes a substitution pattern and adds a negative-control test.

Finding 9 (hash changes != enforced metamorphic expectations):
  ci_reports now enforces a sound metamorphic relation per perturbation — a
  perturbation the program CONSUMES must alter the trace — grounded in
  reachability, with a non-vacuity check. Test:
  consumed_perturbation_must_alter_trace.

Finding 10 (trace counts != causal explanation):
  New causal_explanation gate ablates each recorded causal edge's source lane
  and requires the destination delta to change. Measured: true-source 0.77 vs
  scrambled-source 0.03. Threshold 0.50. New required report
  causal_explanation_report. Test:
  causal_edges_are_intervention_confirmed_not_counted.

Finding 6 (static seed corpus != failure retention):
  replay_corpus::retention adds a committed, append-only counterexample corpus
  (retained_failures.tsv) re-verified every run against the independent runtime.
  Negative control: reintroduced_bug_is_caught_by_retention.

Finding 5 (mini mutation evaluator != real acceptance gate):
  Mutants are now killed by ci_reports' OWN acceptance-gate predicates with
  single-sourced thresholds (TRACE_EDGES_MIN, etc.); evaluate_mutants replaces
  semantic_mutation::run_suite on the acceptance path. Test:
  mutants_killed_by_real_acceptance_gates.

Findings 2 & 3 (generated report != independent attestation; merkle root !=
provenance without leaves):
  ci_reports persists every Merkle leaf (evidence/leaves.tsv) + claims. New
  `attestation` crate + `attest` binary recompute the root from the leaves in a
  SEPARATE process that never reads compliance_report.json; wired as a distinct
  merge-gates step. Negative control: tampered_leaf_breaks_attestation.

Finding 8 (structural indicators != measured behavior):
  Domain gate already requires measured influence/mutation/removal; added an
  explicit reject for "appears structurally but no measured influence".

Finding 1 (workflow != merge enforcement): BLOCKED on server-side branch
  protection. Added .github/rulesets/main-required-checks.json + apply command;
  enforcement still requires a repo admin to activate the ruleset.

Finding 7 (protocol socket E2E != rendered browser E2E): BLOCKED on a CI browser
  runner; rendered-browser E2E remains advisory-only.

Finding 4 (trace summary != full trace evidence): PARTIAL. Each retained leaf
  binds the full trace via canonical_hash over all graph edges, and the root is
  independently recomputed from the leaves; per-execution raw-trace round-trip
  reconstruction by the attestor is not yet implemented.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-21 21:13:48 -07:00
co-authored by Claude Opus 4.8
parent 93c78d9c76
commit 1e50c80627
14 changed files with 1028 additions and 26 deletions
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "attestation"
version.workspace = true
edition.workspace = true
license.workspace = true
[[bin]]
name = "attest"
path = "src/main.rs"
[dependencies]
# Only the shared hashing primitive — NOT ci_reports. The root is recomputed by
# this crate's own code path, so it is an independent check, not a re-export of
# the producer's claim.
world_model = { path = "../world_model" }
+169
View File
@@ -0,0 +1,169 @@
//! `attestation` — an **independent** verifier of a CI run's evidence
//! (findings 2 and 3).
//!
//! The substitution it removes: a `compliance_report.json` written by the same
//! binary that ran the gates is self-certification, and a Merkle *root* reported
//! without its *leaves* cannot be checked by anyone. This crate is a separate
//! process with its own code path that:
//! * reads the retained per-execution leaves (`evidence/leaves.tsv`),
//! * recomputes the Merkle root from them with its own implementation,
//! * reads the producer's *claims* (`evidence/claims.tsv`) and checks the
//! recomputed root and leaf count match what was claimed,
//! * NEVER reads `compliance_report.json` — it does not trust the producer's
//! own pass/fail verdict.
//!
//! It depends only on `world_model` for the shared hash primitive. What it does
//! NOT do (and cannot, in-repo) is prove the leaves correspond to real
//! executions performed by a trusted third party — that requires external
//! re-execution / signing infrastructure (see the BLOCKED note in the report).
use std::path::Path;
use world_model::Hasher;
/// Recompute the Merkle root over `leaves` (binary tree, FNV-combined). This is
/// an independent re-implementation of the producer's algorithm; agreement is
/// the check.
pub fn merkle_root(leaves: &[u64]) -> u64 {
if leaves.is_empty() {
return 0;
}
let mut level = leaves.to_vec();
while level.len() > 1 {
let mut next = Vec::with_capacity(level.len().div_ceil(2));
for pair in level.chunks(2) {
let mut h = Hasher::new();
h.write_tag("merkle");
h.write_u64(pair[0]);
h.write_u64(if pair.len() > 1 { pair[1] } else { pair[0] });
next.push(h.finish().0);
}
level = next;
}
level[0]
}
/// Parse a leaves file: one hex u64 per data line, `#`/`leaf` headers skipped.
pub fn parse_leaves(text: &str) -> Vec<u64> {
text.lines()
.filter(|l| !l.starts_with('#') && !l.starts_with("leaf") && !l.trim().is_empty())
.filter_map(|l| u64::from_str_radix(l.trim(), 16).ok())
.collect()
}
/// Parse a `key<TAB>value` claims file into `(key, value)` pairs.
pub fn parse_claims(text: &str) -> Vec<(String, String)> {
text.lines()
.filter(|l| !l.starts_with('#') && !l.trim().is_empty())
.filter_map(|l| l.split_once('\t').map(|(k, v)| (k.trim().to_string(), v.trim().to_string())))
.collect()
}
fn claim<'a>(claims: &'a [(String, String)], key: &str) -> Option<&'a str> {
claims.iter().find(|(k, _)| k == key).map(|(_, v)| v.as_str())
}
/// Outcome of an attestation.
#[derive(Clone, Debug)]
pub struct Attestation {
pub ok: bool,
pub leaf_count: usize,
pub recomputed_root: u64,
pub claimed_root: Option<u64>,
pub checks: Vec<(String, bool)>,
}
/// Verify the evidence directory `dir` (which contains `evidence/`).
pub fn verify_dir(dir: &Path) -> Result<Attestation, String> {
let leaves_path = dir.join("evidence/leaves.tsv");
let claims_path = dir.join("evidence/claims.tsv");
let leaves_txt = std::fs::read_to_string(&leaves_path)
.map_err(|e| format!("cannot read {}: {e}", leaves_path.display()))?;
let claims_txt = std::fs::read_to_string(&claims_path)
.map_err(|e| format!("cannot read {}: {e}", claims_path.display()))?;
let leaves = parse_leaves(&leaves_txt);
let claims = parse_claims(&claims_txt);
let recomputed = merkle_root(&leaves);
let claimed_root = claim(&claims, "root").and_then(|v| u64::from_str_radix(v, 16).ok());
let claimed_count = claim(&claims, "leaf_count").and_then(|v| v.parse::<usize>().ok());
let claimed_comparisons =
claim(&claims, "total_comparisons").and_then(|v| v.parse::<usize>().ok());
let mut checks = Vec::new();
let root_ok = claimed_root == Some(recomputed);
checks.push(("recomputed_root == claimed_root".into(), root_ok));
let count_ok = claimed_count == Some(leaves.len());
checks.push(("leaf_count == claimed_leaf_count".into(), count_ok));
let cmp_ok = claimed_comparisons.map(|c| c == leaves.len()).unwrap_or(false);
checks.push(("leaf_count == claimed_total_comparisons".into(), cmp_ok));
let nonempty = !leaves.is_empty();
checks.push(("leaves are present (root has leaves)".into(), nonempty));
let ok = checks.iter().all(|(_, b)| *b);
Ok(Attestation {
ok,
leaf_count: leaves.len(),
recomputed_root: recomputed,
claimed_root,
checks,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn root_matches_known_vector() {
// Mirrors ci_reports::merkle_root over the same leaves.
let a = merkle_root(&[1, 2, 3]);
let b = merkle_root(&[1, 2, 3]);
let c = merkle_root(&[1, 2, 4]);
assert_eq!(a, b);
assert_ne!(a, c);
}
fn write_evidence(dir: &Path, leaves: &[u64], claimed_root: u64) {
let ev = dir.join("evidence");
std::fs::create_dir_all(&ev).unwrap();
let mut lt = String::from("leaf\n");
for l in leaves {
lt.push_str(&format!("{:016x}\n", l));
}
std::fs::write(ev.join("leaves.tsv"), lt).unwrap();
let claims = format!(
"# claims\nroot\t{:016x}\nleaf_count\t{}\ntotal_comparisons\t{}\n",
claimed_root,
leaves.len(),
leaves.len()
);
std::fs::write(ev.join("claims.tsv"), claims).unwrap();
}
#[test]
fn honest_evidence_attests() {
let dir = std::env::temp_dir().join("magicka_attest_ok");
let leaves = [10u64, 20, 30, 40, 50];
write_evidence(&dir, &leaves, merkle_root(&leaves));
let att = verify_dir(&dir).unwrap();
assert!(att.ok, "honest evidence should attest: {:?}", att.checks);
}
/// Negative control: a tampered leaf changes the recomputed root, so the
/// claimed root no longer matches and attestation FAILS. Proves the
/// attestation is not vacuous and that the root genuinely binds the leaves.
#[test]
fn tampered_leaf_breaks_attestation() {
let dir = std::env::temp_dir().join("magicka_attest_bad");
let leaves = [10u64, 20, 30, 40, 50];
let claimed = merkle_root(&leaves);
// Write evidence whose leaves were altered after the root was claimed.
let mut tampered = leaves.to_vec();
tampered[2] ^= 0xdead_beef;
write_evidence(&dir, &tampered, claimed);
let att = verify_dir(&dir).unwrap();
assert!(!att.ok, "tampered leaves must fail attestation");
assert!(att.checks.iter().any(|(n, ok)| n.contains("root") && !ok));
}
}
+36
View File
@@ -0,0 +1,36 @@
//! `attest` — independently verify a CI run's evidence directory.
//!
//! Usage: `attest <dir>` (default `ci_out`). Exits non-zero if the recomputed
//! Merkle root does not match the claimed root, the leaf count is inconsistent,
//! or the leaves are missing. It deliberately ignores `compliance_report.json`.
use std::path::PathBuf;
fn main() {
let dir = std::env::args().nth(1).unwrap_or_else(|| "ci_out".to_string());
let path = PathBuf::from(&dir);
match attestation::verify_dir(&path) {
Ok(att) => {
eprintln!("=== independent attestation of {dir} ===");
eprintln!(" leaves: {}", att.leaf_count);
eprintln!(" recomputed root: {:016x}", att.recomputed_root);
match att.claimed_root {
Some(r) => eprintln!(" claimed root: {:016x}", r),
None => eprintln!(" claimed root: <missing>"),
}
for (name, ok) in &att.checks {
eprintln!(" [{}] {}", if *ok { "PASS" } else { "FAIL" }, name);
}
if att.ok {
eprintln!("ATTESTATION: PASS — leaves recompute to the claimed root.");
} else {
eprintln!("ATTESTATION: FAIL — evidence is inconsistent.");
std::process::exit(1);
}
}
Err(e) => {
eprintln!("ATTESTATION: ERROR — {e}");
std::process::exit(2);
}
}
}