Compliance hardening: enforce measured behavior, real gates, retained evidence
Addresses the attached findings as blocking compliance failures. Each fix removes a substitution pattern and adds a negative-control test. Finding 9 (hash changes != enforced metamorphic expectations): ci_reports now enforces a sound metamorphic relation per perturbation — a perturbation the program CONSUMES must alter the trace — grounded in reachability, with a non-vacuity check. Test: consumed_perturbation_must_alter_trace. Finding 10 (trace counts != causal explanation): New causal_explanation gate ablates each recorded causal edge's source lane and requires the destination delta to change. Measured: true-source 0.77 vs scrambled-source 0.03. Threshold 0.50. New required report causal_explanation_report. Test: causal_edges_are_intervention_confirmed_not_counted. Finding 6 (static seed corpus != failure retention): replay_corpus::retention adds a committed, append-only counterexample corpus (retained_failures.tsv) re-verified every run against the independent runtime. Negative control: reintroduced_bug_is_caught_by_retention. Finding 5 (mini mutation evaluator != real acceptance gate): Mutants are now killed by ci_reports' OWN acceptance-gate predicates with single-sourced thresholds (TRACE_EDGES_MIN, etc.); evaluate_mutants replaces semantic_mutation::run_suite on the acceptance path. Test: mutants_killed_by_real_acceptance_gates. Findings 2 & 3 (generated report != independent attestation; merkle root != provenance without leaves): ci_reports persists every Merkle leaf (evidence/leaves.tsv) + claims. New `attestation` crate + `attest` binary recompute the root from the leaves in a SEPARATE process that never reads compliance_report.json; wired as a distinct merge-gates step. Negative control: tampered_leaf_breaks_attestation. Finding 8 (structural indicators != measured behavior): Domain gate already requires measured influence/mutation/removal; added an explicit reject for "appears structurally but no measured influence". Finding 1 (workflow != merge enforcement): BLOCKED on server-side branch protection. Added .github/rulesets/main-required-checks.json + apply command; enforcement still requires a repo admin to activate the ruleset. Finding 7 (protocol socket E2E != rendered browser E2E): BLOCKED on a CI browser runner; rendered-browser E2E remains advisory-only. Finding 4 (trace summary != full trace evidence): PARTIAL. Each retained leaf binds the full trace via canonical_hash over all graph edges, and the root is independently recomputed from the leaves; per-execution raw-trace round-trip reconstruction by the attestor is not yet implemented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,15 @@
|
||||
[package]
|
||||
name = "attestation"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[[bin]]
|
||||
name = "attest"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dependencies]
|
||||
# Only the shared hashing primitive — NOT ci_reports. The root is recomputed by
|
||||
# this crate's own code path, so it is an independent check, not a re-export of
|
||||
# the producer's claim.
|
||||
world_model = { path = "../world_model" }
|
||||
@@ -0,0 +1,169 @@
|
||||
//! `attestation` — an **independent** verifier of a CI run's evidence
|
||||
//! (findings 2 and 3).
|
||||
//!
|
||||
//! The substitution it removes: a `compliance_report.json` written by the same
|
||||
//! binary that ran the gates is self-certification, and a Merkle *root* reported
|
||||
//! without its *leaves* cannot be checked by anyone. This crate is a separate
|
||||
//! process with its own code path that:
|
||||
//! * reads the retained per-execution leaves (`evidence/leaves.tsv`),
|
||||
//! * recomputes the Merkle root from them with its own implementation,
|
||||
//! * reads the producer's *claims* (`evidence/claims.tsv`) and checks the
|
||||
//! recomputed root and leaf count match what was claimed,
|
||||
//! * NEVER reads `compliance_report.json` — it does not trust the producer's
|
||||
//! own pass/fail verdict.
|
||||
//!
|
||||
//! It depends only on `world_model` for the shared hash primitive. What it does
|
||||
//! NOT do (and cannot, in-repo) is prove the leaves correspond to real
|
||||
//! executions performed by a trusted third party — that requires external
|
||||
//! re-execution / signing infrastructure (see the BLOCKED note in the report).
|
||||
|
||||
use std::path::Path;
|
||||
use world_model::Hasher;
|
||||
|
||||
/// Recompute the Merkle root over `leaves` (binary tree, FNV-combined). This is
|
||||
/// an independent re-implementation of the producer's algorithm; agreement is
|
||||
/// the check.
|
||||
pub fn merkle_root(leaves: &[u64]) -> u64 {
|
||||
if leaves.is_empty() {
|
||||
return 0;
|
||||
}
|
||||
let mut level = leaves.to_vec();
|
||||
while level.len() > 1 {
|
||||
let mut next = Vec::with_capacity(level.len().div_ceil(2));
|
||||
for pair in level.chunks(2) {
|
||||
let mut h = Hasher::new();
|
||||
h.write_tag("merkle");
|
||||
h.write_u64(pair[0]);
|
||||
h.write_u64(if pair.len() > 1 { pair[1] } else { pair[0] });
|
||||
next.push(h.finish().0);
|
||||
}
|
||||
level = next;
|
||||
}
|
||||
level[0]
|
||||
}
|
||||
|
||||
/// Parse a leaves file: one hex u64 per data line, `#`/`leaf` headers skipped.
|
||||
pub fn parse_leaves(text: &str) -> Vec<u64> {
|
||||
text.lines()
|
||||
.filter(|l| !l.starts_with('#') && !l.starts_with("leaf") && !l.trim().is_empty())
|
||||
.filter_map(|l| u64::from_str_radix(l.trim(), 16).ok())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Parse a `key<TAB>value` claims file into `(key, value)` pairs.
|
||||
pub fn parse_claims(text: &str) -> Vec<(String, String)> {
|
||||
text.lines()
|
||||
.filter(|l| !l.starts_with('#') && !l.trim().is_empty())
|
||||
.filter_map(|l| l.split_once('\t').map(|(k, v)| (k.trim().to_string(), v.trim().to_string())))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn claim<'a>(claims: &'a [(String, String)], key: &str) -> Option<&'a str> {
|
||||
claims.iter().find(|(k, _)| k == key).map(|(_, v)| v.as_str())
|
||||
}
|
||||
|
||||
/// Outcome of an attestation.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Attestation {
|
||||
pub ok: bool,
|
||||
pub leaf_count: usize,
|
||||
pub recomputed_root: u64,
|
||||
pub claimed_root: Option<u64>,
|
||||
pub checks: Vec<(String, bool)>,
|
||||
}
|
||||
|
||||
/// Verify the evidence directory `dir` (which contains `evidence/`).
|
||||
pub fn verify_dir(dir: &Path) -> Result<Attestation, String> {
|
||||
let leaves_path = dir.join("evidence/leaves.tsv");
|
||||
let claims_path = dir.join("evidence/claims.tsv");
|
||||
let leaves_txt = std::fs::read_to_string(&leaves_path)
|
||||
.map_err(|e| format!("cannot read {}: {e}", leaves_path.display()))?;
|
||||
let claims_txt = std::fs::read_to_string(&claims_path)
|
||||
.map_err(|e| format!("cannot read {}: {e}", claims_path.display()))?;
|
||||
|
||||
let leaves = parse_leaves(&leaves_txt);
|
||||
let claims = parse_claims(&claims_txt);
|
||||
let recomputed = merkle_root(&leaves);
|
||||
|
||||
let claimed_root = claim(&claims, "root").and_then(|v| u64::from_str_radix(v, 16).ok());
|
||||
let claimed_count = claim(&claims, "leaf_count").and_then(|v| v.parse::<usize>().ok());
|
||||
let claimed_comparisons =
|
||||
claim(&claims, "total_comparisons").and_then(|v| v.parse::<usize>().ok());
|
||||
|
||||
let mut checks = Vec::new();
|
||||
let root_ok = claimed_root == Some(recomputed);
|
||||
checks.push(("recomputed_root == claimed_root".into(), root_ok));
|
||||
let count_ok = claimed_count == Some(leaves.len());
|
||||
checks.push(("leaf_count == claimed_leaf_count".into(), count_ok));
|
||||
let cmp_ok = claimed_comparisons.map(|c| c == leaves.len()).unwrap_or(false);
|
||||
checks.push(("leaf_count == claimed_total_comparisons".into(), cmp_ok));
|
||||
let nonempty = !leaves.is_empty();
|
||||
checks.push(("leaves are present (root has leaves)".into(), nonempty));
|
||||
|
||||
let ok = checks.iter().all(|(_, b)| *b);
|
||||
Ok(Attestation {
|
||||
ok,
|
||||
leaf_count: leaves.len(),
|
||||
recomputed_root: recomputed,
|
||||
claimed_root,
|
||||
checks,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn root_matches_known_vector() {
|
||||
// Mirrors ci_reports::merkle_root over the same leaves.
|
||||
let a = merkle_root(&[1, 2, 3]);
|
||||
let b = merkle_root(&[1, 2, 3]);
|
||||
let c = merkle_root(&[1, 2, 4]);
|
||||
assert_eq!(a, b);
|
||||
assert_ne!(a, c);
|
||||
}
|
||||
|
||||
fn write_evidence(dir: &Path, leaves: &[u64], claimed_root: u64) {
|
||||
let ev = dir.join("evidence");
|
||||
std::fs::create_dir_all(&ev).unwrap();
|
||||
let mut lt = String::from("leaf\n");
|
||||
for l in leaves {
|
||||
lt.push_str(&format!("{:016x}\n", l));
|
||||
}
|
||||
std::fs::write(ev.join("leaves.tsv"), lt).unwrap();
|
||||
let claims = format!(
|
||||
"# claims\nroot\t{:016x}\nleaf_count\t{}\ntotal_comparisons\t{}\n",
|
||||
claimed_root,
|
||||
leaves.len(),
|
||||
leaves.len()
|
||||
);
|
||||
std::fs::write(ev.join("claims.tsv"), claims).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn honest_evidence_attests() {
|
||||
let dir = std::env::temp_dir().join("magicka_attest_ok");
|
||||
let leaves = [10u64, 20, 30, 40, 50];
|
||||
write_evidence(&dir, &leaves, merkle_root(&leaves));
|
||||
let att = verify_dir(&dir).unwrap();
|
||||
assert!(att.ok, "honest evidence should attest: {:?}", att.checks);
|
||||
}
|
||||
|
||||
/// Negative control: a tampered leaf changes the recomputed root, so the
|
||||
/// claimed root no longer matches and attestation FAILS. Proves the
|
||||
/// attestation is not vacuous and that the root genuinely binds the leaves.
|
||||
#[test]
|
||||
fn tampered_leaf_breaks_attestation() {
|
||||
let dir = std::env::temp_dir().join("magicka_attest_bad");
|
||||
let leaves = [10u64, 20, 30, 40, 50];
|
||||
let claimed = merkle_root(&leaves);
|
||||
// Write evidence whose leaves were altered after the root was claimed.
|
||||
let mut tampered = leaves.to_vec();
|
||||
tampered[2] ^= 0xdead_beef;
|
||||
write_evidence(&dir, &tampered, claimed);
|
||||
let att = verify_dir(&dir).unwrap();
|
||||
assert!(!att.ok, "tampered leaves must fail attestation");
|
||||
assert!(att.checks.iter().any(|(n, ok)| n.contains("root") && !ok));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
//! `attest` — independently verify a CI run's evidence directory.
|
||||
//!
|
||||
//! Usage: `attest <dir>` (default `ci_out`). Exits non-zero if the recomputed
|
||||
//! Merkle root does not match the claimed root, the leaf count is inconsistent,
|
||||
//! or the leaves are missing. It deliberately ignores `compliance_report.json`.
|
||||
|
||||
use std::path::PathBuf;
|
||||
|
||||
fn main() {
|
||||
let dir = std::env::args().nth(1).unwrap_or_else(|| "ci_out".to_string());
|
||||
let path = PathBuf::from(&dir);
|
||||
match attestation::verify_dir(&path) {
|
||||
Ok(att) => {
|
||||
eprintln!("=== independent attestation of {dir} ===");
|
||||
eprintln!(" leaves: {}", att.leaf_count);
|
||||
eprintln!(" recomputed root: {:016x}", att.recomputed_root);
|
||||
match att.claimed_root {
|
||||
Some(r) => eprintln!(" claimed root: {:016x}", r),
|
||||
None => eprintln!(" claimed root: <missing>"),
|
||||
}
|
||||
for (name, ok) in &att.checks {
|
||||
eprintln!(" [{}] {}", if *ok { "PASS" } else { "FAIL" }, name);
|
||||
}
|
||||
if att.ok {
|
||||
eprintln!("ATTESTATION: PASS — leaves recompute to the claimed root.");
|
||||
} else {
|
||||
eprintln!("ATTESTATION: FAIL — evidence is inconsistent.");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("ATTESTATION: ERROR — {e}");
|
||||
std::process::exit(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user