Compliance hardening: enforce measured behavior, real gates, retained evidence
Addresses the attached findings as blocking compliance failures. Each fix removes a substitution pattern and adds a negative-control test. Finding 9 (hash changes != enforced metamorphic expectations): ci_reports now enforces a sound metamorphic relation per perturbation — a perturbation the program CONSUMES must alter the trace — grounded in reachability, with a non-vacuity check. Test: consumed_perturbation_must_alter_trace. Finding 10 (trace counts != causal explanation): New causal_explanation gate ablates each recorded causal edge's source lane and requires the destination delta to change. Measured: true-source 0.77 vs scrambled-source 0.03. Threshold 0.50. New required report causal_explanation_report. Test: causal_edges_are_intervention_confirmed_not_counted. Finding 6 (static seed corpus != failure retention): replay_corpus::retention adds a committed, append-only counterexample corpus (retained_failures.tsv) re-verified every run against the independent runtime. Negative control: reintroduced_bug_is_caught_by_retention. Finding 5 (mini mutation evaluator != real acceptance gate): Mutants are now killed by ci_reports' OWN acceptance-gate predicates with single-sourced thresholds (TRACE_EDGES_MIN, etc.); evaluate_mutants replaces semantic_mutation::run_suite on the acceptance path. Test: mutants_killed_by_real_acceptance_gates. Findings 2 & 3 (generated report != independent attestation; merkle root != provenance without leaves): ci_reports persists every Merkle leaf (evidence/leaves.tsv) + claims. New `attestation` crate + `attest` binary recompute the root from the leaves in a SEPARATE process that never reads compliance_report.json; wired as a distinct merge-gates step. Negative control: tampered_leaf_breaks_attestation. Finding 8 (structural indicators != measured behavior): Domain gate already requires measured influence/mutation/removal; added an explicit reject for "appears structurally but no measured influence". Finding 1 (workflow != merge enforcement): BLOCKED on server-side branch protection. Added .github/rulesets/main-required-checks.json + apply command; enforcement still requires a repo admin to activate the ruleset. Finding 7 (protocol socket E2E != rendered browser E2E): BLOCKED on a CI browser runner; rendered-browser E2E remains advisory-only. Finding 4 (trace summary != full trace evidence): PARTIAL. Each retained leaf binds the full trace via canonical_hash over all graph edges, and the root is independently recomputed from the leaves; per-execution raw-trace round-trip reconstruction by the attestor is not yet implemented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -67,12 +67,21 @@ jobs:
|
||||
- name: Enforce required artifacts exist
|
||||
run: |
|
||||
for r in domain_participation_report causal_rank_report \
|
||||
causal_explanation_report \
|
||||
compression_resistance_report metamorphic_response_report \
|
||||
mutation_survivor_report runtime_equivalence_report \
|
||||
replay_report coverage_report provenance_report \
|
||||
compliance_report; do
|
||||
test -s "ci_out/${r}.json" || { echo "MISSING ARTIFACT: ${r}.json"; exit 1; }
|
||||
done
|
||||
test -s ci_out/evidence/leaves.tsv || { echo "MISSING EVIDENCE: leaves.tsv"; exit 1; }
|
||||
|
||||
# Independent attestation (findings 2, 3): a SEPARATE process recomputes
|
||||
# the Merkle root from the retained leaves and checks it against the
|
||||
# producer's claim. It never reads compliance_report.json. A producer that
|
||||
# reported a root inconsistent with its own leaves fails here.
|
||||
- name: Independent attestation of evidence
|
||||
run: cargo run --release -p attestation --bin attest -- ci_out
|
||||
|
||||
- name: Upload acceptance evidence
|
||||
if: always()
|
||||
|
||||
Reference in New Issue
Block a user