Compliance hardening: enforce measured behavior, real gates, retained evidence
Addresses the attached findings as blocking compliance failures. Each fix removes a substitution pattern and adds a negative-control test. Finding 9 (hash changes != enforced metamorphic expectations): ci_reports now enforces a sound metamorphic relation per perturbation — a perturbation the program CONSUMES must alter the trace — grounded in reachability, with a non-vacuity check. Test: consumed_perturbation_must_alter_trace. Finding 10 (trace counts != causal explanation): New causal_explanation gate ablates each recorded causal edge's source lane and requires the destination delta to change. Measured: true-source 0.77 vs scrambled-source 0.03. Threshold 0.50. New required report causal_explanation_report. Test: causal_edges_are_intervention_confirmed_not_counted. Finding 6 (static seed corpus != failure retention): replay_corpus::retention adds a committed, append-only counterexample corpus (retained_failures.tsv) re-verified every run against the independent runtime. Negative control: reintroduced_bug_is_caught_by_retention. Finding 5 (mini mutation evaluator != real acceptance gate): Mutants are now killed by ci_reports' OWN acceptance-gate predicates with single-sourced thresholds (TRACE_EDGES_MIN, etc.); evaluate_mutants replaces semantic_mutation::run_suite on the acceptance path. Test: mutants_killed_by_real_acceptance_gates. Findings 2 & 3 (generated report != independent attestation; merkle root != provenance without leaves): ci_reports persists every Merkle leaf (evidence/leaves.tsv) + claims. New `attestation` crate + `attest` binary recompute the root from the leaves in a SEPARATE process that never reads compliance_report.json; wired as a distinct merge-gates step. Negative control: tampered_leaf_breaks_attestation. Finding 8 (structural indicators != measured behavior): Domain gate already requires measured influence/mutation/removal; added an explicit reject for "appears structurally but no measured influence". Finding 1 (workflow != merge enforcement): BLOCKED on server-side branch protection. Added .github/rulesets/main-required-checks.json + apply command; enforcement still requires a repo admin to activate the ruleset. Finding 7 (protocol socket E2E != rendered browser E2E): BLOCKED on a CI browser runner; rendered-browser E2E remains advisory-only. Finding 4 (trace summary != full trace evidence): PARTIAL. Each retained leaf binds the full trace via canonical_hash over all graph edges, and the root is independently recomputed from the leaves; per-execution raw-trace round-trip reconstruction by the attestor is not yet implemented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
# Merge enforcement (finding 1 — external dependency)
|
||||
|
||||
A workflow file in `.github/workflows/` **defines** jobs; it does **not** enforce
|
||||
that they pass before merge. Enforcement is a server-side GitHub setting
|
||||
(branch protection / repository ruleset) that marks the jobs as **required
|
||||
status checks** on `main` and the merge queue. That setting lives in the GitHub
|
||||
repository configuration, not in this repository's tree, and applying it
|
||||
requires repository-admin privileges and an authenticated `gh`/API token.
|
||||
|
||||
This is therefore BLOCKED on external infrastructure. To close the gap, a repo
|
||||
admin applies the ruleset in `main-required-checks.json`:
|
||||
|
||||
```bash
|
||||
# Requires: gh auth login as a repo admin
|
||||
gh api -X POST repos/<owner>/<repo>/rulesets \
|
||||
--input .github/rulesets/main-required-checks.json
|
||||
|
||||
# Verify the required checks are active:
|
||||
gh api repos/<owner>/<repo>/rulesets --jq '.[].name'
|
||||
gh api repos/<owner>/<repo>/branches/main/protection 2>/dev/null \
|
||||
|| echo "no classic protection (rulesets in use)"
|
||||
```
|
||||
|
||||
Until that ruleset is active, the `merge-gates` and `web-rust-gates` jobs are
|
||||
*advisory CI*, not merge enforcement. Do not treat their presence in the tree as
|
||||
satisfying the merge-blocking requirement.
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "main-merge-gates",
|
||||
"target": "branch",
|
||||
"enforcement": "active",
|
||||
"conditions": { "ref_name": { "include": ["refs/heads/main"], "exclude": [] } },
|
||||
"rules": [
|
||||
{ "type": "pull_request",
|
||||
"parameters": {
|
||||
"required_approving_review_count": 0,
|
||||
"dismiss_stale_reviews_on_push": true,
|
||||
"require_code_owner_review": false,
|
||||
"require_last_push_approval": false,
|
||||
"required_review_thread_resolution": false
|
||||
}
|
||||
},
|
||||
{ "type": "required_status_checks",
|
||||
"parameters": {
|
||||
"strict_required_status_checks_policy": true,
|
||||
"required_status_checks": [
|
||||
{ "context": "merge-gates" },
|
||||
{ "context": "web-rust-gates" }
|
||||
]
|
||||
}
|
||||
},
|
||||
{ "type": "non_fast_forward" }
|
||||
]
|
||||
}
|
||||
@@ -67,12 +67,21 @@ jobs:
|
||||
- name: Enforce required artifacts exist
|
||||
run: |
|
||||
for r in domain_participation_report causal_rank_report \
|
||||
causal_explanation_report \
|
||||
compression_resistance_report metamorphic_response_report \
|
||||
mutation_survivor_report runtime_equivalence_report \
|
||||
replay_report coverage_report provenance_report \
|
||||
compliance_report; do
|
||||
test -s "ci_out/${r}.json" || { echo "MISSING ARTIFACT: ${r}.json"; exit 1; }
|
||||
done
|
||||
test -s ci_out/evidence/leaves.tsv || { echo "MISSING EVIDENCE: leaves.tsv"; exit 1; }
|
||||
|
||||
# Independent attestation (findings 2, 3): a SEPARATE process recomputes
|
||||
# the Merkle root from the retained leaves and checks it against the
|
||||
# producer's claim. It never reads compliance_report.json. A producer that
|
||||
# reported a root inconsistent with its own leaves fails here.
|
||||
- name: Independent attestation of evidence
|
||||
run: cargo run --release -p attestation --bin attest -- ci_out
|
||||
|
||||
- name: Upload acceptance evidence
|
||||
if: always()
|
||||
|
||||
Reference in New Issue
Block a user