197 lines
8.6 KiB
Plaintext
197 lines
8.6 KiB
Plaintext
Finding 1
|
|
SEVERITY: CRITICAL
|
|
|
|
SPEC REQUIREMENT: Every acceptance requirement must have a merge-blocking enforcement point; merge blocked
|
|
unless all reports pass. See plan.md:20 and plan.md:298.
|
|
|
|
IMPLEMENTATION LOCATION: .github/workflows/merge-gates.yml:37, README.md:111
|
|
|
|
EXPLOIT PATH: The repo contains a workflow, but no enforceable branch-protection or merge-queue
|
|
configuration. The merge-gates job is skipped on ordinary pull_request events and only runs on merge_group
|
|
or push.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: The code and reports can pass locally or in CI while actual repository
|
|
settings do not require the job before merge.
|
|
|
|
WHY THIS VIOLATES THE SPEC: A workflow file plus README instruction is not proof that merge is blocked if
|
|
the gate is absent.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: A verifiable branch-protection or merge-queue ruleset
|
|
export showing merge-gates is a required pre-merge status check for main.
|
|
|
|
Finding 2
|
|
SEVERITY: CRITICAL
|
|
|
|
SPEC REQUIREMENT: Compliance evidence must not be self-validating; every obligation needs artifact,
|
|
provenance, merge-blocking enforcement, and failure if absent.
|
|
|
|
IMPLEMENTATION LOCATION: crates/ci_reports/src/main.rs:360, crates/ci_reports/src/main.rs:375
|
|
|
|
EXPLOIT PATH: The CI binary writes the reports, checks their presence, and emits "merge_blocking": true
|
|
itself.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: The same process that generates evidence declares the compliance model
|
|
satisfied.
|
|
|
|
WHY THIS VIOLATES THE SPEC: The merge-blocking claim is not independently measured; it is a constant in a
|
|
generated artifact.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Compliance report generated or attested by an external CI
|
|
controller with immutable run id, workflow id, and required-check status.
|
|
|
|
Finding 3
|
|
SEVERITY: HIGH
|
|
|
|
SPEC REQUIREMENT: Measured artifacts need a provenance chain from artifact to run.
|
|
|
|
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:553, crates/ci_reports/src/lib.rs:667, crates/
|
|
ci_reports/src/main.rs:219
|
|
|
|
EXPLOIT PATH: The Merkle root is computed from in-memory replay hashes; the leaves, inputs, seeds, reference
|
|
outputs, and runtime-under-test outputs are not persisted.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: The report exposes only root and count, and internally checks only
|
|
merkle_leaves.len() == equiv_total.
|
|
|
|
WHY THIS VIOLATES THE SPEC: A root without independently replayable leaves is not a provenance chain; it is
|
|
a summary generated by the audited process.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Persisted per-execution records sufficient to recompute
|
|
the Merkle root and verify reference/runtime comparison independently.
|
|
|
|
Finding 4
|
|
SEVERITY: HIGH
|
|
|
|
SPEC REQUIREMENT: Full trace information may not be replaced by summarized proxy; collapse gates must prove
|
|
smaller models cannot predict behavior.
|
|
|
|
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:343, crates/ci_reports/src/lib.rs:720, crates/
|
|
ci_reports/src/lib.rs:178
|
|
|
|
EXPLOIT PATH: Collapse analysis uses a 76-feature aggregate row and only scale.collapse_samples rows. Merge
|
|
default is 5,000 samples, and MAGICKA_COLLAPSE can lower it because no merge floor applies.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: Compression gates run on the aggregate subset, not on full serialized
|
|
traces or all executions.
|
|
|
|
WHY THIS VIOLATES THE SPEC: This is summary/subset/proxy laundering for a stronger trace-information
|
|
requirement.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Collapse artifacts over all merge executions using full
|
|
serialized ExecutionTrace records, with no lowering override.
|
|
|
|
Finding 5
|
|
SEVERITY: HIGH
|
|
|
|
SPEC REQUIREMENT: 500 semantic mutants minimum; every mutant must fail at least one named acceptance gate.
|
|
|
|
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:723, crates/semantic_mutation/src/lib.rs:207, crates/
|
|
semantic_mutation/src/lib.rs:345
|
|
|
|
EXPLOIT PATH: Mutants are evaluated against at most 64 inputs and mirrored mini-gates, not the actual full
|
|
acceptance gates. Domain, temporal, and causal checks omit large parts of the real gates.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: mutation.passed() only requires no survivors under these local
|
|
evaluators.
|
|
|
|
WHY THIS VIOLATES THE SPEC: A mirrored evaluator over a representative input slice is not “the named
|
|
acceptance gate.”
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Survivor report showing each mutant executed against the
|
|
actual merge gates and full acceptance corpus.
|
|
|
|
Finding 6
|
|
SEVERITY: HIGH
|
|
|
|
SPEC REQUIREMENT: Replay corpus: every failure becomes permanent.
|
|
|
|
IMPLEMENTATION LOCATION: crates/replay_corpus/src/lib.rs:61, crates/replay_corpus/src/lib.rs:151
|
|
|
|
EXPLOIT PATH: The corpus is generated from deterministic master seeds and current reference outputs. There
|
|
is no path that captures CI failures and appends them to the committed corpus.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: Replay verifies 10,000 static rows have no drift.
|
|
|
|
WHY THIS VIOLATES THE SPEC: Static seed replay is not permanent retention of every discovered failure.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Corpus history or artifact proving failing cases from
|
|
prior CI runs are persisted and rechecked.
|
|
|
|
Finding 7
|
|
SEVERITY: HIGH
|
|
|
|
SPEC REQUIREMENT: Web Phase H requires Playwright end-to-end tests and 100 browser E2E matches.
|
|
|
|
IMPLEMENTATION LOCATION: plan2.md:210, .github/workflows/web-gates.yml:45, crates/web_tests/tests/e2e.rs:1
|
|
|
|
EXPLOIT PATH: The merge-blocking “100 E2E” test is explicitly headless protocol/socket coverage. Rendered-
|
|
browser Playwright is advisory and continue-on-error.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: Browser UI can fail while merge-blocking Rust socket tests pass.
|
|
|
|
WHY THIS VIOLATES THE SPEC: Browser E2E is substituted with protocol E2E.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Required, non-advisory Playwright browser E2E job running
|
|
the 100-match browser gate before merge.
|
|
|
|
Finding 8
|
|
SEVERITY: MEDIUM
|
|
|
|
SPEC REQUIREMENT: Generated case gates include future dependence within 3 turns and hidden/observed
|
|
divergence.
|
|
|
|
IMPLEMENTATION LOCATION: crates/generators/src/lib.rs:240, crates/generators/src/lib.rs:242, crates/
|
|
generators/src/lib.rs:278
|
|
|
|
EXPLOIT PATH: Future dependence is approximated by presence of a Schedule opcode. Hidden/observed divergence
|
|
is approximated by nonzero hidden state or any masked lane, not measured behavior.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: A case can pass generated gates based on structure even if runtime
|
|
behavior does not satisfy the stated property.
|
|
|
|
WHY THIS VIOLATES THE SPEC: Structural indicators are reported as generated-case requirements.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Generated-gate artifact based on measured execution
|
|
traces and measured 3-turn future sensitivity.
|
|
|
|
Finding 9
|
|
SEVERITY: MEDIUM
|
|
|
|
SPEC REQUIREMENT: Perturbations are generated from domain surfaces, not a fixed list, and expected trace
|
|
differences must be meaningful.
|
|
|
|
IMPLEMENTATION LOCATION: crates/world_model/src/domain.rs:184, crates/generators/src/lib.rs:143, crates/
|
|
ci_reports/src/lib.rs:606
|
|
|
|
EXPLOIT PATH: Each domain exposes a small hard-coded axis set. The metamorphic gate mostly compares hashes
|
|
and only uses neutral_explanation; it ignores expect_trace_change, expect_delta_change, and
|
|
expect_future_change.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: Aggregate perturbation thresholds can pass without proving surface-
|
|
derived coverage or per-axis expectations.
|
|
|
|
WHY THIS VIOLATES THE SPEC: Fixed-axis perturbations and unused expectations are weaker than the required
|
|
metamorphic contract.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Per-axis report proving generated axes derive from read/
|
|
write surfaces and each expected difference is enforced.
|
|
|
|
Finding 10
|
|
SEVERITY: MEDIUM
|
|
|
|
SPEC REQUIREMENT: Trace evidence must explain causality; reject if trace evidence cannot explain causality.
|
|
|
|
IMPLEMENTATION LOCATION: crates/trace_model/src/lib.rs:353, crates/ci_reports/src/lib.rs:741
|
|
|
|
EXPLOIT PATH: Trace gates check counts, rank, touched domains, fingerprint collisions, and largest cluster.
|
|
They do not verify that causal edges are independently reconstructable from opcode semantics and world
|
|
state.
|
|
|
|
HOW THE IMPLEMENTATION STILL PASSES: A runtime can emit plausible high-rank causal edges and pass aggregate
|
|
metrics.
|
|
|
|
WHY THIS VIOLATES THE SPEC: Trace quantity is treated as causal explanation.
|
|
|
|
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Independent causal audit artifact mapping trace edges
|
|
back to executed tokens, source values, destination values, and state transitions.
|