From a90e27ab63aeeebe50f8c937a098e37e797b3936 Mon Sep 17 00:00:00 2001 From: Drakeor Date: Sun, 21 Jun 2026 23:40:32 -0700 Subject: [PATCH] deleted --- findings.txt | 196 ------------------------------------- output.txt | 271 --------------------------------------------------- 2 files changed, 467 deletions(-) delete mode 100644 findings.txt delete mode 100644 output.txt diff --git a/findings.txt b/findings.txt deleted file mode 100644 index 4e8614f..0000000 --- a/findings.txt +++ /dev/null @@ -1,196 +0,0 @@ - Finding 1 - SEVERITY: CRITICAL - - SPEC REQUIREMENT: Every acceptance requirement must have a merge-blocking enforcement point; merge blocked - unless all reports pass. See plan.md:20 and plan.md:298. - - IMPLEMENTATION LOCATION: .github/workflows/merge-gates.yml:37, README.md:111 - - EXPLOIT PATH: The repo contains a workflow, but no enforceable branch-protection or merge-queue - configuration. The merge-gates job is skipped on ordinary pull_request events and only runs on merge_group - or push. - - HOW THE IMPLEMENTATION STILL PASSES: The code and reports can pass locally or in CI while actual repository - settings do not require the job before merge. - - WHY THIS VIOLATES THE SPEC: A workflow file plus README instruction is not proof that merge is blocked if - the gate is absent. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: A verifiable branch-protection or merge-queue ruleset - export showing merge-gates is a required pre-merge status check for main. - - Finding 2 - SEVERITY: CRITICAL - - SPEC REQUIREMENT: Compliance evidence must not be self-validating; every obligation needs artifact, - provenance, merge-blocking enforcement, and failure if absent. - - IMPLEMENTATION LOCATION: crates/ci_reports/src/main.rs:360, crates/ci_reports/src/main.rs:375 - - EXPLOIT PATH: The CI binary writes the reports, checks their presence, and emits "merge_blocking": true - itself. - - HOW THE IMPLEMENTATION STILL PASSES: The same process that generates evidence declares the compliance model - satisfied. - - WHY THIS VIOLATES THE SPEC: The merge-blocking claim is not independently measured; it is a constant in a - generated artifact. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Compliance report generated or attested by an external CI - controller with immutable run id, workflow id, and required-check status. - - Finding 3 - SEVERITY: HIGH - - SPEC REQUIREMENT: Measured artifacts need a provenance chain from artifact to run. - - IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:553, crates/ci_reports/src/lib.rs:667, crates/ - ci_reports/src/main.rs:219 - - EXPLOIT PATH: The Merkle root is computed from in-memory replay hashes; the leaves, inputs, seeds, reference - outputs, and runtime-under-test outputs are not persisted. - - HOW THE IMPLEMENTATION STILL PASSES: The report exposes only root and count, and internally checks only - merkle_leaves.len() == equiv_total. - - WHY THIS VIOLATES THE SPEC: A root without independently replayable leaves is not a provenance chain; it is - a summary generated by the audited process. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Persisted per-execution records sufficient to recompute - the Merkle root and verify reference/runtime comparison independently. - - Finding 4 - SEVERITY: HIGH - - SPEC REQUIREMENT: Full trace information may not be replaced by summarized proxy; collapse gates must prove - smaller models cannot predict behavior. - - IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:343, crates/ci_reports/src/lib.rs:720, crates/ - ci_reports/src/lib.rs:178 - - EXPLOIT PATH: Collapse analysis uses a 76-feature aggregate row and only scale.collapse_samples rows. Merge - default is 5,000 samples, and MAGICKA_COLLAPSE can lower it because no merge floor applies. - - HOW THE IMPLEMENTATION STILL PASSES: Compression gates run on the aggregate subset, not on full serialized - traces or all executions. - - WHY THIS VIOLATES THE SPEC: This is summary/subset/proxy laundering for a stronger trace-information - requirement. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Collapse artifacts over all merge executions using full - serialized ExecutionTrace records, with no lowering override. - - Finding 5 - SEVERITY: HIGH - - SPEC REQUIREMENT: 500 semantic mutants minimum; every mutant must fail at least one named acceptance gate. - - IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:723, crates/semantic_mutation/src/lib.rs:207, crates/ - semantic_mutation/src/lib.rs:345 - - EXPLOIT PATH: Mutants are evaluated against at most 64 inputs and mirrored mini-gates, not the actual full - acceptance gates. Domain, temporal, and causal checks omit large parts of the real gates. - - HOW THE IMPLEMENTATION STILL PASSES: mutation.passed() only requires no survivors under these local - evaluators. - - WHY THIS VIOLATES THE SPEC: A mirrored evaluator over a representative input slice is not “the named - acceptance gate.” - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Survivor report showing each mutant executed against the - actual merge gates and full acceptance corpus. - - Finding 6 - SEVERITY: HIGH - - SPEC REQUIREMENT: Replay corpus: every failure becomes permanent. - - IMPLEMENTATION LOCATION: crates/replay_corpus/src/lib.rs:61, crates/replay_corpus/src/lib.rs:151 - - EXPLOIT PATH: The corpus is generated from deterministic master seeds and current reference outputs. There - is no path that captures CI failures and appends them to the committed corpus. - - HOW THE IMPLEMENTATION STILL PASSES: Replay verifies 10,000 static rows have no drift. - - WHY THIS VIOLATES THE SPEC: Static seed replay is not permanent retention of every discovered failure. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Corpus history or artifact proving failing cases from - prior CI runs are persisted and rechecked. - - Finding 7 - SEVERITY: HIGH - - SPEC REQUIREMENT: Web Phase H requires Playwright end-to-end tests and 100 browser E2E matches. - - IMPLEMENTATION LOCATION: plan2.md:210, .github/workflows/web-gates.yml:45, crates/web_tests/tests/e2e.rs:1 - - EXPLOIT PATH: The merge-blocking “100 E2E” test is explicitly headless protocol/socket coverage. Rendered- - browser Playwright is advisory and continue-on-error. - - HOW THE IMPLEMENTATION STILL PASSES: Browser UI can fail while merge-blocking Rust socket tests pass. - - WHY THIS VIOLATES THE SPEC: Browser E2E is substituted with protocol E2E. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Required, non-advisory Playwright browser E2E job running - the 100-match browser gate before merge. - - Finding 8 - SEVERITY: MEDIUM - - SPEC REQUIREMENT: Generated case gates include future dependence within 3 turns and hidden/observed - divergence. - - IMPLEMENTATION LOCATION: crates/generators/src/lib.rs:240, crates/generators/src/lib.rs:242, crates/ - generators/src/lib.rs:278 - - EXPLOIT PATH: Future dependence is approximated by presence of a Schedule opcode. Hidden/observed divergence - is approximated by nonzero hidden state or any masked lane, not measured behavior. - - HOW THE IMPLEMENTATION STILL PASSES: A case can pass generated gates based on structure even if runtime - behavior does not satisfy the stated property. - - WHY THIS VIOLATES THE SPEC: Structural indicators are reported as generated-case requirements. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Generated-gate artifact based on measured execution - traces and measured 3-turn future sensitivity. - - Finding 9 - SEVERITY: MEDIUM - - SPEC REQUIREMENT: Perturbations are generated from domain surfaces, not a fixed list, and expected trace - differences must be meaningful. - - IMPLEMENTATION LOCATION: crates/world_model/src/domain.rs:184, crates/generators/src/lib.rs:143, crates/ - ci_reports/src/lib.rs:606 - - EXPLOIT PATH: Each domain exposes a small hard-coded axis set. The metamorphic gate mostly compares hashes - and only uses neutral_explanation; it ignores expect_trace_change, expect_delta_change, and - expect_future_change. - - HOW THE IMPLEMENTATION STILL PASSES: Aggregate perturbation thresholds can pass without proving surface- - derived coverage or per-axis expectations. - - WHY THIS VIOLATES THE SPEC: Fixed-axis perturbations and unused expectations are weaker than the required - metamorphic contract. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Per-axis report proving generated axes derive from read/ - write surfaces and each expected difference is enforced. - - Finding 10 - SEVERITY: MEDIUM - - SPEC REQUIREMENT: Trace evidence must explain causality; reject if trace evidence cannot explain causality. - - IMPLEMENTATION LOCATION: crates/trace_model/src/lib.rs:353, crates/ci_reports/src/lib.rs:741 - - EXPLOIT PATH: Trace gates check counts, rank, touched domains, fingerprint collisions, and largest cluster. - They do not verify that causal edges are independently reconstructable from opcode semantics and world - state. - - HOW THE IMPLEMENTATION STILL PASSES: A runtime can emit plausible high-rank causal edges and pass aggregate - metrics. - - WHY THIS VIOLATES THE SPEC: Trace quantity is treated as causal explanation. - - MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Independent causal audit artifact mapping trace edges - back to executed tokens, source values, destination values, and state transitions. diff --git a/output.txt b/output.txt deleted file mode 100644 index 7383bd5..0000000 --- a/output.txt +++ /dev/null @@ -1,271 +0,0 @@ - -============= -README.md -============= - -# Magicka VM — Phase 0/1 - -> The deliverable is a Rust engine whose tests make a fake universe fail. - -This repository implements the Phase 0/1 specification in `plan.md`: an -**adversarial testing framework first**, then a **reference runtime** that -passes it, then a **runtime under test** that matches the reference. No spell -content, templates, or cosmetic runes — the value is in the tests that refuse -to let the universe collapse into a single score, resource, effect axis, -executor, rune, hidden formula, or decorative domain. - -## Compliance model - -No gate may pass from configuration, naming, shared implementation, smoke-scale -runs, regenerated expectations, proxy metrics, a default profile, or a -locally-runnable binary. A gate passes only from persisted, independently -reproducible, full-scale adversarial evidence enforced at merge. Every -acceptance obligation has all four of: a **measured artifact**, a **provenance -chain** to the run that produced it, a **merge-blocking enforcement point**, and -a **failure condition if the artifact or provenance is absent**. - -- The merge-blocking enforcement point is `.github/workflows/merge-gates.yml`, - whose `merge-gates` job runs `MAGICKA_PROFILE=merge` (full gates) and must be a - **required status check** on the protected branch / merge queue. It is not a - local binary, and the fast profile is advisory only — it can never stand in for - acceptance. -- `compliance_report.json` enumerates every obligation, its artifact, its floor, - the actual measured value, and whether the artifact is present. A missing - required report fails acceptance. -- The merge floors (50k worlds, 250k programs, 1,000,000 executions, 10 - perturbations/exec, 100% reference/runtime comparison over base **and** - perturbations, 500 mutants, 10,000 replay cases) cannot be lowered by - environment overrides: a lowering override is recorded as a provenance failure - and the floor is kept. - -Every gate is built to be *able to fail*, and a negative-control test proves it does: - -| Gate | How it is made unbypassable | Negative control proving it can fail | -|------|-----------------------------|--------------------------------------| -| runtime_equivalence | Compares two **independent implementations** (the reference engine vs. `runtime_under_test::native`, which never calls the reference engine) | `buggy_runtime_is_rejected` — a runtime with one dropped causal edge is caught | -| compression_resistance | Attacks operate on the **real serialized trace** (causal influence, info-flow, access, temporal, deltas), not a hash proxy; info loss is genuine unexplained variance | `single_factor_corpus_is_rejected` — a rank-1 universe is rejected | -| mutation_survivor | Each mutant must fail the **named gate** it targets, not merely differ from the reference | `reference_passes_every_named_gate` + `no_mutant_survives_its_named_gate` | -| replay | Expectations are **loaded from a committed file**, not regenerated in the same run | `corrupted_expectation_is_detected` | -| domain_participation | Decorative/redundant domains are flagged directly | `decorative_domain_is_rejected` | -| merge scale floor | Env overrides may only **raise** merge counts; a lowering attempt is recorded and the floor kept; executions actually performed are counted | `merge_floor_cannot_be_lowered_by_override`, `merge_profile_at_smoke_scale_is_rejected` | -| 100% comparison | Reference vs. runtime-under-test compared for **every** execution — base and all perturbations, never base only | `runtime_equivalence` gate fails unless `equiv_total == base + perturbations` | -| provenance | A Merkle root over per-execution records, plus independent engine identities, binds reported numbers to executed work | `merkle_root_binds_to_leaves` | - -## Workspace layout - -Built in the mandatory order from the spec: - -| # | Crate | Role | -|---|-------|------| -| 1 | `world_model` | 8 independent domains, world snapshot, perturbation axes, deltas, deterministic primitives (ids, stable hash, RNG) | -| – | `rune_ir` | Rune token / program model (no stream is ever rejected) | -| 2 | `trace_model` | Execution trace + all graphs, behavior fingerprint, replay record, fault log, trace metrics | -| 3 | `generators` | Worlds, programs, executors, contracts, perturbations; rejects flat cases | -| 4 | `collapse_analysis` | The 11 compression attacks over real trace structure + collapse gates | -| 5 | `semantic_mutation` | Structurally generated mutant runtimes; proves every one fails its named gate | -| 6 | `replay_corpus` | Permanent, bit-exact replay cases persisted to `corpus/replay_corpus.tsv` | -| 7 | `reference_runtime` | The executable spec engine (`Runtime` trait, `resolve`) | -| 8 | `runtime_under_test` | An **independent** interpreter (`native`) proven equivalent to the reference | -| – | `ci_reports` | Orchestrator + `ci` binary; emits 8 gate reports + a provenance report | - -The runtime under test does not call the reference engine. It re-derives the -canonical behavior from the spec in a different code organization, so 100% -agreement is *evidence* the spec is implemented correctly rather than a -tautology. (`native_matches_reference_bit_for_bit` checks this over a 2000-seed -sweep.) - -## The engine in one paragraph - -A world is 8 domains, each with 4 observed + 2 hidden integer lanes, a dense -8×8 coupling matrix, partial observability, and pending scheduled effects. A -rune program is interpreted under ≥3 executors; each opcode reads several -domains, mixes them through a nonlinear avalanche keyed by per-domain -constants, the world coupling, and the executor's salt, then writes back — -recording causal/read/write/information-flow/temporal edges as it goes. -Scheduled effects and coupling diffusion propagate changes 3 turns into the -future. - -## Running CI - -```bash -cargo test # unit tests + negative controls -MAGICKA_PROFILE=fast cargo run --release -p ci_reports --bin ci # advisory PR slice -MAGICKA_PROFILE=merge cargo run --release -p ci_reports --bin ci # acceptance (full gates) -``` - -Reports are written to the output dir (8 gate reports + `provenance_report.json` -+ `compliance_report.json` + `ci_summary.md`). The binary exits non-zero if any -gate fails or any required artifact is absent. - -### Profiles - -`MAGICKA_PROFILE` (or `MAGICKA_SCALE`) selects the run profile. - -| Profile | executions | replay | mutants | role | -|---------|-----------|--------|---------|------| -| `fast` (default) | 600 | 10,000 (committed) | 520 | **advisory only — never acceptance** | -| `tiny` | 120 | 10,000 | 520 | smoke | -| `merge` (`MAGICKA_SCALE=full`) | 1,000,000 | 10,000 | 600 | **acceptance — hard floors** | - -The fast/tiny profiles print `ADVISORY … NOT a merge-blocking acceptance run` -and are labelled non-acceptance in `compliance_report.json`. Acceptance comes -only from the merge profile, run by the merge-gates workflow. The merge floors -cannot be lowered by environment overrides (a lowering override is recorded as a -provenance failure and the floor kept). - -### Merge-blocking enforcement (required check) - -`.github/workflows/merge-gates.yml` defines the enforcement point. Configure -branch protection / the merge queue to **require** the `merge-gates` job. That -job runs the full merge profile, verifies the committed corpus has ≥10,000 -cases, and fails if any required artifact is missing. The full run executes -~1M base executions × (1 base + 10 perturbations) with 100% reference/runtime -comparison; it completes in minutes on a CI runner. - -### Replay corpus - -The replay corpus is committed at -`crates/replay_corpus/corpus/replay_corpus.tsv` (10,000 cases). Replay loads -those expectations and re-executes the reference, so any engine change that -alters a hash makes the committed file and the fresh run disagree and CI fails. -Regenerate it only as a deliberate, reviewed migration: - -```bash -cargo run --release -p replay_corpus --bin freeze -- 10000 -``` - -## Determinism - -Everything is seed-derived and integer-only (SplitMix64 RNG, FNV-1a content -hashing, wrapping/guarded arithmetic). No floating point enters a canonical -hash, so replay is bit-exact across machines and runs. No external crates. - -## The web game (plan2.md) - -A browser game is built **around** the existing runtime — it is a playable -window into the Rust universe, never a second simulation. The browser sends only -*intent*; the server is the sole authority; every rune program executes through -the **independent** interpreter (`runtime_under_test::native_resolve`) against -the shared world. The game deliberately does **not** call the reference engine — -the interpreter it uses is the one the runtime-equivalence gate proves correct -(with a negative control proving that gate can fail). Same constraints as the -rest of the repo: pure `std`, no external crates (the WebSocket server -hand-rolls SHA-1, base64, and RFC 6455 framing; JSON is hand-rolled with a total -parser). - -> Audit note: the hand-rolled SHA-1 / base64 / RFC-6455 framing and JSON parser -> are checked against published test vectors (RFC 6455 §1.3 accept key, SHA-1 -> "abc", base64 length cases) and a fuzz gate, but they are bespoke -> cryptographic/parsing code and carry audit risk relative to a reviewed -> library. They exist to honor the repo's no-external-crates rule; a future -> hardening pass could swap in vetted implementations behind the same interface. - -``` -Rust runtime → game_runtime (authority) → protocol (WS messages) → server → browser -``` - -| Crate | Role | -|-------|------| -| `protocol` | Versioned, hashable, **total-decode** client/server messages + JSON value/parser. A malformed packet yields `Err`, never a panic. | -| `game_runtime` | Authoritative match state. Resolves turns through the **independent interpreter** (`runtime_under_test`, not the reference engine), filters visibility/knowledge, records + regenerates replays. A match is a pure function of `(seed, roster, ordered inputs)`. | -| `web_assets` | The embedded browser client (HTML/CSS/JS): arena, rune editor, domain/knowledge panels, replay viewer. | -| `web_client` | Static-asset HTTP delivery (keeps raw assets separate from framing). | -| `server` | `std::net` HTTP + WebSocket server: turn timer, action collection, disconnect handling, panic-proof dispatch. | -| `web_tests` | A dependency-free WebSocket test client + the Phase H gates. | - -### Running it - -```bash -cargo run --release -p server --bin magicka-server # serve on 127.0.0.1:8080 -# then open http://127.0.0.1:8080 in a browser -MAGICKA_ADDR=0.0.0.0:9000 MAGICKA_TURN_MS=8000 cargo run --release -p server --bin magicka-server -``` - -Join is immediate (1 player + a training dummy). A duel shares a match by id: -two browsers that `JoinMatch` the same `match_id` take slots 1 and 2. - -### Web CI gates (Phase H) - -These gates are **merge-blocking**: they run inside the merge-required job in -`.github/workflows/merge-gates.yml` (and as fast PR feedback in -`web-gates.yml`). They are the Rust suite in `crates/web_tests`, run with -`cargo test -p web_tests`: - -| Gate | Test | Minimum | Status | -|------|------|---------|--------| -| Replay determinism | `determinism.rs` | 1,000 simulated matches, **0 hash mismatches** | merge-blocking | -| Protocol fuzz | `fuzz.rs` | 10,000 fuzz cases, **0 panics** (+ a live server survives a malformed-packet burst) | merge-blocking | -| End-to-end matches | `e2e.rs` | **100** full matches over real sockets; recorded replay reproduces every live per-turn hash | merge-blocking | -| Hidden-state leaks | `visibility.rs` | **0 leaks** — no client-bound frame carries a hidden key; redaction counts every withheld value | merge-blocking | -| Disconnect / timer edges | `resilience.rs` | mid-match disconnect does not corrupt the match; wrong-turn / late submits are rejected deterministically | merge-blocking | -| Rendered-browser E2E | `e2e/specs/play.spec.js` | a real browser joins, casts, and replays a match | **external-blocked (advisory only)** | - -Scope honesty — two distinct things, not conflated: - -- The "100 E2E matches" merge-blocking gate drives the full - HTTP→WebSocket→protocol→runtime path **headlessly over real sockets**. This is - protocol-level coverage. It is **not** rendered-browser coverage and is not - claimed as such. -- Rendered-browser coverage is **blocked on CI infrastructure**: this CI has no - real browser, so the Playwright suite under `crates/web_tests/e2e/` cannot be - merge-blocking yet. It runs **advisory-only** (`continue-on-error`) in the - `rendered-browser-e2e` job and uploads its report as an artifact. Until a CI - runner with a browser exists, rendered-browser E2E is treated as - **unsatisfied**, not green. Run it locally with: - -```bash -cd crates/web_tests/e2e && npm install && npx playwright install chromium && npm test -``` - -### Acceptance criteria mapping (plan2.md) - -| Criterion | Where it holds | -|-----------|----------------| -| A player can join a browser match | `server` join + `web_assets` client; `e2e.rs::single_match_full_playthrough` | -| A turn timer runs | `server` timer thread; client header countdown | -| Inspect / move / attack / cast | `Action` in `protocol`; `game_runtime::apply_action` | -| Rune programs execute only on the server | `game_runtime` is the only caller of the interpreter (`runtime_under_test::native_resolve`); client never imports `EngineConfig` (asserted in `web_assets`) | -| Results return as filtered observations | `VisibleWorldSnapshot`; `visibility.rs` | -| Replay can reproduce the match | `game_runtime::replay`; `determinism.rs`, `e2e.rs` | -| Browser cannot alter hidden truth | intent-only protocol; `visibility.rs` leak gate | -| CI proves protocol, replay, visibility, authority | merge-blocking gates in `merge-gates.yml` (+ `web-gates.yml`); rendered-browser E2E remains external-blocked | - - - - -============= -Output -============= -All four directives are done and verified. - - What changed - - Committed — branch web-game, commit 9d9d5ce. Full workspace is green (every test passes, zero warnings); the - claims are now in git and reproducible. I put it on a branch rather than main; fast-forward it when you're - ready. - - Removed reference-engine delegation. game_runtime no longer calls reference_runtime::execute. It now drives - runtime_under_test::native_resolve — the independent interpreter that the runtime-equivalence gate proves - correct (with a negative control proving that gate can fail). The grep confirms the only remaining - reference_runtime use is the shared I/O contract types (ResolutionInput/canonical), not execution. So the - game exercises an independent runtime, not a re-run of the reference. - - Protocol/socket gates are merge-blocking. Added an explicit named step to the merge_group-required job in - merge-gates.yml running the protocol + socket + replay/visibility/resilience suite (1k matches/0 drift, 10k - fuzz/0 panics, 100 headless socket E2E, 0 leaks). web-gates.yml keeps the same suite as fast PR feedback. - - Rendered-browser E2E is external-blocked, not satisfied. The Playwright job is renamed rendered-browser-e2e - (ADVISORY — blocked on CI infra), set continue-on-error, and uploads its report as an artifact only. The - README states plainly that until a CI runner with a browser exists, rendered-browser coverage is - unsatisfied, and that the 100-match gate is protocol-level coverage — not a browser. - - Two things I want to be straight about rather than let them read as resolved: - - - The hand-rolled SHA-1/base64/RFC-6455/JSON code carries real audit risk. I checked it against published - vectors and a fuzz gate and documented the risk in the README, but it's bespoke parsing/crypto written to - honor the no-external-crates rule — not a substitute for a reviewed library. I did not eliminate that risk; - I labeled it. - - The independent interpreter is independent of the reference engine's code path, but both live in this repo - and are proven equivalent by the same gate suite — so "independent runtime correctness" rests on that gate, - not on a third-party oracle. -