linus-d/magicka-vm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add web game (plan2.md) on the independent runtime, merge-blocking gates #1

Merged
linus-d merged 7 commits from web-game into main 2026-06-22 01:42:01 -05:00
Conversation 0 Commits 7 Files Changed 50
-467
2 changed files with 0 additions and 467 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit a90e27ab63 - Show all commits
findings.txt
-196
View File
@@ -1,196 +0,0 @@
Finding 1
SEVERITY: CRITICAL
SPEC REQUIREMENT: Every acceptance requirement must have a merge-blocking enforcement point; merge blocked
unless all reports pass. See plan.md:20 and plan.md:298.
IMPLEMENTATION LOCATION: .github/workflows/merge-gates.yml:37, README.md:111
EXPLOIT PATH: The repo contains a workflow, but no enforceable branch-protection or merge-queue
configuration. The merge-gates job is skipped on ordinary pull_request events and only runs on merge_group
or push.
HOW THE IMPLEMENTATION STILL PASSES: The code and reports can pass locally or in CI while actual repository
settings do not require the job before merge.
WHY THIS VIOLATES THE SPEC: A workflow file plus README instruction is not proof that merge is blocked if
the gate is absent.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: A verifiable branch-protection or merge-queue ruleset
export showing merge-gates is a required pre-merge status check for main.
Finding 2
SEVERITY: CRITICAL
SPEC REQUIREMENT: Compliance evidence must not be self-validating; every obligation needs artifact,
provenance, merge-blocking enforcement, and failure if absent.
IMPLEMENTATION LOCATION: crates/ci_reports/src/main.rs:360, crates/ci_reports/src/main.rs:375
EXPLOIT PATH: The CI binary writes the reports, checks their presence, and emits "merge_blocking": true
itself.
HOW THE IMPLEMENTATION STILL PASSES: The same process that generates evidence declares the compliance model
satisfied.
WHY THIS VIOLATES THE SPEC: The merge-blocking claim is not independently measured; it is a constant in a
generated artifact.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Compliance report generated or attested by an external CI
controller with immutable run id, workflow id, and required-check status.
Finding 3
SEVERITY: HIGH
SPEC REQUIREMENT: Measured artifacts need a provenance chain from artifact to run.
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:553, crates/ci_reports/src/lib.rs:667, crates/
ci_reports/src/main.rs:219
EXPLOIT PATH: The Merkle root is computed from in-memory replay hashes; the leaves, inputs, seeds, reference
outputs, and runtime-under-test outputs are not persisted.
HOW THE IMPLEMENTATION STILL PASSES: The report exposes only root and count, and internally checks only
merkle_leaves.len() == equiv_total.
WHY THIS VIOLATES THE SPEC: A root without independently replayable leaves is not a provenance chain; it is
a summary generated by the audited process.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Persisted per-execution records sufficient to recompute
the Merkle root and verify reference/runtime comparison independently.
Finding 4
SEVERITY: HIGH
SPEC REQUIREMENT: Full trace information may not be replaced by summarized proxy; collapse gates must prove
smaller models cannot predict behavior.
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:343, crates/ci_reports/src/lib.rs:720, crates/
ci_reports/src/lib.rs:178
EXPLOIT PATH: Collapse analysis uses a 76-feature aggregate row and only scale.collapse_samples rows. Merge
default is 5,000 samples, and MAGICKA_COLLAPSE can lower it because no merge floor applies.
HOW THE IMPLEMENTATION STILL PASSES: Compression gates run on the aggregate subset, not on full serialized
traces or all executions.
WHY THIS VIOLATES THE SPEC: This is summary/subset/proxy laundering for a stronger trace-information
requirement.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Collapse artifacts over all merge executions using full
serialized ExecutionTrace records, with no lowering override.
Finding 5
SEVERITY: HIGH
SPEC REQUIREMENT: 500 semantic mutants minimum; every mutant must fail at least one named acceptance gate.
IMPLEMENTATION LOCATION: crates/ci_reports/src/lib.rs:723, crates/semantic_mutation/src/lib.rs:207, crates/
semantic_mutation/src/lib.rs:345
EXPLOIT PATH: Mutants are evaluated against at most 64 inputs and mirrored mini-gates, not the actual full
acceptance gates. Domain, temporal, and causal checks omit large parts of the real gates.
HOW THE IMPLEMENTATION STILL PASSES: mutation.passed() only requires no survivors under these local
evaluators.
WHY THIS VIOLATES THE SPEC: A mirrored evaluator over a representative input slice is not “the named
acceptance gate.”
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Survivor report showing each mutant executed against the
actual merge gates and full acceptance corpus.
Finding 6
SEVERITY: HIGH
SPEC REQUIREMENT: Replay corpus: every failure becomes permanent.
IMPLEMENTATION LOCATION: crates/replay_corpus/src/lib.rs:61, crates/replay_corpus/src/lib.rs:151
EXPLOIT PATH: The corpus is generated from deterministic master seeds and current reference outputs. There
is no path that captures CI failures and appends them to the committed corpus.
HOW THE IMPLEMENTATION STILL PASSES: Replay verifies 10,000 static rows have no drift.
WHY THIS VIOLATES THE SPEC: Static seed replay is not permanent retention of every discovered failure.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Corpus history or artifact proving failing cases from
prior CI runs are persisted and rechecked.
Finding 7
SEVERITY: HIGH
SPEC REQUIREMENT: Web Phase H requires Playwright end-to-end tests and 100 browser E2E matches.
IMPLEMENTATION LOCATION: plan2.md:210, .github/workflows/web-gates.yml:45, crates/web_tests/tests/e2e.rs:1
EXPLOIT PATH: The merge-blocking “100 E2E” test is explicitly headless protocol/socket coverage. Rendered-
browser Playwright is advisory and continue-on-error.
HOW THE IMPLEMENTATION STILL PASSES: Browser UI can fail while merge-blocking Rust socket tests pass.
WHY THIS VIOLATES THE SPEC: Browser E2E is substituted with protocol E2E.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Required, non-advisory Playwright browser E2E job running
the 100-match browser gate before merge.
Finding 8
SEVERITY: MEDIUM
SPEC REQUIREMENT: Generated case gates include future dependence within 3 turns and hidden/observed
divergence.
IMPLEMENTATION LOCATION: crates/generators/src/lib.rs:240, crates/generators/src/lib.rs:242, crates/
generators/src/lib.rs:278
EXPLOIT PATH: Future dependence is approximated by presence of a Schedule opcode. Hidden/observed divergence
is approximated by nonzero hidden state or any masked lane, not measured behavior.
HOW THE IMPLEMENTATION STILL PASSES: A case can pass generated gates based on structure even if runtime
behavior does not satisfy the stated property.
WHY THIS VIOLATES THE SPEC: Structural indicators are reported as generated-case requirements.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Generated-gate artifact based on measured execution
traces and measured 3-turn future sensitivity.
Finding 9
SEVERITY: MEDIUM
SPEC REQUIREMENT: Perturbations are generated from domain surfaces, not a fixed list, and expected trace
differences must be meaningful.
IMPLEMENTATION LOCATION: crates/world_model/src/domain.rs:184, crates/generators/src/lib.rs:143, crates/
ci_reports/src/lib.rs:606
EXPLOIT PATH: Each domain exposes a small hard-coded axis set. The metamorphic gate mostly compares hashes
and only uses neutral_explanation; it ignores expect_trace_change, expect_delta_change, and
expect_future_change.
HOW THE IMPLEMENTATION STILL PASSES: Aggregate perturbation thresholds can pass without proving surface-
derived coverage or per-axis expectations.
WHY THIS VIOLATES THE SPEC: Fixed-axis perturbations and unused expectations are weaker than the required
metamorphic contract.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Per-axis report proving generated axes derive from read/
write surfaces and each expected difference is enforced.
Finding 10
SEVERITY: MEDIUM
SPEC REQUIREMENT: Trace evidence must explain causality; reject if trace evidence cannot explain causality.
IMPLEMENTATION LOCATION: crates/trace_model/src/lib.rs:353, crates/ci_reports/src/lib.rs:741
EXPLOIT PATH: Trace gates check counts, rank, touched domains, fingerprint collisions, and largest cluster.
They do not verify that causal edges are independently reconstructable from opcode semantics and world
state.
HOW THE IMPLEMENTATION STILL PASSES: A runtime can emit plausible high-rank causal edges and pass aggregate
metrics.
WHY THIS VIOLATES THE SPEC: Trace quantity is treated as causal explanation.
MINIMUM EVIDENCE REQUIRED TO DISPROVE THE FINDING: Independent causal audit artifact mapping trace edges
back to executed tokens, source values, destination values, and state transitions.
output.txt
-271
View File
@@ -1,271 +0,0 @@
=============
README.md
=============
# Magicka VM — Phase 0/1
> The deliverable is a Rust engine whose tests make a fake universe fail.
This repository implements the Phase 0/1 specification in `plan.md`: an
**adversarial testing framework first**, then a **reference runtime** that
passes it, then a **runtime under test** that matches the reference. No spell
content, templates, or cosmetic runes — the value is in the tests that refuse
to let the universe collapse into a single score, resource, effect axis,
executor, rune, hidden formula, or decorative domain.
## Compliance model
No gate may pass from configuration, naming, shared implementation, smoke-scale
runs, regenerated expectations, proxy metrics, a default profile, or a
locally-runnable binary. A gate passes only from persisted, independently
reproducible, full-scale adversarial evidence enforced at merge. Every
acceptance obligation has all four of: a **measured artifact**, a **provenance
chain** to the run that produced it, a **merge-blocking enforcement point**, and
a **failure condition if the artifact or provenance is absent**.
- The merge-blocking enforcement point is `.github/workflows/merge-gates.yml`,
whose `merge-gates` job runs `MAGICKA_PROFILE=merge` (full gates) and must be a
**required status check** on the protected branch / merge queue. It is not a
local binary, and the fast profile is advisory only — it can never stand in for
acceptance.
- `compliance_report.json` enumerates every obligation, its artifact, its floor,
the actual measured value, and whether the artifact is present. A missing
required report fails acceptance.
- The merge floors (50k worlds, 250k programs, 1,000,000 executions, 10
perturbations/exec, 100% reference/runtime comparison over base **and**
perturbations, 500 mutants, 10,000 replay cases) cannot be lowered by
environment overrides: a lowering override is recorded as a provenance failure
and the floor is kept.
Every gate is built to be *able to fail*, and a negative-control test proves it does:
| Gate | How it is made unbypassable | Negative control proving it can fail |
|------|-----------------------------|--------------------------------------|
| runtime_equivalence | Compares two **independent implementations** (the reference engine vs. `runtime_under_test::native`, which never calls the reference engine) | `buggy_runtime_is_rejected` — a runtime with one dropped causal edge is caught |
| compression_resistance | Attacks operate on the **real serialized trace** (causal influence, info-flow, access, temporal, deltas), not a hash proxy; info loss is genuine unexplained variance | `single_factor_corpus_is_rejected` — a rank-1 universe is rejected |
| mutation_survivor | Each mutant must fail the **named gate** it targets, not merely differ from the reference | `reference_passes_every_named_gate` + `no_mutant_survives_its_named_gate` |
| replay | Expectations are **loaded from a committed file**, not regenerated in the same run | `corrupted_expectation_is_detected` |
| domain_participation | Decorative/redundant domains are flagged directly | `decorative_domain_is_rejected` |
| merge scale floor | Env overrides may only **raise** merge counts; a lowering attempt is recorded and the floor kept; executions actually performed are counted | `merge_floor_cannot_be_lowered_by_override`, `merge_profile_at_smoke_scale_is_rejected` |
| 100% comparison | Reference vs. runtime-under-test compared for **every** execution — base and all perturbations, never base only | `runtime_equivalence` gate fails unless `equiv_total == base + perturbations` |
| provenance | A Merkle root over per-execution records, plus independent engine identities, binds reported numbers to executed work | `merkle_root_binds_to_leaves` |
## Workspace layout
Built in the mandatory order from the spec:
| # | Crate | Role |
|---|-------|------|
| 1 | `world_model` | 8 independent domains, world snapshot, perturbation axes, deltas, deterministic primitives (ids, stable hash, RNG) |
| – | `rune_ir` | Rune token / program model (no stream is ever rejected) |
| 2 | `trace_model` | Execution trace + all graphs, behavior fingerprint, replay record, fault log, trace metrics |
| 3 | `generators` | Worlds, programs, executors, contracts, perturbations; rejects flat cases |
| 4 | `collapse_analysis` | The 11 compression attacks over real trace structure + collapse gates |
| 5 | `semantic_mutation` | Structurally generated mutant runtimes; proves every one fails its named gate |
| 6 | `replay_corpus` | Permanent, bit-exact replay cases persisted to `corpus/replay_corpus.tsv` |
| 7 | `reference_runtime` | The executable spec engine (`Runtime` trait, `resolve`) |
| 8 | `runtime_under_test` | An **independent** interpreter (`native`) proven equivalent to the reference |
| – | `ci_reports` | Orchestrator + `ci` binary; emits 8 gate reports + a provenance report |
The runtime under test does not call the reference engine. It re-derives the
canonical behavior from the spec in a different code organization, so 100%
agreement is *evidence* the spec is implemented correctly rather than a
tautology. (`native_matches_reference_bit_for_bit` checks this over a 2000-seed
sweep.)
## The engine in one paragraph
A world is 8 domains, each with 4 observed + 2 hidden integer lanes, a dense
8×8 coupling matrix, partial observability, and pending scheduled effects. A
rune program is interpreted under ≥3 executors; each opcode reads several
domains, mixes them through a nonlinear avalanche keyed by per-domain
constants, the world coupling, and the executor's salt, then writes back —
recording causal/read/write/information-flow/temporal edges as it goes.
Scheduled effects and coupling diffusion propagate changes 3 turns into the
future.
## Running CI
```bash
cargo test # unit tests + negative controls
MAGICKA_PROFILE=fast cargo run --release -p ci_reports --bin ci # advisory PR slice
MAGICKA_PROFILE=merge cargo run --release -p ci_reports --bin ci # acceptance (full gates)
```
Reports are written to the output dir (8 gate reports + `provenance_report.json`
+ `compliance_report.json` + `ci_summary.md`). The binary exits non-zero if any
gate fails or any required artifact is absent.
### Profiles
`MAGICKA_PROFILE` (or `MAGICKA_SCALE`) selects the run profile.
| Profile | executions | replay | mutants | role |
|---------|-----------|--------|---------|------|
| `fast` (default) | 600 | 10,000 (committed) | 520 | **advisory only — never acceptance** |
| `tiny` | 120 | 10,000 | 520 | smoke |
| `merge` (`MAGICKA_SCALE=full`) | 1,000,000 | 10,000 | 600 | **acceptance — hard floors** |
The fast/tiny profiles print `ADVISORY … NOT a merge-blocking acceptance run`
and are labelled non-acceptance in `compliance_report.json`. Acceptance comes
only from the merge profile, run by the merge-gates workflow. The merge floors
cannot be lowered by environment overrides (a lowering override is recorded as a
provenance failure and the floor kept).
### Merge-blocking enforcement (required check)
`.github/workflows/merge-gates.yml` defines the enforcement point. Configure
branch protection / the merge queue to **require** the `merge-gates` job. That
job runs the full merge profile, verifies the committed corpus has ≥10,000
cases, and fails if any required artifact is missing. The full run executes
~1M base executions × (1 base + 10 perturbations) with 100% reference/runtime
comparison; it completes in minutes on a CI runner.
### Replay corpus
The replay corpus is committed at
`crates/replay_corpus/corpus/replay_corpus.tsv` (10,000 cases). Replay loads
those expectations and re-executes the reference, so any engine change that
alters a hash makes the committed file and the fresh run disagree and CI fails.
Regenerate it only as a deliberate, reviewed migration:
```bash
cargo run --release -p replay_corpus --bin freeze -- 10000
```
## Determinism
Everything is seed-derived and integer-only (SplitMix64 RNG, FNV-1a content
hashing, wrapping/guarded arithmetic). No floating point enters a canonical
hash, so replay is bit-exact across machines and runs. No external crates.
## The web game (plan2.md)
A browser game is built **around** the existing runtime — it is a playable
window into the Rust universe, never a second simulation. The browser sends only
*intent*; the server is the sole authority; every rune program executes through
the **independent** interpreter (`runtime_under_test::native_resolve`) against
the shared world. The game deliberately does **not** call the reference engine —
the interpreter it uses is the one the runtime-equivalence gate proves correct
(with a negative control proving that gate can fail). Same constraints as the
rest of the repo: pure `std`, no external crates (the WebSocket server
hand-rolls SHA-1, base64, and RFC 6455 framing; JSON is hand-rolled with a total
parser).
> Audit note: the hand-rolled SHA-1 / base64 / RFC-6455 framing and JSON parser
> are checked against published test vectors (RFC 6455 §1.3 accept key, SHA-1
> "abc", base64 length cases) and a fuzz gate, but they are bespoke
> cryptographic/parsing code and carry audit risk relative to a reviewed
> library. They exist to honor the repo's no-external-crates rule; a future
> hardening pass could swap in vetted implementations behind the same interface.
```
Rust runtime → game_runtime (authority) → protocol (WS messages) → server → browser
```
| Crate | Role |
|-------|------|
| `protocol` | Versioned, hashable, **total-decode** client/server messages + JSON value/parser. A malformed packet yields `Err`, never a panic. |
| `game_runtime` | Authoritative match state. Resolves turns through the **independent interpreter** (`runtime_under_test`, not the reference engine), filters visibility/knowledge, records + regenerates replays. A match is a pure function of `(seed, roster, ordered inputs)`. |
| `web_assets` | The embedded browser client (HTML/CSS/JS): arena, rune editor, domain/knowledge panels, replay viewer. |
| `web_client` | Static-asset HTTP delivery (keeps raw assets separate from framing). |
| `server` | `std::net` HTTP + WebSocket server: turn timer, action collection, disconnect handling, panic-proof dispatch. |
| `web_tests` | A dependency-free WebSocket test client + the Phase H gates. |
### Running it
```bash
cargo run --release -p server --bin magicka-server # serve on 127.0.0.1:8080
# then open http://127.0.0.1:8080 in a browser
MAGICKA_ADDR=0.0.0.0:9000 MAGICKA_TURN_MS=8000 cargo run --release -p server --bin magicka-server
```
Join is immediate (1 player + a training dummy). A duel shares a match by id:
two browsers that `JoinMatch` the same `match_id` take slots 1 and 2.
### Web CI gates (Phase H)
These gates are **merge-blocking**: they run inside the merge-required job in
`.github/workflows/merge-gates.yml` (and as fast PR feedback in
`web-gates.yml`). They are the Rust suite in `crates/web_tests`, run with
`cargo test -p web_tests`:
| Gate | Test | Minimum | Status |
|------|------|---------|--------|
| Replay determinism | `determinism.rs` | 1,000 simulated matches, **0 hash mismatches** | merge-blocking |
| Protocol fuzz | `fuzz.rs` | 10,000 fuzz cases, **0 panics** (+ a live server survives a malformed-packet burst) | merge-blocking |
| End-to-end matches | `e2e.rs` | **100** full matches over real sockets; recorded replay reproduces every live per-turn hash | merge-blocking |
| Hidden-state leaks | `visibility.rs` | **0 leaks** — no client-bound frame carries a hidden key; redaction counts every withheld value | merge-blocking |
| Disconnect / timer edges | `resilience.rs` | mid-match disconnect does not corrupt the match; wrong-turn / late submits are rejected deterministically | merge-blocking |
| Rendered-browser E2E | `e2e/specs/play.spec.js` | a real browser joins, casts, and replays a match | **external-blocked (advisory only)** |
Scope honesty — two distinct things, not conflated:
- The "100 E2E matches" merge-blocking gate drives the full
HTTP→WebSocket→protocol→runtime path **headlessly over real sockets**. This is
protocol-level coverage. It is **not** rendered-browser coverage and is not
claimed as such.
- Rendered-browser coverage is **blocked on CI infrastructure**: this CI has no
real browser, so the Playwright suite under `crates/web_tests/e2e/` cannot be
merge-blocking yet. It runs **advisory-only** (`continue-on-error`) in the
`rendered-browser-e2e` job and uploads its report as an artifact. Until a CI
runner with a browser exists, rendered-browser E2E is treated as
**unsatisfied**, not green. Run it locally with:
```bash
cd crates/web_tests/e2e && npm install && npx playwright install chromium && npm test
```
### Acceptance criteria mapping (plan2.md)
| Criterion | Where it holds |
|-----------|----------------|
| A player can join a browser match | `server` join + `web_assets` client; `e2e.rs::single_match_full_playthrough` |
| A turn timer runs | `server` timer thread; client header countdown |
| Inspect / move / attack / cast | `Action` in `protocol`; `game_runtime::apply_action` |
| Rune programs execute only on the server | `game_runtime` is the only caller of the interpreter (`runtime_under_test::native_resolve`); client never imports `EngineConfig` (asserted in `web_assets`) |
| Results return as filtered observations | `VisibleWorldSnapshot`; `visibility.rs` |
| Replay can reproduce the match | `game_runtime::replay`; `determinism.rs`, `e2e.rs` |
| Browser cannot alter hidden truth | intent-only protocol; `visibility.rs` leak gate |
| CI proves protocol, replay, visibility, authority | merge-blocking gates in `merge-gates.yml` (+ `web-gates.yml`); rendered-browser E2E remains external-blocked |
=============
Output
=============
All four directives are done and verified.
What changed
Committed — branch web-game, commit 9d9d5ce. Full workspace is green (every test passes, zero warnings); the
claims are now in git and reproducible. I put it on a branch rather than main; fast-forward it when you're
ready.
Removed reference-engine delegation. game_runtime no longer calls reference_runtime::execute. It now drives
runtime_under_test::native_resolve — the independent interpreter that the runtime-equivalence gate proves
correct (with a negative control proving that gate can fail). The grep confirms the only remaining
reference_runtime use is the shared I/O contract types (ResolutionInput/canonical), not execution. So the
game exercises an independent runtime, not a re-run of the reference.
Protocol/socket gates are merge-blocking. Added an explicit named step to the merge_group-required job in
merge-gates.yml running the protocol + socket + replay/visibility/resilience suite (1k matches/0 drift, 10k
fuzz/0 panics, 100 headless socket E2E, 0 leaks). web-gates.yml keeps the same suite as fast PR feedback.
Rendered-browser E2E is external-blocked, not satisfied. The Playwright job is renamed rendered-browser-e2e
(ADVISORY — blocked on CI infra), set continue-on-error, and uploads its report as an artifact only. The
README states plainly that until a CI runner with a browser exists, rendered-browser coverage is
unsatisfied, and that the 100-match gate is protocol-level coverage — not a browser.
Two things I want to be straight about rather than let them read as resolved:
- The hand-rolled SHA-1/base64/RFC-6455/JSON code carries real audit risk. I checked it against published
vectors and a fuzz gate and documented the risk in the README, but it's bespoke parsing/crypto written to
honor the no-external-crates rule — not a substitute for a reviewed library. I did not eliminate that risk;
I labeled it.
- The independent interpreter is independent of the reference engine's code path, but both live in this repo
and are proven equivalent by the same gate suite — so "independent runtime correctness" rests on that gate,
not on a third-party oracle.