Architecture

How the run is built, from the container boundary down to the arithmetic.

The reasons behind each piece are on the design decisions page.

The repositories

One writer and one trust level per repository. Boxes are git repositories and the dashed enclosure is the container boundary. The findings site moves with the run; this site is a snapshot.drawn from the repository layout; the test count is the suite as collected at build timeas of 2026-09-10 (US Central), from the code and data at build time
mounted into the run containerrk-harnessthe scorer: verifier, evaluator,cost models, 1,833 testsread-only in the containerrk-workrun data: append-only archive,hypothesis ledger, resultsthe one writable mountrk-findingsmachine-generated numbers site,rebuilt by the run every cycleLIVErk-overview (this site)hand-written explainer pages,built from the archive by handSNAPSHOTverifies + scoresrebuilt every cycleread by a hand-run build, 2026-09-10jgoetzmann.github.io/rk-findings ↗ (live)jgoetzmann.github.io/rk-overview (you are here)

rk-work holds the run's data: the append-only archive of every verified tableau and the hypothesis ledger. The event stream and run state are gitignored and stay on the host. A fifth, private repository holds the scripts and configuration that run all of this on one machine; it is not linked here.

Trust boundaries

The system as built, left to right: the Windows host, the docker container, and the services the run talks to. The verifier sits inside the read-only mount, and no arrow carries the GitHub credential into the container.drawn from the run scripts and the container configurationas of 2026-09-10 (US Central), from the code and data at build time
Windows hostwatchdog: heartbeat kill,spend / disk stop, battery pause,git pushes + config + watchercredentialsGitHub PAT: host only, not mountedCodex auth: mounted read-onlyrun container (docker)rk-harness (read-only mount)verifier + evaluator + cost modelssha256 pinned, checked at startrunnercycle loop; the only LLM callersearchCMA-ES islands + enumerationrk-work (writable mount)append-only archive, events, ledgerCodex (planning)directives, hypotheses, digestsrk-findingsauto site, rebuilt every cycleGitHub Pagesdeploys on push from the hostdocker rungit push (host)throttled LLM callscommit each cycle

The design assumes the search would optimize the scorer instead of the score if it could. Four boundaries rule that out.

The cycle loop

One explicit cycle, along the top row and then the bottom.drawn from the runner's cycle loopas of 2026-09-10 (US Central), from the code and data at build time
replayarchive → stateencouragerladder + calendarcandidatesenumerate / CMA-ESverify ×9nine checks, exactevaluateQ15, 3 cost modelstiervs cell incumbentappendfsync JSONLledgerverdicts by codesiterebuild + guardcommithost pushes later

The runner executes one cycle at a time. Replay rebuilds all state from the append-only archive, nothing before the fsynced append has side effects, and state files are written atomically. Replay also drops a torn last line, so a crash costs at most one cycle and repeating a cycle is safe.

Three classes, taking turns

Each cycle belongs to one method class, and the loop rotates among them. An explicit cycle runs the diagram above. An adaptive or implicit cycle runs that class's lane search instead: it enumerates candidates deterministically and measures how many cycles each needs to reach a tolerance on the validation problems, in float64.

Lane cycles append to their own archives, rk-work/adaptive_archive and rk-work/implicit_archive. The verifier hash does not pin them and no score reads them, so the explicit archive stays the only scored record. Each lane keeps a capped elites document ranked by cycles to tolerance, and that document is what the findings class pages publish. A separate side-track executor measures fixed design questions for the same two classes and records every point in rk-work/sidetrack/ledger.jsonl.

Verification, in order

The start gate, top to bottom, on every container start. The runner starts only after the read-only probe, the hash check and the golden and canary tests pass against the harness as mounted. The tests run at start rather than at image build, so every restart repeats them.drawn from entrypoint.sh; the gate size is the node ids in tests/golden_gate.txtas of 2026-09-10 (US Central), from the code and data at build time
container start1 · write heartbeatthe first line of the entrypoint2 · read-only probea write to /harness must fail3 · verifier hash checksha256 over ten files vs the pinned value4 · golden + canary tests55 cases with pytestrunner startsscience on a proven environment onlyexit 1the runner never starts;the watchdog restarts, the gate re-runsany failure

Six modules decide every score and four fixture files pin their golden values. VERIFIER_HASH is a sha256 over those ten files. The entrypoint recomputes it at every start and every archive record stores it, so a change to any byte shows up twice. The pre-flight drill changed one line of coeffrep.py and the container refused to start with "VERIFIER HASH MISMATCH". With the line restored, it started normally.

Once the runner is up, each candidate passes nine checks, all plain code. Only the primary cost model's columns can reject a candidate; the slow and AVR columns are kept for breadth.

The nine checks, cheapest first
  1. NOT_EXPLICIT: A must be strictly lower triangular.
  2. ROW_SUM_INCONSISTENT: c must equal the row sums of A.
  3. DYADIC_IMPOSSIBLE: no all-dyadic tableau reaches order 3, because the order-3 condition sums to 1/3, which no dyadic rational equals.
  4. ORDER_NOT_MET: exact rational residuals over rooted trees, never floats.
  5. COEFF_UNREPRESENTABLE: a range check only. An inexact coefficient such as 1/3 is recorded, not rejected.
  6. Q15_OVERFLOW: the trajectory must keep a 2× amplitude margin.
  7. UNSTABLE: the stability interval must be usable.
  8. NO_ASYMPTOTIC_WINDOW: the measured-order fit found no consistent slope.
  9. NAN_OR_INF: any non-finite value anywhere is fatal.

Arithmetic, exactly

States are Q15: int16 at scale 2−15. A multiply is (a*b) >> 15 with an arithmetic shift, which rounds toward negative infinity, as ARM's ASRS does and C's truncation does not (q15_mul(−1, 1) is −1 under floor and 0 under truncation); the vectors where the two disagree are pinned as fixtures. Nothing saturates and nothing wraps. A value outside int16 raises and becomes a verifier rejection, because the M0+ has no SSAT and saturating in software would need a branch, which breaks exact cycle counts.

The harness keeps floor rounding because shipped code would floor too; key finding 2 measures what it does to the error.

Coefficients are not Q15. Each one is an integer and a shift, m/2s, applied as (v*m) >> s, since rk4's 1 and kutta3's 2 do not fit in [−1, 1). When no exact pair exists (1/3, for instance) the closest one is used and the gap is recorded as quantization error, a measured property rather than a rejection.

A derivative can be larger than the state it drives, so such a problem stores it with an extra power-of-two factor (DERIV_SCALE, 1/8 on rc_thermal, 1 elsewhere) that the step undoes exactly through the step-size constant.

The cost model

Cycles are counted analytically, with no compiler, emulator or board in the loop, so the count is a pure function of the tableau and the same on every machine. Three models price five kinds of operation; the table is read from the pinned costmodel module.

modelmuladdshiftloadstorerole
m0plus_fast11122M0+ with a single-cycle multiplier; primary
m0plus_slow321122M0+ with an iterative multiplier; primary
avr_approx1428228-bit AVR approximation; advisory only

A coefficient of 0 or ±1 costs nothing extra. Any other costs the cheaper of a shift-add expansion (w shifts and w − 1 adds, w being the CSD weight of m) and one hardware multiply plus a shift. With a 1-cycle multiplier the multiply nearly always wins; with a 32-cycle one the shift-add usually does, so low-weight coefficients get much cheaper. Derivative evaluation is left out, since it costs the same for every method with the same stage count. A hand-counted ARMv6-M sequence pinned as a fixture must match the model under both M0+ models, and the anchor result is pinned as golden test G21.

The model is analytic, so for most of the run nothing had compiled the instruction sequence it describes. That check now exists, off the run: the Q15 step was compiled for cortex-m0plus with arm-none-eabi-gcc at -O2, executed under an instruction-accurate emulator, and the executed instructions were priced with the same table. The emulator reports which instructions run and in what order rather than what a part would take, so it is not cycle accurate and the comparison does not claim to be. At the scope the model prices, the stage and b combinations and nothing else, the traced count runs 1.00 to 2.03 times the analytic one across 9 methods, and 6 of them sit more than 25% away, the widest being rk4 at 51%. The gap is in the model rather than in the measurement: in these nine traces the compiler turns every applied coefficient, dyadic or not, into shifts and adds, so coeff_cost prices a multiply it never emits in these nine traces. The ordering moves with it. Under the slow multiplier, where the model already expects a shift-add chain, 2 pairs change places at this scope: the model puts 11e898cb under ralston2 at 29 cycles against 30, and the trace has them at 35 and 27; the model puts rk38 under kutta3 at 64 cycles against 65, and the trace has them at 59 and 49. The per-method rows are on the findings validation page.

Where candidates come from

Phases 0 and 1 enumerate their spaces completely, so their results hold over the whole enumerated space, and the findings site labels them that way; phase 0 is key finding 5. Phases 2 and 3 run CMA-ES islands over the lower triangle of A, snapped to dyadic values, then solve b exactly over the rationals from the order conditions, which are linear in b once A is fixed. Snapping b as well would make order 3 and above impossible (the DYADIC_IMPOSSIBLE check above). When the requested order has no exact solution for a snapped A, as often happens at order 4, the projection falls back to the highest order it can solve and the candidate is verified at that order.

CMA-ES minimizes search-set error plus a large penalty on order-condition residuals; only the three search problems are visible to it.

The archive

Every verified candidate becomes one fsynced JSON line in a per-day file: the tableau as exact fractions, its content hash, the full score vector, the tier, the verifier hash in force, what motivated it, and a UTC timestamp. Replay drops a line that fails to validate and rejects a record whose hash does not match its tableau.

The working structure is MAP-Elites: one grid per order, with cells keyed by stage count and a log2 band of the m0plus_fast cycle count. A cell's elite is the record with the lowest held-out error, and a tie keeps the earlier record. Replay also keeps running statistics per order and stage count, which the hypothesis ledger's predicates read. A problem admitted from quarantine runs in shadow for a fixed number of cycles before it counts.

The outer loop

The language model (Codex, authenticated on the host, mounted read-only) never touches the inner loop. It emits JSON directives that can only narrow the search, such as target order or stage counts. A schema rejects unknown keys and bounds every field, and a malformed directive is discarded in favor of a deterministic fallback that searches the emptiest grid cell. Calls stop at a plan-usage cap.

The model in the outer loop does see held-out results, and this is exactly what it sees. Every directive, hypothesis and interpretation prompt carries one held-out error per occupied archive cell, plus the count, mean and lowest held-out error for each (order, stage count) group. A code path reads the same number without asking the model: when the gap between search-set and held-out error widens, the encourager rotates problems. What never reaches the model is a per-problem held-out value, the practical validation suite or the benchmark, and the inner optimizer is still scored on search-set error alone. So the held-out set steers where the search looks rather than sitting outside the loop, and the elites chosen on it carry selection bias.

The model also proposes falsifiable hypotheses. Code assigns their ids and verdicts, and each predicate must parse under a closed grammar with a hand-written parser. No eval runs anywhere near model output, and a grep canary enforces that. A field such as fast.p3s4.heldout resolves against the archive's cell statistics; how the numbers were made covers how a verdict is assigned.

The predicate grammar, in full
expr    := term (("AND" | "OR") term)*
term    := field op field | field op number
field   := model "." cell "." metric
model   := "fast" | "slow" | "avr_approx"
cell    := "p" digit "s" digit
metric  := "heldout" | "search" | "cycles" | "order"
op      := "<" | ">" | "<=" | ">=" | "=="

A literature loop searches the web for one topic at a time and feeds each digest into later prompts. The digests and the model's interpretations are published in the findings site's research log (digests, interpretations), labeled as model-written, after a softener removes priority-claim words.

The host layer

A PowerShell watchdog on the host kills the container on a stale heartbeat, stops it on overspend or low disk, and pauses it on battery or foreground CPU load. A wrongful kill heals itself, since the restart policy brings the container back through the start gate. Operational settings live in a workspace config.json; scientific thresholds live in the hash-pinned code, because changing one would invalidate the archive. Storage is UTC everywhere, and only display code converts to US Central.

Testing

The suite collects 1,833 tests in 18 numbered tiers that follow the dependency stack, from fixed-point arithmetic up to the findings pages. Golden tests (G-numbered) pin behavior to fixture values written before the code existed, down to exact measured orders and cycle counts. Canary tests (K-numbered) guard against gaming: K1 plants a candidate tuned on the search set and asserts that it can earn search_only but never heldout_verified, and K2 asserts that a winner on a single problem family lands at no_improvement, the tier for a candidate whose cell had an incumbent and for which neither the heldout_verified nor the search_only rule applied. Its counterpart no_incumbent marks a record written into an empty cell. Before the split both were stored as one word, unreplicated, and no record in the archive this page was built from carries it. The 55 golden and canary cases are also the container's start gate.

What each test tier covers
tiertestscovers
T1353fixed point, coefficient representation, cycle counting
T2216order conditions, evaluator, verifier
T3276archive, search, directive validation
T4259ledger, runner, site generator, epoch panel
T5101operational config, the watch view, status, checkout hygiene
T68Central-time display formatting
T723the findings methodology page
T869the practical validation suite, stiff subset included
T916the epoch saturation orchestrator
T1055the library benchmark harness
T1116the adaptive embedded-pair prototype
T127the SDIRK implicit prototype
T1329the side-track executor and its ledger
T1423second-pass metrics over the archive: cycles to tolerance, the stability frontier
T1525the stiff-problem screen, admissions and rejections alike
T16208the lane scheduler, the lane cycle and the lane search
T1762the two-axis validation document across the three method classes
T1887the per-class findings pages: explicit, implicit, adaptive

The pre-flight, run as a program

scripts/preflight.py executes every item on the pre-flight checklist that a machine can check, and writes the report. Its drills act rather than assert: A1 writes to /harness from a running container and expects "Read-only file system", and D5 feeds the optimizer the held-out set, watches the search/held-out gap collapse to zero, and watches it come back on revert.

Reproducibility

The same seed produces a byte-identical archive, and an acceptance test requires it: the evaluator reads no clock and draws no random numbers, and derived state replays from the archive files alone. The findings site is a pure function of the archive. This site is not, because it also reads the test collection and the latest analysis at build time.