How the run is built, from the container boundary down to the arithmetic.
The reasons behind each piece are on the design decisions page.
rk-work holds the run's data: the append-only archive of every verified
tableau and the hypothesis ledger. The event stream and run state are gitignored and stay
on the host. A fifth, private repository holds the scripts and configuration that run all
of this on one machine; it is not linked here.
The design assumes the search would optimize the scorer instead of the score if it could. Four boundaries rule that out.
rk-harness is mounted read-only, and the
container will not start if the verifier hash has changed
(verification).The runner executes one cycle at a time. Replay rebuilds all state from the append-only archive, nothing before the fsynced append has side effects, and state files are written atomically. Replay also drops a torn last line, so a crash costs at most one cycle and repeating a cycle is safe.
Each cycle belongs to one method class, and the loop rotates among them. An explicit cycle runs the diagram above. An adaptive or implicit cycle runs that class's lane search instead: it enumerates candidates deterministically and measures how many cycles each needs to reach a tolerance on the validation problems, in float64.
Lane cycles append to their own archives, rk-work/adaptive_archive and
rk-work/implicit_archive. The verifier hash does not pin them and no score
reads them, so the explicit archive stays the only scored record. Each lane keeps a
capped elites document ranked by cycles to tolerance, and that document is what the
findings class pages publish. A separate side-track executor measures fixed design
questions for the same two classes and records every point in
rk-work/sidetrack/ledger.jsonl.
Six modules decide every score and four fixture files pin their golden values.
VERIFIER_HASH is a sha256 over those ten files. The entrypoint recomputes it
at every start and every archive record stores it, so a change to any byte shows up
twice. The pre-flight drill changed one line of coeffrep.py and the
container refused to start with "VERIFIER HASH MISMATCH". With the line restored, it
started normally.
Once the runner is up, each candidate passes nine checks, all plain code. Only the primary cost model's columns can reject a candidate; the slow and AVR columns are kept for breadth.
NOT_EXPLICIT: A must be strictly lower triangular.ROW_SUM_INCONSISTENT: c must equal the row sums of A.DYADIC_IMPOSSIBLE: no all-dyadic tableau reaches order 3, because the
order-3 condition sums to 1/3, which no dyadic rational equals.ORDER_NOT_MET: exact rational residuals over rooted trees, never
floats.COEFF_UNREPRESENTABLE: a range check only. An inexact coefficient such
as 1/3 is recorded, not rejected.Q15_OVERFLOW: the trajectory must keep a 2× amplitude margin.UNSTABLE: the stability interval must be usable.NO_ASYMPTOTIC_WINDOW: the measured-order fit found no consistent
slope.NAN_OR_INF: any non-finite value anywhere is fatal.States are Q15: int16 at scale 2−15. A multiply is
(a*b) >> 15 with an arithmetic shift, which rounds toward negative
infinity, as ARM's ASRS does and C's truncation does not (q15_mul(−1, 1) is
−1 under floor and 0 under truncation); the vectors where the two disagree are
pinned as fixtures. Nothing saturates and nothing wraps. A value outside int16 raises and
becomes a verifier rejection, because the M0+ has no SSAT and saturating in software
would need a branch, which breaks exact cycle counts.
The harness keeps floor rounding because shipped code would floor too; key finding 2 measures what it does to the error.
Coefficients are not Q15. Each one is an integer and a shift, m/2s,
applied as (v*m) >> s, since rk4's 1 and kutta3's 2 do not fit in
[−1, 1). When no exact pair exists (1/3, for instance) the closest one is used and
the gap is recorded as quantization error, a measured property rather than a
rejection.
A derivative can be larger than the state it drives, so such a problem stores it with an extra power-of-two factor (DERIV_SCALE, 1/8 on rc_thermal, 1 elsewhere) that the step undoes exactly through the step-size constant.
Cycles are counted analytically, with no compiler, emulator or board in the loop, so
the count is a pure function of the tableau and the same on every machine. Three models
price five kinds of operation; the table is read from the pinned
costmodel module.
| model | mul | add | shift | load | store | role |
|---|---|---|---|---|---|---|
| m0plus_fast | 1 | 1 | 1 | 2 | 2 | M0+ with a single-cycle multiplier; primary |
| m0plus_slow | 32 | 1 | 1 | 2 | 2 | M0+ with an iterative multiplier; primary |
| avr_approx | 14 | 2 | 8 | 2 | 2 | 8-bit AVR approximation; advisory only |
A coefficient of 0 or ±1 costs nothing extra. Any other costs the cheaper of a shift-add expansion (w shifts and w − 1 adds, w being the CSD weight of m) and one hardware multiply plus a shift. With a 1-cycle multiplier the multiply nearly always wins; with a 32-cycle one the shift-add usually does, so low-weight coefficients get much cheaper. Derivative evaluation is left out, since it costs the same for every method with the same stage count. A hand-counted ARMv6-M sequence pinned as a fixture must match the model under both M0+ models, and the anchor result is pinned as golden test G21.
The model is analytic, so for most of the run nothing had compiled the instruction
sequence it describes. That check now exists, off the run: the Q15 step was compiled for
cortex-m0plus with arm-none-eabi-gcc at -O2, executed under an instruction-accurate
emulator, and the executed instructions were priced with the same table. The emulator
reports which instructions run and in what order rather than what a part would take, so it
is not cycle accurate and the comparison does not claim to be. At the scope the model prices, the stage and b combinations and nothing else, the traced count runs 1.00 to 2.03 times the analytic one across 9 methods, and 6 of them sit more than 25% away, the widest being rk4 at 51%. The gap is in the model rather than in the measurement: in these nine traces the compiler turns every applied coefficient, dyadic or not, into shifts and adds, so coeff_cost prices a multiply it never emits in these nine traces. The ordering moves with it. Under the slow multiplier, where the model already expects a shift-add chain, 2 pairs change places at this scope: the model puts 11e898cb under ralston2 at 29 cycles against 30, and the trace has them at 35 and 27; the model puts rk38 under kutta3 at 64 cycles against 65, and the trace has them at 59 and 49. The per-method
rows are on the findings validation page.
Phases 0 and 1 enumerate their spaces completely, so their results hold over the
whole enumerated space, and the findings site labels them that way; phase 0 is
key finding 5. Phases 2 and 3 run CMA-ES islands over
the lower triangle of A, snapped to dyadic values, then solve b exactly over the
rationals from the order conditions, which are linear in b once A is fixed. Snapping b
as well would make order 3 and above impossible (the DYADIC_IMPOSSIBLE
check above). When the requested order has no exact solution for a snapped A, as often
happens at order 4, the projection falls back to the highest order it can solve and the
candidate is verified at that order.
CMA-ES minimizes search-set error plus a large penalty on order-condition residuals; only the three search problems are visible to it.
Every verified candidate becomes one fsynced JSON line in a per-day file: the tableau as exact fractions, its content hash, the full score vector, the tier, the verifier hash in force, what motivated it, and a UTC timestamp. Replay drops a line that fails to validate and rejects a record whose hash does not match its tableau.
The working structure is MAP-Elites: one grid per order, with cells keyed by stage count and a log2 band of the m0plus_fast cycle count. A cell's elite is the record with the lowest held-out error, and a tie keeps the earlier record. Replay also keeps running statistics per order and stage count, which the hypothesis ledger's predicates read. A problem admitted from quarantine runs in shadow for a fixed number of cycles before it counts.
The language model (Codex, authenticated on the host, mounted read-only) never touches the inner loop. It emits JSON directives that can only narrow the search, such as target order or stage counts. A schema rejects unknown keys and bounds every field, and a malformed directive is discarded in favor of a deterministic fallback that searches the emptiest grid cell. Calls stop at a plan-usage cap.
The model in the outer loop does see held-out results, and this is exactly what it sees. Every directive, hypothesis and interpretation prompt carries one held-out error per occupied archive cell, plus the count, mean and lowest held-out error for each (order, stage count) group. A code path reads the same number without asking the model: when the gap between search-set and held-out error widens, the encourager rotates problems. What never reaches the model is a per-problem held-out value, the practical validation suite or the benchmark, and the inner optimizer is still scored on search-set error alone. So the held-out set steers where the search looks rather than sitting outside the loop, and the elites chosen on it carry selection bias.
The model also proposes falsifiable hypotheses. Code assigns their ids and verdicts, and each predicate must parse under a closed grammar with a hand-written parser. No eval runs anywhere near model output, and a grep canary enforces that. A field such as fast.p3s4.heldout resolves against the archive's cell statistics; how the numbers were made covers how a verdict is assigned.
expr := term (("AND" | "OR") term)*
term := field op field | field op number
field := model "." cell "." metric
model := "fast" | "slow" | "avr_approx"
cell := "p" digit "s" digit
metric := "heldout" | "search" | "cycles" | "order"
op := "<" | ">" | "<=" | ">=" | "=="
A literature loop searches the web for one topic at a time and feeds each digest into later prompts. The digests and the model's interpretations are published in the findings site's research log (digests, interpretations), labeled as model-written, after a softener removes priority-claim words.
A PowerShell watchdog on the host kills the container on a stale heartbeat, stops it
on overspend or low disk, and pauses it on battery or foreground CPU load. A wrongful
kill heals itself, since the restart policy brings the container back through the start
gate. Operational settings live in a workspace config.json; scientific
thresholds live in the hash-pinned code, because changing one would invalidate the
archive. Storage is UTC everywhere, and only display code converts to US Central.
The suite collects 1,833 tests in 18 numbered tiers that follow the dependency stack, from fixed-point arithmetic up to the findings pages. Golden tests (G-numbered) pin behavior to fixture values written before the code existed, down to exact measured orders and cycle counts. Canary tests (K-numbered) guard against gaming: K1 plants a candidate tuned on the search set and asserts that it can earn search_only but never heldout_verified, and K2 asserts that a winner on a single problem family lands at no_improvement, the tier for a candidate whose cell had an incumbent and for which neither the heldout_verified nor the search_only rule applied. Its counterpart no_incumbent marks a record written into an empty cell. Before the split both were stored as one word, unreplicated, and no record in the archive this page was built from carries it. The 55 golden and canary cases are also the container's start gate.
| tier | tests | covers |
|---|---|---|
| T1 | 353 | fixed point, coefficient representation, cycle counting |
| T2 | 216 | order conditions, evaluator, verifier |
| T3 | 276 | archive, search, directive validation |
| T4 | 259 | ledger, runner, site generator, epoch panel |
| T5 | 101 | operational config, the watch view, status, checkout hygiene |
| T6 | 8 | Central-time display formatting |
| T7 | 23 | the findings methodology page |
| T8 | 69 | the practical validation suite, stiff subset included |
| T9 | 16 | the epoch saturation orchestrator |
| T10 | 55 | the library benchmark harness |
| T11 | 16 | the adaptive embedded-pair prototype |
| T12 | 7 | the SDIRK implicit prototype |
| T13 | 29 | the side-track executor and its ledger |
| T14 | 23 | second-pass metrics over the archive: cycles to tolerance, the stability frontier |
| T15 | 25 | the stiff-problem screen, admissions and rejections alike |
| T16 | 208 | the lane scheduler, the lane cycle and the lane search |
| T17 | 62 | the two-axis validation document across the three method classes |
| T18 | 87 | the per-class findings pages: explicit, implicit, adaptive |
scripts/preflight.py executes every item on the pre-flight checklist
that a machine can check, and writes the report. Its drills act rather than assert: A1
writes to /harness from a running container and expects "Read-only file
system", and D5 feeds the optimizer the held-out set, watches the search/held-out gap
collapse to zero, and watches it come back on revert.
The same seed produces a byte-identical archive, and an acceptance test requires it: the evaluator reads no clock and draws no random numbers, and derived state replays from the archive files alone. The findings site is a pure function of the archive. This site is not, because it also reads the test collection and the latest analysis at build time.