How the run measures, checks and reproduces its numbers.
The rk run ended on 2026-11-10, and no more compute goes to it. Epoch 2 is frozen and these pages are its final snapshot: the site is no longer rebuilt. The overview site sums up the final findings, and the harness is public for anyone to download and run.
| Object of study | Explicit Runge-Kutta tableaus, orders 1 to 4, stages 2 to 6 |
|---|---|
| State arithmetic | Q15: int16 in [-32768, 32767], scale 2-15 |
| Target | ARM Cortex-M0+, analytic cycle model, two multiplier variants; AVR model advisory only; coefficient prices compiled at pinned flags |
| Evaluation budget | 65,536 cycles per problem run |
| Test problems | 7 fixed (3 search, 4 held out) |
| Verification gate | Nine ordered checks, pure code, never raises |
| Integrity pin | sha256 over 14 files, checked at container start |
rk-harness searches for explicit Runge-Kutta tableaus that give the lowest integration error in Q15 fixed-point arithmetic at a fixed cycle budget, priced by an ARM Cortex-M0+ cost model. Classical tableaus were derived for exact real arithmetic. On a microcontroller with no floating-point unit, roundoff competes with truncation at practical step sizes, and each coefficient costs cycles that depend on its bit pattern[21].
Scoring is pure code: a language model may steer the search and propose hypotheses, but it never scores, tiers or verifies anything. Results are on the other tabs and the machinery on the overview architecture page. Terms are in the glossary below, and a bracketed number such as [21] cites a file in the rk-harness repository, a section of the frozen specification, docs/HANDOFF.md, or a published paper.
The object of study is the explicit tableau with symbolic order 1 to 4 and 2 to 6
stages. A entries sit on dyadic lattices no finer than 1/32768, and each coefficient is
stored as m / 2s with |m| ≤ 32767 and 0 ≤ s ≤ 20. Where no exact
pair exists (1/3 becomes 21845 / 216) the closest one is used and the gap is
recorded as coeff_quant_error, a measured property rather than a reason to
reject[2].
State arithmetic is Q15: 16-bit signed integers at scale 2-15, multiplied
as (a * b) >> 15 with an arithmetic shift that floors toward negative
infinity, as the ARM ASRS instruction does, and an out-of-range value becomes
a verifier rejection. Because the floor is one-sided, each
product loses about half a least-significant bit on average, and at small step sizes
that bias dominates the Q15 error; the harness copies the hardware on purpose, since
code shipped to this target would floor too. The semantics are on the
architecture page and under
floor rounding[1][21].
Methods are always compared at an equal cycle budget, never at equal step size. A tableau costing k cycles per step takes n = B // k steps with B = 65,536 cycles, so a two-stage method takes about twice the steps of a four-stage one. Comparing at equal h would hand the expensive methods free work[5].
Cycle counts are analytic: a pure function of the tableau, with no compiler and no
hardware in the scoring loop. m0plus_fast (single-cycle multiplier) and m0plus_slow
(32-cycle multiplier) are primary; avr_approx is advisory. Under the two M0+ models a
coefficient application costs the instructions arm-none-eabi-gcc 13.2.1 emits for that
term at -mcpu=cortex-m0plus -mthumb -O2 -ffreestanding -nostdlib, counted by op class
(mul, shift, add, load) in the pinned table fixtures/m0plus_coeff_ops.json
and priced by the model's per-class cycles, so the compiler enters the price once, at pin
time, and never per evaluation. The table is keyed by the signed multiplier and by whether
a shift is present, so a negative coefficient can cost more than its positive twin, and a
multiplier outside the two keyed domains raises rather than guessing. A multiplier of plus
or minus 1 at shift 0 costs nothing, because the compiler emits a bare adds or subs that
the combination add already prices; zero entries and exact plus or minus 1 stay free.
avr_approx keeps the earlier charge, the cheaper of a shift-add chain of its
CSD weight and a hardware multiply, which is why
bit structure matters under its slow multiplier. The table holds what one compiler emits
for an isolated term at those flags: it over-prices work the compiler shares across a
combination, and it says nothing about other compilers, flags or tunings, about AVR, or
about cycle accuracy. The correction landed at the epoch-2 boundary (DECISIONS D45);
scores produced under the earlier charge keep the pin they were scored with. The tables
are in the cost model section below and the
counting rules on the architecture page[7][2].
Seven fixed initial-value problems drive every error measurement: three form the
search set the optimizer sees, and four are held out to catch overfitting. Peaks and
scales come from fixtures/problems.json[3].
| Problem | Set | Dynamics | States | t_end | Family | Scale | Reference |
|---|---|---|---|---|---|---|---|
| dahlquist | search | y' = -y, y(0) = 1 | 1 | 10 | linear | 2-2 | exp(-t) |
| damped_osc | search | x'' + 2ζωx' + ω²x = 0, ζ = 0.1, ω = 1 | 2 | 40 | oscillatory | 2-2 | closed form |
| vanderpol_mild | search | x'' = μ(1 - x²)x' - x, μ = 0.5 | 2 | 20 | nonlinear | 2-4 | mpmath, 30 digits |
| pendulum | held out | θ'' = -sin θ, θ(0) = 1 rad | 2 | 60 | nonlinear | 2-2 | energy invariant |
| dc_motor | held out | affine RL motor: R = 2, L = 0.5, Ke = Kt = 0.1, B = J = 0.02, V = 1 | 2 | 5 | linear | 2-3 | matrix exponential |
| rc_thermal | held out | y' = Ay, 3-compartment linear network | 3 | 4 | stiff | 2-2 | matrix exponential |
| quaternion | held out | q' = ½Ω(ω)q, body rates (0.3, 0.2, 0.5) | 4 | 30 | geometric | 2-2 | norm invariant |
rc_thermal is the stiff member, with a stiffness ratio of about 70. Its derivative would exceed Q15 range at the state scale, so it carries an extra factor (DERIV_SCALE = 1/8) that the step size undoes exactly. The held-out set cannot be reached from the search module's import graph, and a canary test checks that[3][4][9][17].
Error is the Euclidean distance between the final state, converted back to physical units, and the reference at t_end, divided by the problem's peak amplitude. pendulum uses relative energy drift |E - E0| / E0 instead, and quaternion the drift of its norm from 1[3].
Search error and held-out error are each the root mean square of the per-problem errors
over their set, computed by evaluator.set_error[5]. An RMS sums squares,
so a problem's influence on the aggregate grows with the square of its error and the
problems do not carry equal weight: the problem a method does worst on dominates that
method's score, and which problem that is differs from method to method. The weighting is
deliberate. Every per-problem error is already divided by its own problem's scale, so the
squares are summed on one shared scale, and that magnitude weighting is what every score in
the archive was computed under. The equal-weighting answer is a different scale: divide each
problem by the median error of the classical anchors before the squares are summed. The
counterfactual under that scale, and under a reference-norm scale that concentrates weight
instead of spreading it, is published on the
overview results page. The two scales disagree about the
outcome, and that disagreement is itself the result.
The Q15 integrator takes n equal steps of h = t_end / n, and an overflow anywhere aborts the run. The evaluator never raises: a failure becomes an inf entry, a zero overflow margin or a None measured order in the ScoreVector it returns, and its m0plus_fast results are primary[4][5].
A verifier then applies nine checks in a fixed order and returns the earliest failure as a coded rejection. Five symbolic checks run before any simulation: NOT_EXPLICIT, ROW_SUM_INCONSISTENT, DYADIC_IMPOSSIBLE, ORDER_NOT_MET (exact residuals over rooted trees) and COEFF_UNREPRESENTABLE. Then come Q15_OVERFLOW, UNSTABLE, NO_ASYMPTOTIC_WINDOW and NAN_OR_INF. The verifier never raises, never calls a language model, never opens a socket and never writes[6][8].
Symbolic order says what a tableau should do; measured order says what the implementation does. The float64 integrator runs dahlquist to t_end = 10 at n = 8 · 2k steps for k = 0 to 11. Slopes between consecutive halvings are kept where the error is finite and above 1e-12, the longest run of slopes whose spread stays within 0.08 is fitted by least squares, and the fitted slope is the measured order. With no run of at least two slopes the verifier rejects the tableau with NO_ASYMPTOTIC_WINDOW[5].
The golden tests reproduce the specification's values, rk4 at 4.0706 over 3 points among them. The study runs in float64 because Q15 roundoff would flatten every method's slope; the problem runs measure the Q15 error separately[5][17].
Code assigns each verified candidate an evidence tier when it is archived. heldout_verified: it improved on the incumbent elite in both search-set and held-out error, across at least two problem families. search_only: the search error improved and the held-out error did not, which is the signature of overfitting. no_incumbent: the cell was empty. no_improvement: there was an incumbent and neither rule above applied, which includes improving held-out error but not search error. Those last two were one word, unreplicated, before the split, and no archived record carries it now. The tier words appear in no prompt template, and a planted tableau tuned to the search set must land in search_only; canaries check both. The archive behind the cells is on the architecture page[11][13].
The two early phases enumerate their spaces completely, so their entries are marked "exhaustive" with the note "optimal within the enumerated space", CMA-ES entries are marked "search result", and tableaus seeded at startup "seeded classical baseline". The lattices are on the architecture page[10][20].
The model may record predictions only as predicates in a closed grammar over archive statistics, and code writes every verdict. A predicate that names an empty cell is inconclusive, because absence of evidence is not refutation. No verdict is issued below the declared minimum sample count, and an effect size below Cohen's d = 0.2 forces inconclusive whatever the comparison says; otherwise the verdict is supported or refuted. Every model call receives the refuted list. The grammar is on the architecture page[12][21].
Before the long run started, its premise was tested on rk4 and heun2 in Q15 on damped_osc, against criteria fixed in advance: coefficient arithmetic must be a material share of per-step cost, and Q15 roundoff must dominate the error somewhere in the practical step range. Code computes the verdict (kill, mixed or proceed), and the validation tab publishes it with its thresholds, whichever way it falls[19].
A practical validation suite sits outside the search: equations from real applications that the optimizer never sees and that no score, tier or archive statistic includes. The archived champions and the classical anchors run each one at the same 65,536-cycle budget, to see whether methods selected on the fixed problems hold up on dynamics they were not selected on[5][21].
Two numeric filters run before any win or loss from that suite is published. A problem is flagged degenerate when its finishers' Q15 errors span less than 5 percent from best to worst, because a field that tight is reporting the problem rather than the method; a flagged problem is left out of every tally with its reason printed. The second criterion, an error within 5 percent of the reference solution's norm, catches a state that decayed to nothing. validation/results.json stores that norm for each problem, on the same scale as the errors, so the pages read it from the document rather than assuming it. A third criterion catches the case that defeats both: when any two finishers agree to within 5 percent of each other while both sit within 5 percent of that norm, the field is not separating methods even though one live method widens the spread. Any error ratio within 2 percent of 1.0 is counted as a tie. A further criterion, an identical peak magnitude equal to the initial condition, was dropped because it flags healthy problems[5][20].
The design assumes the language model will try shortcuts. Only the runner talks to it, every path from its output into the system passes a validator, and a directive can narrow the search but never change the objective, the problems, the cost model or the tier rules[15].
The container mounts rk-harness read-only and exits at start-up unless a sha256 over the fourteen scoring files matches VERIFIER_HASH and the golden tests pass, and every record stores the hash it was scored under. Model-written code runs only inside the problem quarantine, and the container never receives the GitHub token. The architecture page goes through the checks in order[13][14][16].
The golden fixtures are ground truth computed before the implementation existed (the classical tableaus, rooted-tree counts, the rk4-against-rk38 anchor, the Q15 multiplication vectors and a hand-counted ARM sequence), kept verbatim under fixtures/ so the tests check against an outside truth instead of ratifying the specification. A preflight script runs every machine-checkable item of the review checklist, and the architecture page describes the suite[17][18].
The Q15 core is exact integer arithmetic, the evaluator reads no clock, starts no threads and draws no random numbers, and the search is seeded, so two runs from one seed write byte-identical archives. This site is rebuilt with no clock reads, so the same archive always gives the same pages; the architecture page has the detail[5][9][11][20].
The cost model is analytic and has never been checked against silicon inside the loop: an assembly fixture and a reference C emitter anchor it, but no hardware measurement feeds a score. The per-coefficient prices under the two M0+ models come from compiled probes at fixed flags rather than from silicon, and they describe an isolated term: work the compiler shares across a combination is over-priced, and nothing is established for other compilers, flags or tunings, for AVR, or for cycle accuracy. Derivative evaluation is left out of the cycle count, which is harmless between methods with the same stage count and a real simplification otherwise[7].
Measured order comes from one scalar problem in float64, so it certifies the coefficients, not fixed-point behavior across the suite. Stability extents are sampled along the real and imaginary axes only. Error is measured at the final time, so a method could trade path accuracy for endpoint accuracy without penalty. Floor rounding is one hardware convention; a target that rounds or saturates would shift every Q15 result[5][1].
The search space is narrow by design, and optimality from the enumeration phases holds only inside the enumerated lattices. Four held-out problems make held-out verification a strong filter, not a statistical guarantee, and the two-family rule and the d = 0.2 threshold are conventions chosen in advance, frozen by the specification so results stay comparable across the run[21].
Cycles per integration step, computed from instruction counts: the stage and b combinations a tableau's sparsity pattern implies, with each coefficient application under the two M0+ models priced from the coefficient price table, which records the instructions arm-none-eabi-gcc 13.2.1 emits for that term at -mcpu=cortex-m0plus -mthumb -O2 -ffreestanding -nostdlib. The cycle budget and the cycle buckets on the other tabs are built from these numbers.
| method | cost model | cycles per step |
|---|---|---|
| rk4 | m0plus_fast | 53 |
| rk4 | m0plus_slow | 53 |
| rk38 | m0plus_fast | 49 |
| rk38 | m0plus_slow | 49 |
| tableau | n_states | m0plus_fast | m0plus_slow |
|---|---|---|---|
| rk4 | 1 | 53 | 53 |
| rk4 | 2 | 106 | 106 |
| rk4 | 4 | 212 | 212 |
| rk38 | 1 | 49 | 49 |
| rk38 | 2 | 98 | 98 |
| rk38 | 4 | 196 | 196 |
| tableau | stages | m0plus_fast | m0plus_slow | avr_approx |
|---|---|---|---|---|
| euler | 1 | 5 | 5 | 6 |
| heun2 | 2 | 13 | 13 | 30 |
| heun3 | 3 | 34 | 34 | 90 |
| kutta3 | 3 | 41 | 41 | 106 |
| midpoint | 2 | 11 | 11 | 20 |
| ralston2 | 2 | 22 | 22 | 62 |
| rk38 | 4 | 49 | 49 | 132 |
| rk4 | 4 | 53 | 53 | 134 |
avr_approx is an approximate model, shown for context only. It never drives the search or the archive grids. A tableau of simple dyadic values can cost much less than its stage count suggests.
| model | mul | add | shift | load | store |
|---|---|---|---|---|---|
| m0plus_fast | 1 | 1 | 1 | 2 | 2 |
| m0plus_slow | 32 | 1 | 1 | 2 | 2 |
| avr_approx | 14 | 2 | 8 | 2 | 2 |
m0plus_fast and m0plus_slow differ only in the multiplier: both read the same op counts from one entry of the coefficient price table, so the gap between them is 31 cycles per coefficient multiply per state. The per-coefficient op counts are the instructions arm-none-eabi-gcc 13.2.1 emits for that term at -mcpu=cortex-m0plus -mthumb -O2 -ffreestanding -nostdlib, counted by op class; the five numbers above turn those counts into cycles, and no number here is a hardware measurement. A coefficient whose multiplier is plus or minus 1 at shift 0 costs nothing, because the compiler emits a bare adds or subs that the combination add already prices. The off-archive lanes add their own stated terms, named on the implicit and adaptive tabs.
Every measurement the run takes outside the scored archive is a line in rk-work/sidetrack/ledger.jsonl, stamped with the code hash that produced it and the artifact it was read from. The ledger holds 103 points across 10 jobs. The readings are on the implicit and adaptive tabs. The newest measurement ran under code hash 2b9f0b0f9005.
Every artifact is a pure function of the code and the point's parameters, so measuring a point again reproduces it byte for byte. A point counts as measured only under the code hash that measured it, a digest over the executor and the prototype modules, so editing a prototype re-opens its points instead of leaving stale numbers beside fresh ones.
A point that raises is recorded with its message and retried on later firings; after three failures under one code hash it is set aside and listed here. A side-track failure never costs a cycle, because the executor catches it and the run continues.
Rounding error in time integration is an active line of work, and this project did not discover it. Croci and Giles analyse Runge-Kutta discretizations of the heat equation in low-precision floating point and show that under round-to-nearest the computed solution stagnates once the timestep is small enough, with global rounding error growing like unit roundoff divided by the timestep until it does[22]. Hopkins and colleagues measure stochastic rounding and reduced-precision fixed point in ODE solvers built for neuromorphic hardware, the same arithmetic regime as this work on different hardware[23]. Croci and Rosilho de Souza build mixed-precision explicit stabilized Runge-Kutta methods and show where the low-precision part can sit without costing accuracy[24].
The crossover this run measures, where Q15 error turns back up as the step size falls, is the fixed-point counterpart of that stagnation result in a different arithmetic. Finding the same shape under a different rounding rule is a check on this harness rather than a new phenomenon, and it is consistent with what that literature shows.
Two things here are different from that line of work, and both are narrow. The arithmetic is fixed point with directed rounding: every Q15 multiply ends in an arithmetic right shift that floors, so the error is biased one way, where round-to-nearest is symmetric and stochastic rounding is unbiased by construction. And the object of study is the tableau rather than the arithmetic: this run searches coefficient space against that bias and prices every coefficient on a cost model, where the work above analyses fixed methods under a rounding mode. Neither difference is a claim of priority over that work.
Terms used across this site, in alphabetical order. Each definition says what the code does.