Two searches for better Runge-Kutta coefficients

Both projects asked whether a search can find Runge-Kutta coefficients that do better than the textbook ones. In float64, established methods were more accurate in both, though the 2025 half of that is weak evidence. Inside Q15 fixed point with floor rounding, at a 65,536-cycle budget on the analytic cost model with magnitude weighting, the rk run found a three-stage method with 2.95x lower held-out error than the best of eight classical methods. That figure comes from the epoch-1 cost model, which a later trace found had rk4 and rk38 in the wrong order; epoch 2 has not re-measured it, and it moves with the cost basis and the weighting.

Figures date from 2026-09-23. The rk run ended on 2026-11-10, after cycle 17,364, and no more compute goes to it; its findings site holds the final numbers, and rk-harness, the code behind them, stays public for anyone to download and run.

Terms

Q15
A sixteen-bit fixed-point fraction between minus one and one, for chips without a floating-point unit.
Floor rounding
Each Q15 multiply rounds toward minus infinity, so every product errs the same way.
Cycle budget
65,536 modeled clock cycles per method per problem, so a cheaper step buys more steps.
Analytic and traced cost
The analytic model prices coefficient arithmetic only; the traced count prices the compiled step, minus the derivative routine's body.
Held-out problems
Four problems that elites are picked on and the optimizer never reads: a selection set, not a test set.
Cell and elite
The archive sorts methods into cells by order, stage count and cost band; a cell's elite is its method with the lowest held-out error.
Weighting
How the four held-out errors become one number. Magnitude weighting, which the archive uses, is their RMS, so small errors count for little; median-anchor weighting counts the four equally.
Epoch
A stretch of the rk run under one set of pinned scoring files.

The two projects

The rk run

2026-08-29 onward

An autonomous search, run in a container, for coefficients that do well in Q15 on a modeled Cortex-M0+. A language model steers it through a JSON directive that can only narrow the search, and code does the scoring. Epoch 1 ran until 2026-09-10: 3,931 search cycles and 141,364 scored records. No person chose candidates or scores, though the runner started 24 times, including restarts to deploy changes to unpinned code. Epoch 2 started on 2026-09-17 under a corrected cost model, stopped late that day, and stayed down until 2026-09-23.

The 2025 ML project

Active 2025-09-13 to 2025-09-21

Novel-Numerical-Integration-Methods tried to generate new explicit Runge-Kutta tables with a neural generator, a surrogate model and evolutionary search, in float64. Its evaluation harness runs end to end for explicit tables. Its main conclusions from 2025 did not hold.

The same author built both, and audited the 2025 project in September 2026, after the rk run began. The code and data for both are in five repositories.

Results

The Q15 lead

The champion, 11e898cb, was picked on held-out error from 141,364 archived tableaus, so its lead over midpoint, the best classical method, carries selection bias. It has higher error than midpoint on pendulum, and much of the gap comes from rc_thermal, where every classical method stalls near a quantization floor.

On the traced whole-step basis the lead falls to 1.01x, which dropping one held-out problem overturns. With the four problems weighted equally it is 1.1x on the analytic basis, and dropping one can reverse that too (lowest 0.814x). Under round-to-nearest, at the same budget and cost model and with no extra cycles charged for the rounding, rk38, one of the four classical methods run under both rounding modes, reaches 0.0263 held-out error, below the champion's 0.0286 under floor rounding.

Supported with limits

Discovered elites lead in every cell they hold

In Q15 with floor rounding at the 65,536-cycle budget, on the analytic cost model with magnitude weighting and with the same selection bias, all 16 discovered methods that hold one of the 18 occupied archive cells have lower held-out error than every classical method, whatever its cost. The closest is 0.0836 against midpoint's 0.0844.

Supported with limits

Out-of-sample problems

On eight problems chosen after the search began, the best of three discovered methods had lower Q15 error than the best of five classical ones on 4 of 5 non-stiff problems, the champion by itself on 3. On the stiff robertson_scaled, every discovered method overflowed, as did rk4 and rk38.

Supported with limits

In float64, rk4 is far more accurate

At the step count the 65,536-cycle budget gives each method, the champion's float64 error was 1,992x to 420,975x that of rk4 on the seven validation problems where both finish, second order against fourth. Dormand-Prince never ran against the champion in the same arithmetic.

Supported with limits

A trace caught the cost model's error

The compiled step matched the Python evaluator in 144 of 144 cases, 25 of them overflow cases, on an emulator that is instruction-accurate but not cycle-accurate. Once the trace found rk4 and rk38 in the wrong order, the run froze epoch 1 instead of rescoring it.

Supported

The 2025 benchmark ran

In float64, over 12 entries and 10,000 ODEs (8,113 scorable), Dormand-Prince had the lowest mean error (0.0791) and RK4 the next (0.14). The other entries were copies of those two, random tables or truncated methods, and the reference solver was itself Dormand-Prince.

Supported with limits

The saved tables never left the seeds

Evolutionary search independently rediscovered RK4 and Dormand-Prince.

The genetic operators ran, but RK4 and Dormand-Prince were seeded in and every score tied at the clip of 1, so the saved table never changed.

Not supported

The generator was never trained

Gradient descent collapsed into local minima.

The generator's optimizer was never stepped. Gradient descent trained only a surrogate model, and nothing in training reads its predictions.

Not supported

How each project scored candidates

AspectThe 2025 ML projectThe rk run
What a candidate is scored onA composite clipped at 1. All 10,000 rows of the one training log (100 epochs of 100 candidates) score 1.Search-set error for the optimizer and held-out error for choosing elites, both in Q15 at the cycle budget.
Order conditionsNone enforced. Its order-4 test labels RK4 as order 3.Exact rational order conditions, checked before a candidate is scored.
Where the search startsRK4 and Dormand-Prince in the starting populations, and a random generator reseeded to 42 for empty slots.An exhaustive lattice of dyadic coefficients, then CMA-ES over the stage matrix.
Guarding the scorerNo automated test suite.A read-only harness, a sha256 over the pinned files (10 in epoch 1, 14 since epoch 2) checked at every start and stored in every record, and 91 gate cases that must pass before any search cycle.
Problems outside the searchA final benchmark on 10,000 generated ODEs.Four held-out problems, then eight out-of-sample ones.
What gets publishedEvery benchmark entry, the weakest included.Negative results too: a mixed premise test and the float64 gap.

Neither project isolated any of these differences, so none of them can be tied to a particular result.

What neither project tested

Neither project kept its comparisons hidden from the search. The language model that steers the rk run sees held-out errors, and elites are picked on them, so the nearest independent check is the out-of-sample suite. The 2025 search started from the tables it was said to rediscover.

Neither search measured whether large local minima exist. The 2025 project's results stayed flat because of its clipped score, untrained generator and reseeded random fallback. The rk run's best held-out error stopped improving after search cycle 33 while other cells kept improving, which describes one search, not the space it searched.