What the explicit search found, at snapshot 2026-09-10.
At a fixed budget of 65,536 cycles on a Cortex-M0+ cost model, under magnitude weighting on the analytic cost basis, which prices the stage and b combinations only, the search found Runge-Kutta methods with up to 2.9508× lower held-out error than the best classical method. On that weighting and basis, dropping any one of the four held-out problems leaves a lead of 1.9236× to 3.1739×, so there the lead does not rest on one of them. Of the five other cells in the grid in finding 1, only the reference-norm scale on the analytic basis keeps both claims; in the other four a classical anchor leads on at least one reduced held-out set, and under median-anchor equal weighting on the traced whole-step basis it leads on the full set too. The data also shows why: Q15's floor rounding adds a bias of about half an LSB to every multiply, and that alone reorders the classical methods before any search starts. The demo lets you watch it happen.
Unless a figure says otherwise: a 65,536-cycle budget per problem, the m0plus_fast cost model, Q15 with floor rounding, the four held-out problems. Charts come from tools/key_findings.json; finding 6 reads rk-work/validation/results.json. Findings 1 to 5 were computed at 141,364 archive records, finding 6 and the method matrix at 141,364, and each figure carries the state it was computed at.
Every chart on this page is a static drawing, and the table folded under it, which holds the same numbers, is the keyboard and tap path to them.
rk4 and rk38 are both four-stage, order-4 methods with the same stability polynomial, and textbooks treat them as interchangeable. The analytic cost model does not. With a 1-cycle multiplier it prices rk4 cheaper (33 against 36 cycles per step), and with a 32-cycle multiplier it prices rk38 cheaper (64 against 85), because rk4's coefficients need more shift-add work there. That swap belongs to the frozen epoch-1 rule, which charges each coefficient the cheaper of a shift-add chain and a hardware multiply, and the compiled, traced step is what checks it. At the scope the model prices, the stage and b combinations and nothing else, the trace has rk38 at 59 cycles and rk4 at 67 under m0plus_fast, and 59 and 67 under m0plus_slow, so there rk38 is the cheaper of the two under both multiplier models and the fast half of the swap does not reproduce. Those counts come from an emulator that is instruction accurate rather than cycle accurate, and the findings validation page has every traced row.
| method | cost model | cycles per step |
|---|---|---|
| rk4 | m0plus_fast | 33 |
| rk4 | m0plus_slow | 85 |
| rk38 | m0plus_fast | 36 |
| rk38 | m0plus_slow | 64 |
The archive keeps one elite per MAP-Elites cell (order, stage count, cycle-cost bucket). 18 cells are occupied: 16 by discovered methods and 2 by classical ones (euler and rk38). Each discovered cell is compared with every classical method of equal or lower per-step cost, since a cheaper method could always be swapped in. All 16 discovered cells have the lower held-out error. The median error ratio, discovered over classical, is 0.71, and the best is 0.34.
| method | kind | order | stages | cycles/step | held-out error | tier |
|---|---|---|---|---|---|---|
| euler | classical | 1 | 1 | 5 | 0.32 | no_incumbent |
| midpoint | classical | n/a | 2 | 11 | 0.0844 | no_incumbent |
| heun2 | classical | n/a | 2 | 13 | 0.0961 | no_incumbent |
| discovered 2d2c2b75d719 | discovered | 2 | 2 | 15 | 0.0613 | no_improvement |
| ralston2 | classical | n/a | 2 | 16 | 0.0919 | no_incumbent |
| discovered 4feb237eaaa3 | discovered | 2 | 2 | 17 | 0.0494 | no_improvement |
| discovered 11e898cbab59 | discovered | 2 | 3 | 22 | 0.0286 | no_improvement |
| heun3 | classical | n/a | 3 | 23 | 0.0852 | no_incumbent |
| kutta3 | classical | n/a | 3 | 26 | 0.0873 | no_incumbent |
| discovered 4f501ec6a5ce | discovered | 3 | 3 | 29 | 0.0477 | no_improvement |
| discovered 5fef0bd2b5fe | discovered | 2 | 4 | 30 | 0.0836 | no_incumbent |
| rk4 | classical | n/a | 4 | 33 | 0.0919 | no_incumbent |
| rk38 | classical | 4 | 4 | 36 | 0.0853 | no_incumbent |
| discovered 2bceb12dc1c2 | discovered | 2 | 4 | 39 | 0.0587 | heldout_verified |
| discovered 19b66f506eec | discovered | 3 | 4 | 46 | 0.0651 | heldout_verified |
| discovered 6d0004de713f | discovered | 2 | 5 | 57 | 0.0533 | heldout_verified |
| discovered 274563728359 | discovered | 3 | 5 | 59 | 0.0587 | no_improvement |
| discovered 196b1d17d0b7 | discovered | 4 | 6 | 63 | 0.0603 | no_incumbent |
| discovered c10bc5417b5b | discovered | 3 | 6 | 63 | 0.0822 | no_incumbent |
| discovered 407c65d76723 | discovered | 3 | 5 | 65 | 0.0604 | no_improvement |
| discovered 935100ec0e46 | discovered | 2 | 5 | 65 | 0.0639 | no_improvement |
| discovered aea95967c0fd | discovered | 2 | 6 | 72 | 0.0453 | heldout_verified |
| discovered 42863b93bf7d | discovered | 3 | 6 | 75 | 0.0377 | heldout_verified |
| discovered 013ba3020051 | discovered | 4 | 6 | 77 | 0.0808 | no_incumbent |
Held-out error here is the published figure, magnitude weighting on the analytic basis. The tier is read from the archive at build time: for a method that holds a cell it is the stored tier of that cell's elite, for a classical method that holds no cell it is the stored tier of its one seed record, and a method with no archive record prints n/a.
| order | stages | cycle bucket | held by | held-out error | counted |
|---|---|---|---|---|---|
| 1 | 1 | b0 | seeded euler | 0.32 | outside the searched stage range |
| 2 | 2 | b0 | discovered 2d2c2b75d719 | 0.0613 | counted in the fraction |
| 2 | 2 | b1 | discovered 4feb237eaaa3 | 0.0494 | counted in the fraction |
| 2 | 3 | b1 | discovered 11e898cbab59 | 0.0286 | counted in the fraction |
| 2 | 4 | b1 | discovered 5fef0bd2b5fe | 0.0836 | counted in the fraction |
| 2 | 4 | b2 | discovered 2bceb12dc1c2 | 0.0587 | counted in the fraction |
| 2 | 5 | b2 | discovered 6d0004de713f | 0.0533 | counted in the fraction |
| 2 | 5 | b3 | discovered 935100ec0e46 | 0.0639 | counted in the fraction |
| 2 | 6 | b3 | discovered aea95967c0fd | 0.0453 | counted in the fraction |
| 3 | 3 | b1 | discovered 4f501ec6a5ce | 0.0477 | counted in the fraction |
| 3 | 4 | b2 | discovered 19b66f506eec | 0.0651 | counted in the fraction |
| 3 | 5 | b2 | discovered 274563728359 | 0.0587 | counted in the fraction |
| 3 | 5 | b3 | discovered 407c65d76723 | 0.0604 | counted in the fraction |
| 3 | 6 | b2 | discovered c10bc5417b5b | 0.0822 | counted in the fraction |
| 3 | 6 | b3 | discovered 42863b93bf7d | 0.0377 | counted in the fraction |
| 4 | 4 | b2 | classical baseline rk38 | 0.0853 | counted in the fraction |
| 4 | 6 | b2 | discovered 196b1d17d0b7 | 0.0603 | counted in the fraction |
| 4 | 6 | b3 | discovered 013ba3020051 | 0.0808 | counted in the fraction |
The best discovered method is a 3-stage, order-2 tableau at 22 cycles per step for one state (m0plus_fast, n_states=1), with held-out error 0.0286 against 0.0844 for midpoint, the best classical method. Cost scales with the problem's state dimension, so the same tableau costs 44 cycles per step on a two-state problem. Every coefficient is dyadic, so the method needs only shifts and adds.
A = [[0, 0, 0], [13/16, 0, 0], [5/32, 11/32, 0]], b = (0, 0, 1), c = (0, 13/16, 1/2). Measured order 2.02. Held-out error per problem, each with its share of the held-out sum of squares: pendulum 0.00474 (0.69%), dc_motor 0.0276 (23.33%), rc_thermal (flagged) 0.0461 (64.84%), quaternion 0.0191 (11.14%). rc_thermal is marked flagged because the degeneracy detector flags it over the 9 methods finding 1's counterfactual scores on the analytic basis (finding 2 gives every verdict and its reason).
Held-out error is the root mean square of the four per-problem errors, so a problem's influence grows with the square of its error and the four do not carry a quarter each. For the champion, rc_thermal carries 64.8% of the held-out sum of squares and pendulum 0.69%; for midpoint it is rc_thermal at 85.1%. Which problem dominates is a property of the method rather than of the metric: euler's largest share is pendulum at 86.1%. rc_thermal is the problem where every classical method lands on the same quantization floor (finding 4). Under magnitude weighting on the analytic basis (the stage and b combinations only), dropping rc_thermal leaves 1.9236×, the lowest ratio in the table below; the highest is 3.1739×, with dc_motor dropped, and the best classical method changes from midpoint to rk38 when quaternion goes.
| held-out problem dropped | champion | best classical method | ratio |
|---|---|---|---|
| none, all four kept | 0.0286 | midpoint 0.0844 | 2.9508× |
| pendulum | 0.0329 | midpoint 0.0975 | 2.9610× |
| dc_motor | 0.0289 | midpoint 0.0918 | 3.1739× |
| rc_thermal | 0.0196 | midpoint 0.0377 | 1.9236× |
| quaternion | 0.0311 | rk38 0.095 | 3.0499× |
Held-out error is an RMS, so each row is recomputed over the three problems it keeps, under magnitude weighting on the analytic basis, from tools/key_findings.json, efficiency.numbers.leave_one_out.
Two caveats. Elites are chosen by held-out error from 141,364 archived candidates, so the best values carry selection bias. The outer loop is shown one held-out error per occupied cell; only the inner optimizer is scored on search-set error alone. And a tier is a mechanical comparison with the cell's incumbent when the record is written, not a validation grade. The best cell's elite is stored as no_improvement, its cell had an incumbent and neither the heldout_verified nor the search_only rule applied. Of the 16 discovered elites, 5 are heldout_verified.
The ratio above is one cell of a grid. It is measured under magnitude weighting, the aggregate the archive selects on (an unweighted RMS of the four per-problem errors), and on the analytic cost basis, the per-step cost the analytic cost model charges for the stage and b combinations. The grid adds two other weightings and one other cost basis, and all six combinations are below. Every archived per-problem error is already normalized before any weighting: the distance problems divide by their own peak amplitude, and pendulum and quaternion measure a relative drift. The median-anchor scale then divides each error by the median error of the eight classical anchors on that problem, and it is the equal-weighting answer, named that way wherever it appears. The reference-norm scale divides instead by the error a method would report if its state collapsed to zero, which is the size of the reference solution on that same normalized scale. It is published beside the median-anchor scale and never alone, because it does not equalize the weights: rc_thermal's reference norm is the smallest of the four held-out problems', so that scale lifts rc_thermal's share of the champion's sum of squares from 64.8% to 98.7% and midpoint's from 85.1% to 99.6%. The two scales disagree about the headline, which is why neither is printed without the other.
The second basis prices a step at what the compiled code executes, read from the traced document rather than from the model, and it changes the step count rather than the arithmetic. On damped_osc, a two-state problem, euler takes 1,598 steps inside the budget where the model gave it 6,553, and the champion 455 where the model gave it 1,489. Every method loses steps on every problem, and no two of the 9 lose the same fraction on damped_osc, so the field is reshuffled rather than shifted.
| weighting | cost basis | champion | best classical anchor | ratio, anchor over champion | lowest leave-one-out ratio | reduced sets where an anchor leads | ahead on the full set |
|---|---|---|---|---|---|---|---|
| magnitude (published) | analytic (published) | 0.0286 | midpoint 0.0844 | 2.9508x | 1.9236x (drop rc_thermal) | 0 of 4 | champion |
| magnitude (published) | traced whole step | 0.0151 | heun3 0.0152 | 1.0112x | 0.6143x (drop rc_thermal) | 3 of 4 | champion |
| equal, median anchor | analytic (published) | 0.653 | midpoint 0.719 | 1.1020x | 0.8144x (drop rc_thermal) | 3 of 4 | champion |
| equal, median anchor | traced whole step | 3.16 | midpoint 0.156 | 0.0494x | 0.0314x (drop dc_motor) | 3 of 4 | anchor |
| reference norm | analytic (published) | 0.146 | midpoint 0.49 | 3.3646x | 1.9200x (drop rc_thermal) | 0 of 4 | champion |
| reference norm | traced whole step | 0.0424 | heun3 0.078 | 1.8386x | 0.6042x (drop rc_thermal) | 1 of 4 | champion |
Each row is one cell of the grid, and a ratio above 1.0 means the champion has the lower error. The analytic basis prices the stage and b combinations only; the traced whole-step basis buys steps at the count for the whole rk_step with the derivative body excluded. Read from tools/key_findings.json, counterfactual.numbers.heldout.
Both published figures are measured under magnitude weighting on the analytic basis (the stage and b combinations only): the 2.9508x lead, and the 1.9236x floor with rc_thermal dropped, which is the claim that the champion stays ahead on every reduced set. Taken one at a time, one of the five other cells keeps both and four do not:
Two cells change the weighting and the cost basis together, moving to the traced whole-step basis (the whole rk_step, derivative body excluded). Under median-anchor equal weighting there the ratio reads 0.0494x. pendulum has the smallest median anchor error of the four held-out problems at 0.00412, so dividing by it amplifies any method whose pendulum error rises when steps are taken away. The champion's pendulum error goes from 0.00474 at 1,489 steps on the analytic basis to 0.026 at 455 steps on the traced one, which is most of why the median anchor ratio falls below one there. Read that cell as the metric magnifying one problem, not as the champion losing by that factor. Under the reference-norm scale on the same traced whole-step basis it reads 1.8386x, a classical anchor leads on 1 of the 4 reduced sets, and the floor is 0.6042x once rc_thermal is dropped.
| weighting | cost basis | dropped | champion | best classical anchor | its error | ratio | runner-up anchor |
|---|---|---|---|---|---|---|---|
| magnitude (published) | analytic (published) | none, all four kept | 0.0286 | midpoint | 0.0844 | 2.9508x | heun3 |
| magnitude (published) | analytic (published) | dc_motor | 0.0289 | midpoint | 0.0918 | 3.1739x | heun3 |
| magnitude (published) | analytic (published) | pendulum | 0.0329 | midpoint | 0.0975 | 2.9610x | heun3 |
| magnitude (published) | analytic (published) | quaternion | 0.0311 | rk38 | 0.095 | 3.0499x | kutta3 |
| magnitude (published) | analytic (published) | rc_thermal | 0.0196 | midpoint | 0.0377 | 1.9236x | heun3 |
| magnitude (published) | traced whole step | none, all four kept | 0.0151 | heun3 | 0.0152 | 1.0112x | midpoint |
| magnitude (published) | traced whole step | dc_motor | 0.0169 | heun3 | 0.0156 | 0.9229x | midpoint |
| magnitude (published) | traced whole step | pendulum | 0.00875 | heun3 | 0.0169 | 1.9323x | midpoint |
| magnitude (published) | traced whole step | quaternion | 0.0173 | heun3 | 0.0171 | 0.9912x | midpoint |
| magnitude (published) | traced whole step | rc_thermal | 0.0157 | midpoint | 0.00967 | 0.6143x | rk38 |
| equal, median anchor | analytic (published) | none, all four kept | 0.653 | midpoint | 0.719 | 1.1020x | rk38 |
| equal, median anchor | analytic (published) | dc_motor | 0.715 | midpoint | 0.669 | 0.9362x | rk38 |
| equal, median anchor | analytic (published) | pendulum | 0.356 | midpoint | 0.831 | 2.3307x | heun3 |
| equal, median anchor | analytic (published) | quaternion | 0.727 | kutta3 | 0.722 | 0.9933x | rk38 |
| equal, median anchor | analytic (published) | rc_thermal | 0.734 | midpoint | 0.598 | 0.8144x | rk38 |
| equal, median anchor | traced whole step | none, all four kept | 3.16 | midpoint | 0.156 | 0.0494x | rk38 |
| equal, median anchor | traced whole step | dc_motor | 3.64 | midpoint | 0.114 | 0.0314x | rk38 |
| equal, median anchor | traced whole step | pendulum | 0.0875 | heun3 | 0.17 | 1.9392x | midpoint |
| equal, median anchor | traced whole step | quaternion | 3.64 | midpoint | 0.172 | 0.0473x | rk38 |
| equal, median anchor | traced whole step | rc_thermal | 3.64 | midpoint | 0.148 | 0.0407x | rk38 |
| reference norm | analytic (published) | none, all four kept | 0.146 | midpoint | 0.49 | 3.3646x | heun3 |
| reference norm | analytic (published) | dc_motor | 0.168 | midpoint | 0.565 | 3.3738x | heun3 |
| reference norm | analytic (published) | pendulum | 0.168 | midpoint | 0.566 | 3.3650x | heun3 |
| reference norm | analytic (published) | quaternion | 0.168 | midpoint | 0.566 | 3.3700x | heun3 |
| reference norm | analytic (published) | rc_thermal | 0.0193 | midpoint | 0.0371 | 1.9200x | heun3 |
| reference norm | traced whole step | none, all four kept | 0.0424 | heun3 | 0.078 | 1.8386x | midpoint |
| reference norm | traced whole step | dc_motor | 0.0488 | heun3 | 0.0897 | 1.8380x | midpoint |
| reference norm | traced whole step | pendulum | 0.0466 | heun3 | 0.0899 | 1.9290x | midpoint |
| reference norm | traced whole step | quaternion | 0.0489 | heun3 | 0.09 | 1.8384x | midpoint |
| reference norm | traced whole step | rc_thermal | 0.0157 | midpoint | 0.0095 | 0.6042x | rk38 |
| method | kind | weighting | held-out RMS, analytic (published) | held-out RMS, traced whole step |
|---|---|---|---|---|
| 11e898cb | discovered | magnitude (published) | 0.0286 | 0.0151 |
| midpoint | classical | magnitude (published) | 0.0844 | 0.0161 |
| heun3 | classical | magnitude (published) | 0.0852 | 0.0152 |
| rk38 | classical | magnitude (published) | 0.0853 | 0.0179 |
| kutta3 | classical | magnitude (published) | 0.0873 | 0.0216 |
| ralston2 | classical | magnitude (published) | 0.0919 | 0.0223 |
| rk4 | classical | magnitude (published) | 0.0919 | 0.0204 |
| heun2 | classical | magnitude (published) | 0.0961 | 0.0261 |
| euler | classical | magnitude (published) | 0.32 | 1.5 |
Each value is the root mean square of one method's four held-out errors under magnitude weighting. The analytic column prices the stage and b combinations only. The traced column buys steps at the count for the whole rk_step with the derivative body excluded, a budget denominator taken from an emulator that is instruction accurate and not cycle accurate. The two columns are errors at two step counts and are never divided by each other. Read from counterfactual.series.per_method.
| cost basis | set | method | problem | states | cycles/step | steps | error | share, magnitude | share, median anchor | share, reference norm |
|---|---|---|---|---|---|---|---|---|---|---|
| analytic (published) | heldout | 11e898cb | dc_motor | 2 | 44 | 1,489 | 0.0276 | 23.33% | 10.12% | 0.86% |
| analytic (published) | heldout | 11e898cb | pendulum | 2 | 44 | 1,489 | 0.00474 | 0.69% | 77.65% | 0.026% |
| analytic (published) | heldout | 11e898cb | quaternion | 4 | 88 | 744 | 0.0191 | 11.14% | 7.11% | 0.43% |
| analytic (published) | heldout | 11e898cb | rc_thermal (flagged) | 3 | 66 | 992 | 0.0461 | 64.84% | 5.12% | 98.68% |
| analytic (published) | heldout | euler | dc_motor | 2 | 10 | 6,553 | 0.171 | 7.13% | 0.032% | 2.09% |
| analytic (published) | heldout | euler | pendulum | 2 | 10 | 6,553 | 0.594 | 86.08% | 99.96% | 26.27% |
| analytic (published) | heldout | euler | quaternion | 4 | 20 | 3,276 | 0.0596 | 0.87% | 0.0057% | 0.26% |
| analytic (published) | heldout | euler | rc_thermal (flagged) | 3 | 15 | 4,369 | 0.156 | 5.92% | 0.0048% | 71.37% |
| analytic (published) | heldout | heun2 | dc_motor | 2 | 26 | 2,520 | 0.0974 | 25.69% | 40.98% | 0.94% |
| analytic (published) | heldout | heun2 | pendulum | 2 | 26 | 2,520 | 0.00421 | 0.048% | 19.99% | 0.0018% |
| analytic (published) | heldout | heun2 | quaternion | 4 | 52 | 1,260 | 0.0559 | 8.47% | 19.90% | 0.32% |
| analytic (published) | heldout | heun2 | rc_thermal (flagged) | 3 | 39 | 1,680 | 0.156 | 65.79% | 19.13% | 98.74% |
| analytic (published) | heldout | heun3 | dc_motor | 2 | 46 | 1,424 | 0.0543 | 10.17% | 8.58% | 0.29% |
| analytic (published) | heldout | heun3 | pendulum | 2 | 46 | 1,424 | 0.00971 | 0.33% | 71.49% | 0.0098% |
| analytic (published) | heldout | heun3 | quaternion | 4 | 92 | 712 | 0.0406 | 5.67% | 7.04% | 0.17% |
| analytic (published) | heldout | heun3 | rc_thermal (flagged) | 3 | 69 | 949 | 0.156 | 83.84% | 12.89% | 99.53% |
| analytic (published) | heldout | kutta3 | dc_motor | 2 | 52 | 1,260 | 0.0482 | 7.63% | 20.22% | 0.22% |
| analytic (published) | heldout | kutta3 | pendulum | 2 | 52 | 1,260 | 0.000383 | 0.00048% | 0.33% | 1.5e-05% |
| analytic (published) | heldout | kutta3 | quaternion | 4 | 104 | 630 | 0.0557 | 10.18% | 39.72% | 0.31% |
| analytic (published) | heldout | kutta3 | rc_thermal (flagged) | 3 | 78 | 840 | 0.158 | 82.20% | 39.72% | 99.46% |
| analytic (published) | heldout | midpoint | dc_motor | 2 | 22 | 2,978 | 0.0568 | 11.30% | 35.14% | 0.32% |
| analytic (published) | heldout | midpoint | pendulum | 2 | 22 | 2,978 | 6.45e-05 | 1.5e-05% | 0.012% | 4.3e-07% |
| analytic (published) | heldout | midpoint | quaternion | 4 | 44 | 1,489 | 0.0322 | 3.64% | 16.66% | 0.11% |
| analytic (published) | heldout | midpoint | rc_thermal (flagged) | 3 | 33 | 1,985 | 0.156 | 85.06% | 48.19% | 99.57% |
| analytic (published) | heldout | ralston2 | dc_motor | 2 | 32 | 2,048 | 0.0788 | 18.40% | 29.30% | 0.62% |
| analytic (published) | heldout | ralston2 | pendulum | 2 | 32 | 2,048 | 0.00473 | 0.066% | 27.54% | 0.0023% |
| analytic (published) | heldout | ralston2 | quaternion | 4 | 64 | 1,024 | 0.0566 | 9.50% | 22.27% | 0.33% |
| analytic (published) | heldout | ralston2 | rc_thermal (flagged) | 3 | 48 | 1,365 | 0.156 | 72.03% | 20.90% | 99.05% |
| analytic (published) | heldout | rk38 | dc_motor | 2 | 72 | 910 | 0.052 | 9.30% | 26.01% | 0.27% |
| analytic (published) | heldout | rk38 | pendulum | 2 | 72 | 910 | 0.0011 | 0.0042% | 3.04% | 0.00013% |
| analytic (published) | heldout | rk38 | quaternion | 4 | 144 | 455 | 0.0447 | 6.86% | 28.24% | 0.21% |
| analytic (published) | heldout | rk38 | rc_thermal (flagged) | 3 | 108 | 606 | 0.156 | 83.83% | 42.72% | 99.52% |
| analytic (published) | heldout | rk4 | dc_motor | 2 | 66 | 992 | 0.0764 | 17.26% | 30.81% | 0.56% |
| analytic (published) | heldout | rk4 | pendulum | 2 | 66 | 992 | 0.00403 | 0.048% | 22.39% | 0.0016% |
| analytic (published) | heldout | rk4 | quaternion | 4 | 132 | 496 | 0.0541 | 8.65% | 22.73% | 0.29% |
| analytic (published) | heldout | rk4 | rc_thermal (flagged) | 3 | 99 | 661 | 0.158 | 74.04% | 24.08% | 99.14% |
| analytic (published) | search | 11e898cb | dahlquist (flagged) | 1 | 22 | 2,978 | 0.000199 | 0.01% | 94.46% | 99.74% |
| analytic (published) | search | 11e898cb | damped_osc | 2 | 44 | 1,489 | 0.00394 | 3.93% | 2.51% | 0.26% |
| analytic (published) | search | 11e898cb | vanderpol_mild | 2 | 44 | 1,489 | 0.0195 | 96.06% | 3.03% | 0.0033% |
| analytic (published) | search | euler | dahlquist (flagged) | 1 | 5 | 13,107 | 4.54e-05 | 0.00041% | 12.36% | 60.64% |
| analytic (published) | search | euler | damped_osc | 2 | 10 | 6,553 | 0.0142 | 40.48% | 81.69% | 39.33% |
| analytic (published) | search | euler | vanderpol_mild | 2 | 10 | 6,553 | 0.0172 | 59.52% | 5.95% | 0.03% |
| analytic (published) | search | heun2 | dahlquist (flagged) | 1 | 13 | 5,041 | 4.54e-05 | 0.00023% | 29.15% | 90.89% |
| analytic (published) | search | heun2 | damped_osc | 2 | 26 | 2,520 | 0.00554 | 3.38% | 29.33% | 8.98% |
| analytic (published) | search | heun2 | vanderpol_mild | 2 | 26 | 2,520 | 0.0296 | 96.62% | 41.53% | 0.13% |
| analytic (published) | search | heun3 | dahlquist (flagged) | 1 | 23 | 2,849 | 0.000167 | 0.0035% | 83.81% | 99.00% |
| analytic (published) | search | heun3 | damped_osc | 2 | 46 | 1,424 | 0.00652 | 5.29% | 8.57% | 0.99% |
| analytic (published) | search | heun3 | vanderpol_mild | 2 | 46 | 1,424 | 0.0276 | 94.71% | 7.61% | 0.0093% |
| analytic (published) | search | kutta3 | dahlquist (flagged) | 1 | 26 | 2,520 | 0.000167 | 0.00031% | 46.96% | 99.22% |
| analytic (published) | search | kutta3 | damped_osc | 2 | 52 | 1,260 | 0.00535 | 0.32% | 3.24% | 0.67% |
| analytic (published) | search | kutta3 | vanderpol_mild | 2 | 52 | 1,260 | 0.0943 | 99.68% | 49.80% | 0.11% |
| analytic (published) | search | midpoint | dahlquist (flagged) | 1 | 11 | 5,957 | 7.67e-05 | 0.0011% | 56.10% | 95.18% |
| analytic (published) | search | midpoint | damped_osc | 2 | 22 | 2,978 | 0.00668 | 8.61% | 28.77% | 4.79% |
| analytic (published) | search | midpoint | vanderpol_mild | 2 | 22 | 2,978 | 0.0218 | 91.39% | 15.13% | 0.026% |
| analytic (published) | search | ralston2 | dahlquist (flagged) | 1 | 16 | 4,096 | 4.54e-05 | 0.00012% | 9.10% | 57.09% |
| analytic (published) | search | ralston2 | damped_osc | 2 | 32 | 2,048 | 0.0153 | 13.82% | 69.45% | 42.77% |
| analytic (published) | search | ralston2 | vanderpol_mild | 2 | 32 | 2,048 | 0.0381 | 86.18% | 21.45% | 0.14% |
| analytic (published) | search | rk38 | dahlquist (flagged) | 1 | 36 | 1,820 | 4.54e-05 | 0.00014% | 18.44% | 82.16% |
| analytic (published) | search | rk38 | damped_osc | 2 | 72 | 910 | 0.00817 | 4.63% | 40.35% | 17.65% |
| analytic (published) | search | rk38 | vanderpol_mild | 2 | 72 | 910 | 0.0371 | 95.37% | 41.21% | 0.19% |
| analytic (published) | search | rk4 | dahlquist (flagged) | 1 | 33 | 1,985 | 0.000167 | 0.00027% | 35.38% | 94.24% |
| analytic (published) | search | rk4 | damped_osc | 2 | 66 | 992 | 0.0159 | 2.43% | 21.60% | 5.65% |
| analytic (published) | search | rk4 | vanderpol_mild | 2 | 66 | 992 | 0.101 | 97.57% | 43.02% | 0.12% |
| traced whole step | heldout | 11e898cb | dc_motor | 2 | 144 | 455 | 0.00733 | 5.93% | 0.03% | 0.72% |
| traced whole step | heldout | 11e898cb | pendulum | 2 | 144 | 455 | 0.026 | 74.66% | 99.94% | 9.40% |
| traced whole step | heldout | 11e898cb | quaternion | 4 | 490 | 133 | 0.00353 | 1.38% | 0.01% | 0.17% |
| traced whole step | heldout | 11e898cb | rc_thermal | 3 | 297 | 220 | 0.0128 | 18.04% | 0.017% | 89.70% |
| traced whole step | heldout | euler | dc_motor | 2 | 41 | 1,598 | 0.0296 | 0.0097% | 3.7e-05% | 0.0091% |
| traced whole step | heldout | euler | pendulum | 2 | 41 | 1,598 | 3 | 99.85% | 100.00% | 97.10% |
| traced whole step | heldout | euler | quaternion | 4 | 129 | 508 | 0.0758 | 0.064% | 0.00036% | 0.062% |
| traced whole step | heldout | euler | rc_thermal | 3 | 52 | 1,260 | 0.0814 | 0.074% | 5.2e-05% | 2.83% |
| traced whole step | heldout | heun2 | dc_motor | 2 | 92 | 712 | 0.0264 | 25.61% | 6.41% | 0.89% |
| traced whole step | heldout | heun2 | pendulum | 2 | 92 | 712 | 0.0061 | 1.37% | 89.02% | 0.049% |
| traced whole step | heldout | heun2 | quaternion | 4 | 305 | 214 | 0.0102 | 3.84% | 1.42% | 0.14% |
| traced whole step | heldout | heun2 | rc_thermal | 3 | 160 | 409 | 0.0434 | 69.18% | 3.15% | 98.92% |
| traced whole step | heldout | heun3 | dc_motor | 2 | 164 | 399 | 0.0142 | 21.65% | 1.09% | 0.79% |
| traced whole step | heldout | heun3 | pendulum | 2 | 164 | 399 | 0.00833 | 7.48% | 97.92% | 0.28% |
| traced whole step | heldout | heun3 | quaternion | 4 | 543 | 120 | 0.00697 | 5.25% | 0.39% | 0.20% |
| traced whole step | heldout | heun3 | rc_thermal | 3 | 363 | 180 | 0.0247 | 65.63% | 0.60% | 98.72% |
| traced whole step | heldout | kutta3 | dc_motor | 2 | 187 | 350 | 0.0169 | 15.23% | 0.19% | 0.77% |
| traced whole step | heldout | kutta3 | pendulum | 2 | 187 | 350 | 0.0241 | 31.21% | 99.58% | 1.65% |
| traced whole step | heldout | kutta3 | quaternion | 4 | 582 | 112 | 0.0114 | 6.99% | 0.13% | 0.37% |
| traced whole step | heldout | kutta3 | rc_thermal | 3 | 419 | 156 | 0.0295 | 46.56% | 0.10% | 97.21% |
| traced whole step | heldout | midpoint | dc_motor | 2 | 74 | 885 | 0.016 | 24.74% | 59.63% | 0.82% |
| traced whole step | heldout | midpoint | pendulum | 2 | 74 | 885 | 3.67e-05 | 0.00013% | 0.081% | 4.5e-06% |
| traced whole step | heldout | midpoint | quaternion | 4 | 296 | 221 | 0.00491 | 2.33% | 8.26% | 0.08% |
| traced whole step | heldout | midpoint | rc_thermal | 3 | 144 | 455 | 0.0275 | 72.93% | 32.03% | 99.10% |
| traced whole step | heldout | ralston2 | dc_motor | 2 | 116 | 564 | 0.0208 | 21.63% | 0.44% | 0.94% |
| traced whole step | heldout | ralston2 | pendulum | 2 | 116 | 564 | 0.0193 | 18.66% | 99.22% | 0.84% |
| traced whole step | heldout | ralston2 | quaternion | 4 | 362 | 181 | 0.00948 | 4.51% | 0.14% | 0.20% |
| traced whole step | heldout | ralston2 | rc_thermal | 3 | 264 | 248 | 0.0332 | 55.20% | 0.21% | 98.02% |
| traced whole step | heldout | rk38 | dc_motor | 2 | 235 | 278 | 0.0147 | 16.86% | 39.47% | 0.54% |
| traced whole step | heldout | rk38 | pendulum | 2 | 235 | 278 | 0.000236 | 0.0044% | 2.67% | 0.00015% |
| traced whole step | heldout | rk38 | quaternion | 4 | 785 | 83 | 0.00972 | 7.42% | 25.56% | 0.25% |
| traced whole step | heldout | rk38 | rc_thermal | 3 | 532 | 123 | 0.0311 | 75.72% | 32.30% | 99.22% |
| traced whole step | heldout | rk4 | dc_motor | 2 | 256 | 256 | 0.0188 | 21.30% | 5.52% | 0.72% |
| traced whole step | heldout | rk4 | pendulum | 2 | 256 | 256 | 0.00467 | 1.31% | 88.74% | 0.046% |
| traced whole step | heldout | rk4 | quaternion | 4 | 716 | 91 | 0.0102 | 6.23% | 2.38% | 0.22% |
| traced whole step | heldout | rk4 | rc_thermal | 3 | 478 | 137 | 0.0344 | 71.15% | 3.36% | 99.01% |
| traced whole step | search | 11e898cb | dahlquist (flagged) | 1 | 86 | 762 | 0.000199 | 1.27% | 99.91% | 99.99% |
| traced whole step | search | 11e898cb | damped_osc | 2 | 144 | 455 | 0.000628 | 12.71% | 0.067% | 0.0066% |
| traced whole step | search | 11e898cb | vanderpol_mild | 2 | 144 | 455 | 0.00163 | 86.02% | 0.023% | 2.3e-05% |
| traced whole step | search | euler | dahlquist (flagged) | 1 | 32 | 2,048 | 4.54e-05 | 9.7e-05% | 12.25% | 72.00% |
| traced whole step | search | euler | damped_osc | 2 | 41 | 1,598 | 0.011 | 5.67% | 48.12% | 27.76% |
| traced whole step | search | euler | vanderpol_mild | 2 | 41 | 1,598 | 0.0447 | 94.33% | 39.63% | 0.24% |
| traced whole step | search | heun2 | dahlquist (flagged) | 1 | 55 | 1,191 | 4.54e-05 | 0.00068% | 57.51% | 97.50% |
| traced whole step | search | heun2 | damped_osc | 2 | 92 | 712 | 0.0028 | 2.57% | 14.75% | 2.45% |
| traced whole step | search | heun2 | vanderpol_mild | 2 | 92 | 712 | 0.0172 | 97.43% | 27.75% | 0.048% |
| traced whole step | search | heun3 | dahlquist (flagged) | 1 | 93 | 704 | 4.54e-05 | 0.00084% | 62.81% | 97.99% |
| traced whole step | search | heun3 | damped_osc | 2 | 164 | 399 | 0.0025 | 2.56% | 12.89% | 1.97% |
| traced whole step | search | heun3 | vanderpol_mild | 2 | 164 | 399 | 0.0154 | 97.44% | 24.30% | 0.039% |
| traced whole step | search | kutta3 | dahlquist (flagged) | 1 | 102 | 642 | 4.54e-05 | 0.00055% | 61.47% | 99.70% |
| traced whole step | search | kutta3 | damped_osc | 2 | 187 | 350 | 0.000868 | 0.20% | 1.52% | 0.24% |
| traced whole step | search | kutta3 | vanderpol_mild | 2 | 187 | 350 | 0.0193 | 99.80% | 37.01% | 0.062% |
| traced whole step | search | midpoint | dahlquist (flagged) | 1 | 50 | 1,310 | 7.67e-05 | 0.00091% | 70.18% | 99.38% |
| traced whole step | search | midpoint | damped_osc | 2 | 74 | 885 | 0.00228 | 0.80% | 4.17% | 0.58% |
| traced whole step | search | midpoint | vanderpol_mild | 2 | 74 | 885 | 0.0253 | 99.20% | 25.64% | 0.037% |
| traced whole step | search | ralston2 | dahlquist (flagged) | 1 | 66 | 992 | 4.54e-05 | 0.0004% | 40.91% | 93.99% |
| traced whole step | search | ralston2 | damped_osc | 2 | 116 | 564 | 0.00443 | 3.83% | 26.33% | 5.94% |
| traced whole step | search | ralston2 | vanderpol_mild | 2 | 116 | 564 | 0.0222 | 96.17% | 32.75% | 0.077% |
| traced whole step | search | rk38 | dahlquist (flagged) | 1 | 130 | 504 | 0.00029 | 0.031% | 98.11% | 99.91% |
| traced whole step | search | rk38 | damped_osc | 2 | 235 | 278 | 0.00335 | 4.19% | 0.89% | 0.089% |
| traced whole step | search | rk38 | vanderpol_mild | 2 | 235 | 278 | 0.016 | 95.78% | 1.01% | 0.0011% |
| traced whole step | search | rk4 | dahlquist (flagged) | 1 | 139 | 471 | 0.000167 | 0.0064% | 93.71% | 99.84% |
| traced whole step | search | rk4 | damped_osc | 2 | 256 | 256 | 0.00254 | 1.46% | 1.45% | 0.15% |
| traced whole step | search | rk4 | vanderpol_mild | 2 | 256 | 256 | 0.0208 | 98.53% | 4.84% | 0.0053% |
(flagged) marks a problem the degeneracy detector flags over the 9 methods in this table on that row's own cost basis: on the analytic basis it flags dahlquist and rc_thermal; on the traced whole-step basis it flags dahlquist (finding 2 gives every verdict and its reason).
The champion was selected under magnitude weighting, so a ratio for that same tableau under another weighting describes the metric and not the search. Every archived record stores its four held-out errors, so re-aggregating all of them answers the other question directly, with no step count and no solve changed. Under magnitude weighting the champion is rank 1 of 141,364; under the median-anchor scale it is rank 9,974, and the tableau that would have won has 6 stages and costs 73 cycles per step; under the reference-norm scale it is rank 15, and the tableau that would have won has 2 stages and costs 17 cycles per step. No classical anchor is ahead of the champion under any of the three, so neither the median-anchor scale nor the reference-norm scale changes whether a discovered tableau would have won, only which one. This is an analytic-basis result by construction: re-pricing steps for 141,364 records would mean replaying the archive rather than re-aggregating it, so it does not say what the search would have found on a part.
| weighting | tableau that would have won | its cycles/step | its error | the published champion's rank | classical anchors ahead of it |
|---|---|---|---|---|---|
| magnitude (published) | 11e898cbab59 (the published champion) | 22 | 0.0286 | 1 of 141,364 | 0 |
| equal, median anchor | b1a8c184b541 | 73 | 0.418 | 9,974 of 141,364 | 0 |
| reference norm | 4feb237eaaa3 | 17 | 0.0801 | 15 of 141,364 | 0 |
Every archived record re-aggregated under each weighting and reduced to one row per unique tableau, on the analytic cost basis, from tools/key_findings.json, counterfactual.numbers.archive_reranking.
| weighting | rank | tableau | stages | cycles/step | tier | measured order | held-out error | the published champion |
|---|---|---|---|---|---|---|---|---|
| magnitude (published) | 1 | 11e898cbab59 | 3 | 22 | no_improvement | 2.018 | 0.0286 | yes |
| magnitude (published) | 2 | 42863b93bf7d | 6 | 75 | heldout_verified | 3.041 | 0.0377 | |
| magnitude (published) | 3 | 6fd075d05984 | 6 | 75 | no_improvement | 3.041 | 0.0384 | |
| magnitude (published) | 4 | a93e22525856 | 6 | 76 | heldout_verified | 3.041 | 0.042 | |
| magnitude (published) | 5 | 83a12de98425 | 3 | 18 | search_only | 2.011 | 0.0424 | |
| magnitude (published) | 6 | 8e9f832cef67 | 6 | 83 | no_improvement | 3.035 | 0.0436 | |
| magnitude (published) | 7 | b1a8c184b541 | 6 | 73 | no_improvement | 3.073 | 0.0445 | |
| magnitude (published) | 8 | b5cf281cc448 | 6 | 75 | no_improvement | 3.011 | 0.045 | |
| magnitude (published) | 9 | aea95967c0fd | 6 | 72 | heldout_verified | 1.989 | 0.0453 | |
| magnitude (published) | 10 | 70b5da1e1bda | 6 | 73 | no_improvement | 3.041 | 0.0462 | |
| equal, median anchor | 1 | b1a8c184b541 | 6 | 73 | no_improvement | 3.073 | 0.418 | |
| equal, median anchor | 2 | 6b7e429a36fd | 6 | 76 | no_improvement | 3.026 | 0.432 | |
| equal, median anchor | 3 | 0b1d0c191699 | 6 | 74 | no_improvement | 3.003 | 0.44 | |
| equal, median anchor | 4 | 9bbd04f081b1 | 6 | 73 | no_improvement | 3.027 | 0.442 | |
| equal, median anchor | 5 | de4bc8f53f01 | 6 | 73 | no_improvement | 3.027 | 0.446 | |
| equal, median anchor | 6 | d8f394874283 | 6 | 74 | no_improvement | 3.104 | 0.447 | |
| equal, median anchor | 7 | 296cd47861a6 | 6 | 70 | no_improvement | 3.030 | 0.447 | |
| equal, median anchor | 8 | 2d21aaf11d93 | 6 | 71 | no_improvement | 3.025 | 0.448 | |
| equal, median anchor | 9 | 3bcc9564053a | 6 | 76 | no_improvement | 3.044 | 0.448 | |
| equal, median anchor | 10 | 4a9a0c03cf41 | 6 | 76 | no_improvement | 3.005 | 0.45 | |
| reference norm | 1 | 4feb237eaaa3 | 2 | 17 | no_improvement | 2.011 | 0.0801 | |
| reference norm | 2 | 757d041459cf | 2 | 17 | no_improvement | 2.011 | 0.081 | |
| reference norm | 3 | f7e5852d9797 | 2 | 17 | search_only | 2.011 | 0.0814 | |
| reference norm | 4 | 704f56750516 | 2 | 17 | no_improvement | 2.011 | 0.0816 | |
| reference norm | 5 | 98d9a14cdd80 | 2 | 17 | no_improvement | 2.011 | 0.0819 | |
| reference norm | 6 | 18e3d22acbe7 | 2 | 17 | search_only | 2.011 | 0.0819 | |
| reference norm | 7 | 205485ec87aa | 2 | 17 | no_improvement | 2.011 | 0.0832 | |
| reference norm | 8 | 99a2da54ecd1 | 2 | 17 | no_improvement | 2.011 | 0.0847 | |
| reference norm | 9 | e36296c9d6f9 | 2 | 17 | no_improvement | 2.011 | 0.0848 | |
| reference norm | 10 | b46b6eaea2ac | 2 | 17 | search_only | 2.011 | 0.085 |
The grid leaves out 15 discovered cell elites that hold a MAP-Elites cell but were never compiled and traced, so they are named as excluded rather than priced by a fit. Every method in the grid has a trace row and every state count the problem set uses is priced, so nothing in the grid is extrapolated. The excluded elites are still counted in the re-ranking above, which changes the aggregate and re-prices nothing.
| tableau | analytic cycles/step | held-out error |
|---|---|---|
| 2d2c2b75d719 | 15 | 0.0613 |
| 4feb237eaaa3 | 17 | 0.0494 |
| 4f501ec6a5ce | 29 | 0.0477 |
| 5fef0bd2b5fe | 30 | 0.0836 |
| 2bceb12dc1c2 | 39 | 0.0587 |
| 19b66f506eec | 46 | 0.0651 |
| 6d0004de713f | 57 | 0.0533 |
| 274563728359 | 59 | 0.0587 |
| 196b1d17d0b7 | 63 | 0.0603 |
| c10bc5417b5b | 63 | 0.0822 |
| 407c65d76723 | 65 | 0.0604 |
| 935100ec0e46 | 65 | 0.0639 |
| aea95967c0fd | 72 | 0.0453 |
| 42863b93bf7d | 75 | 0.0377 |
| 013ba3020051 | 77 | 0.0808 |
Whole rk_step, derivative body excluded. Neither basis is the full per-step cost on a part: the analytic model prices less than the traced count, the traced count prices less than a real application step, and the pair is a bracket rather than a correction. The traced count is used only as a budget denominator, because a real part pays those cycles out of the same 65,536: quoting it against the analytic count as a ratio would report a scope difference as a model error, which DECISIONS.md D42 and D43 forbid. The prices come from an emulator that is instruction accurate and not cycle accurate, under TRACE_HASH 61f3a1bc and VERIFIER_HASH de5bec22, from rk-work/trace/results.json; the aggregates are from tools/key_findings.json, key counterfactual.
Every Q15 multiply rounds toward negative infinity, as ARM's ASRS does,
so the error from each product leans the same way. At these budgets that bias piles up
faster than truncation error shrinks. The counterfactual reruns the same 4
methods, budget and problems with round-to-nearest.
| method | rank, round-to-nearest | RMS, round-to-nearest | rank, floor | RMS, floor |
|---|---|---|---|---|
| euler | 3 | 0.0469 | 1 | 0.0129 |
| heun2 | 4 | 0.061 | 2 | 0.0174 |
| rk38 | 1 | 0.0196 | 3 | 0.0219 |
| rk4 | 2 | 0.0289 | 4 | 0.059 |
| problem | method | floor error | floor weight share | floor rank | round-to-nearest error | round-to-nearest weight share | round-to-nearest rank | verdict over these 4 methods |
|---|---|---|---|---|---|---|---|---|
| dahlquist | euler | 4.54e-05 | 0.00041% | 1 | 0.0799 | 96.74% | 3 | degenerate |
| dahlquist | heun2 | 4.54e-05 | 0.00023% | 2 | 0.0922 | 76.12% | 4 | degenerate |
| dahlquist | rk4 | 0.000167 | 0.00027% | 4 | 0.0362 | 52.44% | 2 | degenerate |
| dahlquist | rk38 | 4.54e-05 | 0.00014% | 3 | 0.0333 | 95.65% | 1 | degenerate |
| damped_osc | euler | 0.0142 | 40.48% | 3 | 0.00801 | 0.97% | 3 | clean |
| damped_osc | heun2 | 0.00554 | 3.38% | 1 | 0.013 | 1.52% | 4 | clean |
| damped_osc | rk4 | 0.0159 | 2.43% | 4 | 0.00736 | 2.17% | 2 | clean |
| damped_osc | rk38 | 0.00817 | 4.63% | 2 | 0.000533 | 0.025% | 1 | clean |
| vanderpol_mild | euler | 0.0172 | 59.52% | 1 | 0.0123 | 2.29% | 2 | clean |
| vanderpol_mild | heun2 | 0.0296 | 96.62% | 2 | 0.05 | 22.36% | 4 | clean |
| vanderpol_mild | rk4 | 0.101 | 97.57% | 4 | 0.0337 | 45.40% | 3 | clean |
| vanderpol_mild | rk38 | 0.0371 | 95.37% | 3 | 0.00708 | 4.33% | 1 | clean |
| pendulum | euler | 0.594 | 86.08% | 4 | 0.554 | 93.16% | 4 | clean |
| pendulum | heun2 | 0.00421 | 0.048% | 3 | 0.00116 | 0.003% | 1 | clean |
| pendulum | rk4 | 0.00403 | 0.048% | 2 | 0.00277 | 0.17% | 3 | clean |
| pendulum | rk38 | 0.0011 | 0.0042% | 1 | 0.00244 | 0.21% | 2 | clean |
| dc_motor | euler | 0.171 | 7.13% | 4 | 0.0162 | 0.08% | 3 | clean |
| dc_motor | heun2 | 0.0974 | 25.69% | 3 | 0.0718 | 11.54% | 4 | clean |
| dc_motor | rk4 | 0.0764 | 17.26% | 2 | 0.0121 | 3.21% | 2 | clean |
| dc_motor | rk38 | 0.052 | 9.30% | 1 | 0.0114 | 4.68% | 1 | clean |
| rc_thermal | euler | 0.156 | 5.92% | 1 | 0.149 | 6.71% | 3 | degenerate |
| rc_thermal | heun2 | 0.156 | 65.79% | 2 | 0.197 | 86.78% | 4 | degenerate |
| rc_thermal | rk4 | 0.158 | 74.04% | 4 | 0.0663 | 96.56% | 2 | degenerate |
| rc_thermal | rk38 | 0.156 | 83.83% | 3 | 0.0511 | 93.98% | 1 | degenerate |
| quaternion | euler | 0.0596 | 0.87% | 4 | 0.0124 | 0.047% | 3 | clean |
| quaternion | heun2 | 0.0559 | 8.47% | 3 | 0.0274 | 1.68% | 4 | clean |
| quaternion | rk4 | 0.0541 | 8.65% | 2 | 0.00171 | 0.064% | 1 | clean |
| quaternion | rk38 | 0.0447 | 6.86% | 1 | 0.00558 | 1.12% | 2 | clean |
The weight share is the problem's share of the sum of squares behind its own aggregate, the search set or the held-out set, which is the weight an RMS gives it. The verdict is the findings site's own degeneracy detector run over the four methods in this table, and it names that field because a verdict belongs to the field it was computed over; the drawing marks the same problems (flagged). The table under the Dahlquist paragraph gives every problem's verdict on this field and on two others.
On the three search problems, under magnitude weighting on the analytic basis (the stage and b combinations only), floor puts euler at rank 1 of these 4 methods (RMS 0.0129) and rk4 at rank 4 (0.059, 4.6× worse), a rank 1 that survives none of the five other cells of the grid (below). Round-to-nearest, on the same weighting and basis, restores the textbook order, with rk4 ahead of euler (0.0289 against 0.0469). Per problem, the best method changes on 5 of the 7 problems, and one of those 5, rc_thermal, is flagged degenerate over these four methods; dahlquist, also flagged over these four methods, is among the problems where it does not change.
Dahlquist shows the mechanism most clearly. Under floor, euler, heun2 and rk38 all report error 4.54e-05, which is the entire reference value: the Q15 state decays to exactly zero, the true answer is below one LSB, and floor's downward bias lands on it. Under round-to-nearest the state stalls above zero and the error is up to 2,032× larger.
Two of the seven problems are flagged degenerate by the same detector the findings validation page uses, computed over the four methods in the table above: dahlquist and rc_thermal. Over the 9 methods finding 1's counterfactual scores on the analytic basis, 8 classical and one discovered, it flags dahlquist and rc_thermal, and over the same 9 on the traced whole-step basis it flags dahlquist. On rc_thermal, 11e898cb, the one discovered method in that field, is the only finisher more than 5 percent from the reference norm, and the other 8 span 1.59 percent, under the 5 percent spread threshold. Of the problems flagged over both the 4-method and the 9-method analytic fields, the rule that fires differs between the two for rc_thermal, the spread rule over the 4 and the pair rule over the 9; the same rule fires on both for dahlquist, the pair rule. The table below gives every problem's verdict and its reason on all three fields.
| problem | over the 4 methods in this finding | over the 9 methods, analytic basis | over the 9 methods, traced whole-step basis |
|---|---|---|---|
| dahlquist | flagged: 3 of the 4 finishers sit within 5 percent of the reference solution's norm of 4.53999e-05 and agree with each other to within 5 percent, so those runs returned approximately the reference and the comparison is reporting the problem rather than the method | flagged: 4 of the 9 finishers sit within 5 percent of the reference solution's norm of 4.53999e-05 and agree with each other to within 5 percent, so those runs returned approximately the reference and the comparison is reporting the problem rather than the method | flagged: 5 of the 9 finishers sit within 5 percent of the reference solution's norm of 4.53999e-05 and agree with each other to within 5 percent, so those runs returned approximately the reference and the comparison is reporting the problem rather than the method |
| damped_osc | not flagged: the 4 finishers' Q15 errors span 187.30 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.0176384, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 304.24 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.0176384, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 1643.90 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.0176384, so the comparison is about the methods |
| vanderpol_mild | not flagged: the 4 finishers' Q15 errors span 486.22 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.77394, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 486.22 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.77394, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 2632.54 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.77394, so the comparison is about the methods |
| pendulum | not flagged: the 4 finishers' Q15 errors span 53902.14 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 921909.64 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 8181338.72 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods |
| dc_motor | not flagged: the 4 finishers' Q15 errors span 228.95 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.01989, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 518.66 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.01989, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 303.76 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.01989, so the comparison is about the methods |
| rc_thermal | flagged: the 4 finishers' Q15 errors span 1.45 percent from best to worst, under the 5 percent threshold, so the comparison is reporting the problem rather than the method | flagged: 8 of the 9 finishers sit within 5 percent of the reference solution's norm of 0.159158 and agree with each other to within 5 percent, so those runs returned approximately the reference and the comparison is reporting the problem rather than the method | not flagged: the 9 finishers' Q15 errors span 536.87 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.159158, so the comparison is about the methods |
| quaternion | not flagged: the 4 finishers' Q15 errors span 33.41 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 211.86 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods | not flagged: the 9 finishers' Q15 errors span 2045.50 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1, so the comparison is about the methods |
euler holds rank 1 of these four methods in exactly one cell of the six: magnitude weighting on the analytic basis, which is the cell the paragraph above is written from. At the traced whole-step price (the whole rk_step, derivative body excluded) under magnitude weighting it is rank 4 of 4 behind rk38, so its rank 1 does not survive the compiled cost basis. On the analytic basis (the stage and b combinations only) it is rank 3 of 4 behind heun2 under both the median-anchor scale and the reference-norm scale. On the traced whole-step basis (the whole rk_step, derivative body excluded) it is rank 2 of 4 behind heun2 under both the median-anchor scale and the reference-norm scale. So euler's rank 1 survives in none of the five other cells.
| weighting | cost basis | method | search-set RMS | rank of the nine scored | rank of the four in this finding |
|---|---|---|---|---|---|
| magnitude (published) | analytic (published) | 11e898cb | 0.0115 | 1 of 9 | n/a |
| magnitude (published) | analytic (published) | euler | 0.0129 | 2 of 9 | 1 of 4 |
| magnitude (published) | analytic (published) | midpoint | 0.0131 | 3 of 9 | n/a |
| magnitude (published) | analytic (published) | heun3 | 0.0164 | 4 of 9 | n/a |
| magnitude (published) | analytic (published) | heun2 | 0.0174 | 5 of 9 | 2 of 4 |
| magnitude (published) | analytic (published) | rk38 | 0.0219 | 6 of 9 | 3 of 4 |
| magnitude (published) | analytic (published) | ralston2 | 0.0237 | 7 of 9 | n/a |
| magnitude (published) | analytic (published) | kutta3 | 0.0545 | 8 of 9 | n/a |
| magnitude (published) | analytic (published) | rk4 | 0.059 | 9 of 9 | 4 of 4 |
| magnitude (published) | traced whole step | 11e898cb | 0.00102 | 1 of 9 | n/a |
| magnitude (published) | traced whole step | heun3 | 0.00903 | 2 of 9 | n/a |
| magnitude (published) | traced whole step | rk38 | 0.00946 | 3 of 9 | 1 of 4 |
| magnitude (published) | traced whole step | heun2 | 0.0101 | 4 of 9 | 2 of 4 |
| magnitude (published) | traced whole step | kutta3 | 0.0111 | 5 of 9 | n/a |
| magnitude (published) | traced whole step | rk4 | 0.0121 | 6 of 9 | 3 of 4 |
| magnitude (published) | traced whole step | ralston2 | 0.0131 | 7 of 9 | n/a |
| magnitude (published) | traced whole step | midpoint | 0.0147 | 8 of 9 | n/a |
| magnitude (published) | traced whole step | euler | 0.0265 | 9 of 9 | 4 of 4 |
| equal, median anchor | analytic (published) | heun2 | 0.795 | 1 of 9 | 1 of 4 |
| equal, median anchor | analytic (published) | midpoint | 0.968 | 2 of 9 | n/a |
| equal, median anchor | analytic (published) | rk38 | 1 | 3 of 9 | 2 of 4 |
| equal, median anchor | analytic (published) | euler | 1.22 | 4 of 9 | 3 of 4 |
| equal, median anchor | analytic (published) | ralston2 | 1.42 | 5 of 9 | n/a |
| equal, median anchor | analytic (published) | heun3 | 1.73 | 6 of 9 | n/a |
| equal, median anchor | analytic (published) | 11e898cb | 1.93 | 7 of 9 | n/a |
| equal, median anchor | analytic (published) | kutta3 | 2.31 | 8 of 9 | n/a |
| equal, median anchor | analytic (published) | rk4 | 2.66 | 9 of 9 | 4 of 4 |
| equal, median anchor | traced whole step | heun3 | 0.542 | 1 of 9 | n/a |
| equal, median anchor | traced whole step | kutta3 | 0.548 | 2 of 9 | n/a |
| equal, median anchor | traced whole step | heun2 | 0.566 | 3 of 9 | 1 of 4 |
| equal, median anchor | traced whole step | ralston2 | 0.671 | 4 of 9 | n/a |
| equal, median anchor | traced whole step | midpoint | 0.866 | 5 of 9 | n/a |
| equal, median anchor | traced whole step | euler | 1.23 | 6 of 9 | 2 of 4 |
| equal, median anchor | traced whole step | rk4 | 1.64 | 7 of 9 | 3 of 4 |
| equal, median anchor | traced whole step | 11e898cb | 1.88 | 8 of 9 | n/a |
| equal, median anchor | traced whole step | rk38 | 2.77 | 9 of 9 | 4 of 4 |
| reference norm | analytic (published) | heun2 | 0.606 | 1 of 9 | 1 of 4 |
| reference norm | analytic (published) | rk38 | 0.637 | 2 of 9 | 2 of 4 |
| reference norm | analytic (published) | euler | 0.741 | 3 of 9 | 3 of 4 |
| reference norm | analytic (published) | ralston2 | 0.764 | 4 of 9 | n/a |
| reference norm | analytic (published) | midpoint | 0.999 | 5 of 9 | n/a |
| reference norm | analytic (published) | kutta3 | 2.14 | 6 of 9 | n/a |
| reference norm | analytic (published) | heun3 | 2.14 | 7 of 9 | n/a |
| reference norm | analytic (published) | rk4 | 2.19 | 8 of 9 | 4 of 4 |
| reference norm | analytic (published) | 11e898cb | 2.53 | 9 of 9 | n/a |
| reference norm | traced whole step | kutta3 | 0.578 | 1 of 9 | n/a |
| reference norm | traced whole step | heun3 | 0.583 | 2 of 9 | n/a |
| reference norm | traced whole step | heun2 | 0.585 | 3 of 9 | 1 of 4 |
| reference norm | traced whole step | ralston2 | 0.596 | 4 of 9 | n/a |
| reference norm | traced whole step | euler | 0.68 | 5 of 9 | 2 of 4 |
| reference norm | traced whole step | midpoint | 0.978 | 6 of 9 | n/a |
| reference norm | traced whole step | rk4 | 2.13 | 7 of 9 | 3 of 4 |
| reference norm | traced whole step | 11e898cb | 2.53 | 8 of 9 | n/a |
| reference norm | traced whole step | rk38 | 3.68 | 9 of 9 | 4 of 4 |
Search-set RMS over the three search problems, one row per method in every cell. The analytic basis prices the stage and b combinations only; the traced whole-step basis buys steps at the count for the whole rk_step with the derivative body excluded, from an emulator that is instruction accurate and not cycle accurate, and it is used only as a budget denominator.
The premise was tested before the search started. If truncation error dominated roundoff at practical step sizes, classical analysis would already give the right answer and the search would be pointless. The falsification run swept the step size on damped_osc, computing each method's error twice, in Q15 and in float64 over exactly the same steps, so the gap between the curves is the arithmetic alone.
| method | step size h | Q15 error | float64 error |
|---|---|---|---|
| rk4 | 0.00977 | 0.0258 | 5.84e-11 |
| rk4 | 0.0195 | 0.0399 | 9.35e-10 |
| rk4 | 0.0391 | 0.0156 | 1.5e-08 |
| rk4 | 0.0781 | 0.00479 | 2.41e-07 |
| rk4 | 0.156 | 0.00254 | 3.88e-06 |
| rk4 | 0.312 | 0.00113 | 6.29e-05 |
| rk4 | 0.625 | 0.00134 | 0.00102 |
| rk4 | 1.25 | n/a | 0.0125 |
| rk4 | 2.5 | n/a | 0.0176 |
| rk4 | 5 | n/a | 2.37e+10 |
| heun2 | 0.00977 | 0.0276 | 1.23e-05 |
| heun2 | 0.0195 | 0.0126 | 4.92e-05 |
| heun2 | 0.0391 | 0.00642 | 0.000197 |
| heun2 | 0.0781 | 0.00297 | 0.000786 |
| heun2 | 0.156 | 0.00401 | 0.00313 |
| heun2 | 0.312 | 0.0128 | 0.0123 |
| heun2 | 0.625 | 0.0467 | 0.0467 |
| heun2 | 1.25 | n/a | 17.5 |
| heun2 | 2.5 | n/a | 3.68e+07 |
| heun2 | 5 | n/a | 4.41e+08 |
rk4's Q15 error bottoms out at 0.00113, separates from the float curve at h = 0.156, and climbs back to 0.0258 by h = 0.00977: below the crossover, more steps mean more floored multiplies and less accuracy. heun2 separates at h = 0.0391. Both crossovers fall inside the practical step-size range, as the premise needed.
The stored verdict is "mixed": coefficient arithmetic is 50 to 56% of per-step cycles under the fast multiplier, above the 30% proceed line, but heun2 under the slow one sits at 14.8%, just under the 15% kill line. At the shared budget under floor, rk4 is the best of the 4 methods on none of the 7 problems, and dahlquist and rc_thermal, two of those 7, are flagged degenerate over the same 4 methods (finding 2).
The shape of this is not new, and reporting it that way is the point. Analysis of low-precision Runge-Kutta integration finds that a round-to-nearest solution stagnates once the timestep is small enough, with the rounding term growing as the step shrinks. Measuring the same turn-up in fixed point under floor rounding says the harness is seeing a real effect rather than an artifact of its own arithmetic (related work).
rc_thermal is the archive's stiff problem: a three-state RC network decaying toward equilibrium, with its derivative stored at DERIV_SCALE = 1/8 and one LSB worth 0.000122 in physical units. All 8 classical methods report errors between 0.1558 and 0.1582, a spread of 1.6%. They agree because every one of them collapses.
| method | floor error | share of its held-out sum of squares | round-to-nearest error | steps | final Q15 state |
|---|---|---|---|---|---|
| euler | 0.156 | 5.92% | 0.149 | 4,369 | (14, 15, 17) |
| midpoint | 0.156 | 85.06% | n/a | 1,985 | (15, 15, 18) |
| heun2 | 0.156 | 65.79% | 0.197 | 1,680 | (14, 15, 17) |
| ralston2 | 0.156 | 72.03% | n/a | 1,365 | (14, 15, 17) |
| heun3 | 0.156 | 83.84% | n/a | 949 | (14, 15, 16) |
| kutta3 | 0.158 | 82.20% | n/a | 840 | (4, 4, 5) |
| rk4 | 0.158 | 74.04% | 0.0663 | 661 | (4, 4, 6) |
| rk38 | 0.156 | 83.83% | 0.0511 | 606 | (13, 14, 16) |
| best discovered, under floor | 0.0205 | n/a | n/a | n/a | n/a |
The share is rc_thermal's part of that method's held-out sum of squares under magnitude weighting on the analytic basis, from counterfactual.series.per_method; the best discovered method on rc_thermal is not one of the methods scored there, so its row has none. rc_thermal (flagged): the degeneracy detector flags it over the 4 methods finding 2 publishes and the 9 methods finding 1's counterfactual scores on the analytic basis (finding 2).
Each method drives the Q15 state to within 5 to 18 LSBs of the origin (euler ends at raw state (14, 15, 17), rk4 at (4, 4, 6)) while the true solution still has norm 0.1592 at t = 4, so the reported error is the reference itself. Late in the decay the scaled derivative terms are a few LSBs, and floor turns them into extra decay.
It is a floor effect, not a limit of Q15: under round-to-nearest rk38 reaches 0.0511, and the best discovered method on this problem, a 2-stage, order-2 tableau at 17 cycles per step, reaches 0.0205 while still using floor.
The degeneracy detector flags rc_thermal over the 4 methods finding 2 publishes and the 9 methods finding 1's counterfactual scores on the analytic basis, and not over those 9 on the traced whole-step basis. The share column in the table above is rc_thermal's part of each method's held-out sum of squares under magnitude weighting on the analytic basis.
Phase 0 is the one part of the space small enough to close completely: two-stage methods with a21 on a dyadic lattice, 256 candidates, of which 16 have exactly representable b weights. All 16 were enumerated, verified and archived, so the result holds over the whole space rather than a sample. It is an exhaustive evaluation and not a theorem, and it holds only for these four held-out problems, this 65,536-cycle budget, and Q15 with floor rounding. The top two members finish 0.04% apart, which is a tie at this resolution rather than an ordering.
| a21 | b | held-out error | cycles/step | rank | textbook name |
|---|---|---|---|---|---|
| -1/8 | (5/1, -4/1) | 0.0613 | 15 | 1 | |
| -1/4 | (3/1, -2/1) | 0.0614 | 15 | 2 | |
| -1/32 | (17/1, -16/1) | 0.071 | 15 | 3 | |
| 1/8 | (-3/1, 4/1) | 0.0763 | 15 | 4 | |
| 1/4 | (-1/1, 2/1) | 0.0781 | 13 | 5 | |
| -1/2 | (2/1, -1/1) | 0.0781 | 13 | 6 | |
| 1/2 | (0/1, 1/1) | 0.0844 | 11 | 7 | midpoint |
| -1/64 | (33/1, -32/1) | 0.0849 | 15 | 8 | |
| -1/16 | (9/1, -8/1) | 0.0889 | 15 | 9 | |
| 1/32 | (-15/1, 16/1) | 0.0898 | 15 | 10 | |
| 2/1 | (3/4, 1/4) | 0.0937 | 15 | 11 | |
| 1/1 | (1/2, 1/2) | 0.0961 | 13 | 12 | heun2 |
| 1/16 | (-7/1, 8/1) | 0.102 | 15 | 13 | |
| -2/1 | (5/4, -1/4) | 0.109 | 15 | 14 | |
| -1/1 | (3/2, -1/2) | 0.11 | 14 | 15 | |
| 1/64 | (-31/1, 32/1) | 0.136 | 15 | 16 |
The two best members are a21 = −1/8 with b = (5, −4), at held-out error 0.0613 and 15 cycles per step, and a21 = −1/4 at 0.0614, 0.04% behind it: a tie, not a ranking. Neither is a textbook method, and both have lower held-out error than all 8 classical methods (the best of those is midpoint at 0.0844). Within the family, midpoint ranks 7 and heun2 12 of 16.
A ranking of 16 members whose top two are 0.04% apart is worth testing against the problem set that produced it. Dropping each held-out problem in turn and re-ranking every member gives five orderings. The member ranked first changes across them (−1/32, −1/4, −1/8 and 1/16), and the top two places are not the same pair every time. The two that lead the full set never fall below rank 3 in any of the 5 orderings. The sharpest move is a21 = 1/16, which ranks 13 on the full set and first once dc_motor is dropped.
| a21 | all four kept | drop pendulum | drop dc_motor | drop rc_thermal | drop quaternion |
|---|---|---|---|---|---|
| −1/8 | 1 | 3 | 2 | 1 | 2 |
| −1/4 | 2 | 2 | 3 | 2 | 1 |
| −1/32 | 3 | 1 | 7 | 6 | 3 |
| 1/8 | 4 | 5 | 6 | 7 | 4 |
| 1/4 | 5 | 7 | 9 | 3 | 5 |
| −1/2 | 6 | 6 | 8 | 4 | 6 |
| 1/2 (midpoint) | 7 | 8 | 10 | 5 | 7 |
| −1/64 | 8 | 4 | 11 | 10 | 8 |
| −1/16 | 9 | 9 | 4 | 12 | 9 |
| 1/32 | 10 | 10 | 5 | 11 | 11 |
| 2 | 11 | 12 | 14 | 8 | 10 |
| 1 (heun2) | 12 | 13 | 13 | 9 | 12 |
| 1/16 | 13 | 14 | 1 | 15 | 13 |
| −2 | 14 | 15 | 15 | 13 | 14 |
| −1 | 15 | 16 | 12 | 14 | 15 |
| 1/64 | 16 | 11 | 16 | 16 | 16 |
Rank by held-out RMS over the problems each column keeps, from tools/key_findings.json, phase0_exhaustive.numbers.rank_stability. The first column is the published ranking, recomputed here from the per-problem errors.
Everything above uses the archive's own search and held-out problems. The validation suite asks whether the winners transfer to 5 equations from embedded applications (power electronics, battery management, vehicle dynamics / ADAS, communications / clocking and biomedical devices) that no optimizer saw, run at the same 65,536-cycle budget in Q15 with floor rounding and scored against independent reference solutions.
The comparator is fixed. It is the champion from finding 1, picked at cycle 33 out of 141,364 archived records, before this suite ran, with its coefficients unchanged. Against the best of the 5 classical anchors on each problem it has the lower Q15 error on 3 of 5 non-stiff problems, and on 0 of the 1 stiff problems where both sides finish, with 1 tie inside the 2 percent band. Taking the anchors one at a time instead of the best of them, one discovered method against 5 classical ones with no maximum on either side, it is lower in 27 of 30 cells, higher in 2 and tied in 1. The champion is not ahead everywhere: bicycle_lateral, where midpoint reaches 0.000395 against the champion's 0.000539 (ratio 1.37) and pll_lock, where rk38 reaches 0.0128 against the champion's 0.0159 (ratio 1.24). On bicycle_lateral it is also not the discovered method with the lowest error: 196b1d17d0b7 reaches 0.000177 against 0.000539, so that row counts for the discovered side in the best-of-N view and against the champion here.
| problem | champion Q15 error | best classical anchor | its Q15 error | champion against it |
|---|---|---|---|---|
| buck_converter | 0.00198 | rk38 | 0.0208 | lower |
| battery_2rc | 0.0369 | heun2 | 0.0776 | lower |
| bicycle_lateral | 0.000539 | midpoint | 0.000395 | higher |
| pll_lock | 0.0159 | rk38 | 0.0128 | higher |
| glucose_minimal | 0.0207 | midpoint | 0.0373 | lower |
| servo_load_step (stiff) | 0.0116 | midpoint | 0.0116 | tie |
| enzyme_qssa (stiff) (flagged) | 0.00966 | midpoint | 0.0097 | left out, degenerate |
| robertson_scaled (stiff) | n/a | midpoint | 0.0894 | no finish |
Every problem in the suite, with the champion against the best of the anchors that finished it. Read from rk-work/validation/results.json. (flagged) marks a problem the degeneracy detector flags over the finishers among all 8 methods the suite ran on it, and a flagged problem is left out of both tallies.
One problem is left out of both tallies: enzyme_qssa because the 6 finishers' Q15 errors span 2.68 percent from best to worst, under the 5 percent threshold, so the comparison is reporting the problem rather than the method. A ratio within 2% of 1.0 is counted as a tie rather than a win, on both sides of the comparison. The findings validation page applies the same two rules to the same document, so the two sites report one tally. This suite is summarized by medians of per-problem ratios and by win counts rather than by an RMS, so no problem carries a weight in its tables and there is no share to report in them, unlike the per-problem tables under finding 1 and finding 2.
| problem | best classical | its Q15 error | best discovered | its Q15 error | ratio, discovered over classical |
|---|---|---|---|---|---|
| buck_converter | rk38 | 0.0208 | 11e898cbab59 | 0.00198 | 0.095 |
| battery_2rc | heun2 | 0.0776 | 11e898cbab59 | 0.0369 | 0.475 |
| bicycle_lateral | midpoint | 0.000395 | 196b1d17d0b7 | 0.000177 | 0.450 |
| pll_lock | rk38 | 0.0128 | 11e898cbab59 | 0.0159 | 1.240 |
| glucose_minimal | midpoint | 0.0373 | 11e898cbab59 | 0.0207 | 0.556 |
Taking the lowest error on each side reads 4 of 5 on the non-stiff problems, with a median error ratio of 0.475 and the widest margin on buck_converter (0.00198 against 0.0208 for rk38, 10.5×). Both sides are maxima there, and the discovered side is a maximum over methods already selected on error, so that count rises with the number of discovered methods run. That is why the finding leads with one fixed comparator instead.
Float64 runs of the same tableaus and step counts are at least 337× more accurate in every case, so these errors measure quantization, and the largest raw Q15 value in any run is 20,130 of 32,767, so none leaned on overflow luck. The 3 stiff problems tell a different story (overflow, not accuracy); they and the full tables are on the findings validation page.
The chart shows one column of the full matrix folded below it; no single column ranks the methods.
| method | kind | held-out RMS error |
|---|---|---|
| 11e898cb (discovered champion) | discovered | 0.0286 |
| 42863b93 (best order-3 elite) | discovered | 0.0377 |
| 196b1d17 (best order-4 elite) | discovered | 0.0603 |
| midpoint | classical | 0.0844 |
| heun3 | classical | 0.0852 |
| rk38 | classical | 0.0853 |
| kutta3 | classical | 0.0873 |
| ralston2 | classical | 0.0919 |
| rk4 | classical | 0.0919 |
| heun2 | classical | 0.0961 |
| euler | classical | 0.32 |
At matched tolerance the float64 libraries are far more accurate than any Q15 method (median error ratio, best Q15 over best library, 519), and at identical step counts float64 rk4 has the lower error in all 56 comparable cells. That is the price of 16-bit floor arithmetic. The Q15 methods offer bounded integer arithmetic at a fixed cycle cost known in advance, which no library here provides, and within that regime the discovered methods have the lowest held-out error (finding 1). The stiff column is the weak spot of the whole explicit class: an expensive tableau takes large steps at a fixed budget, leaves the stability region and overflows. That failure is the argument for the implicit class.
| method | kind | order | stages | held-out error at budgeta | validation wins (practical / stiff)b | cycles/step fastc | cycles/step slowc | CSD weightc | measured µs/stepd | stiff validation subsete | notes |
|---|---|---|---|---|---|---|---|---|---|---|---|
| euler | classical | 1 | 1 | 0.32 | 0 / 0 | 5 | 5 | 0 | 6 | finishes all 3 | Cheapest step and smallest code. Its tiny budget-driven steps stay inside the stability region on the stiff runs; the least accurate method elsewhere. |
| midpoint | classical | 2 | 2 | 0.0844 | 0 / 2 | 11 | 11 | 1 | 12 | finishes all 3 | The best classical method at this budget and arithmetic, and the strongest on the stiff problems. |
| heun2 | classical | 2 | 2 | 0.0961 | 0 / 0 | 13 | 13 | 2 | 14 | finishes all 3 | Same cost class as midpoint and slightly worse throughout. |
| rk4 | classical | 4 | 4 | 0.0919 | 0 / 0 | 33 | 85 | 34 | 32 | finishes 2 of 3; overflows on robertson_scaled | The textbook default. Its classically optimal coefficients buy nothing at this budget in Q15, and it is expensive under a slow multiplier. |
| rk38 | classical | 4 | 4 | 0.0853 | 1 / 0 | 36 | 64 | 22 | 33 | finishes 2 of 3; overflows on robertson_scaled | Same order and stages as rk4; under the frozen epoch-1 rule it is cheaper with a slow multiplier. |
| 11e898cb (discovered champion) | discovered | 2 | 3 | 0.0286 | 3 / 1 | 22 | 29 | 8 | 20 | finishes 2 of 3; overflows on robertson_scaled | The three-stage, order-2 discovered champion, with the lowest held-out error in the archive. Its edge comes from how it handles floor bias, not from truncation order. |
| 42863b93 (best order-3 elite) | discovered | 3 | 6 | 0.0377 | 0 / 0 | 75 | 120 | 48 | 66 | overflows on all 3 | The best order-3 archive elite. Strong on the held-out set, but six stages mean large steps at a fixed budget, which the stiff problems punish. |
| 196b1d17 (best order-4 elite) | discovered | 4 | 6 | 0.0603 | 1 / 0 | 63 | 105 | 40 | 72 | overflows on all 3 | The best order-4 archive elite, with the same six-stage stiff weakness. |
| RK45 | library (SciPy, float64, adaptive) | n/a | n/a | n/aa | n/ab | n/ac | n/ac | n/ac | 64 | n/ae | SciPy's general-purpose default: explicit, adaptive, float64. It picks its step count at run time, so there is no fixed cycle cost to certify. |
| Radau | library (SciPy, float64, adaptive) | n/a | n/a | n/aa | n/ab | n/ac | n/ac | n/ac | 224 | n/ae | Implicit Radau IIA, built for the stiff problems that limit the explicit methods. |
| BDF | library (SciPy, float64, adaptive) | n/a | n/a | n/aa | n/ab | n/ac | n/ac | n/ac | 133 | n/ae | Implicit multistep for stiff problems, with adaptive step counts and no cycle model. |
| LSODA | library (SciPy, float64, adaptive) | n/a | n/a | n/aa | n/ab | n/ac | n/ac | n/ac | 15 | n/ae | Switches between stiff and non-stiff modes on its own; the same caveats as the other library rows. |
tools/key_findings.json (series
frontier_cycles_vs_heldout, matched by tableau hash). The libraries never
ran this protocol, because they are adaptive and float64.rk-work/validation/results.json, split into
practical (5 non-stiff) and stiff (3). The libraries are not part of the
validation suite.costmodel and
coeffrep modules. CSD weight is the shift-add length of the coefficient
multiplies, a proxy for code size. Adaptive libraries have no fixed cycles per step.rk-work/benchmark/results.json. Q15 rows run the pinned solver in the
interpreter and library rows run compiled code, so times compare like with like only
within a regime. Methods outside the benchmark run have no time.rk-work/validation/results.json. An
overflow is a Q15OverflowError from the pinned solver. The libraries were
not run on these problems.The benchmark checks the cycle model against a clock: it runs the champion and rk4 through the same pinned Q15 solver on all 7 scored problems, so only the tableau differs and the per-step time ratio isolates what the coefficients cost.
| problem | measured ratio | cycle model ratio | champion µs/step | rk4 µs/step | error ratio, champion over rk4 |
|---|---|---|---|---|---|
| dahlquist (flagged) | 1.602 | 1.50 | 13.17 | 21.09 | 1.187 |
| damped_osc | 1.698 | 1.50 | 20.47 | 34.75 | 0.247 |
| vanderpol_mild | 1.615 | 1.50 | 19.81 | 31.99 | 0.193 |
| pendulum | 1.332 | 1.50 | 21.84 | 29.08 | 1.177 |
| dc_motor | 1.660 | 1.50 | 18.61 | 30.88 | 0.362 |
| rc_thermal (flagged) | 1.175 | 1.50 | 32.83 | 38.58 | 0.291 |
| quaternion | 1.191 | 1.50 | 38.10 | 45.37 | 0.353 |
(flagged) marks a problem the degeneracy detector flags over the 9 methods finding 1's counterfactual scores on the analytic basis, a field that holds both methods in this table (finding 2 gives every verdict and its reason).
Every problem lands within 22% of the predicted ratio. At the same budget the champion also has the lower error on 5 of the 7 problems (rk4 keeps dahlquist and pendulum; median error ratio 0.35), so the time saved per step does not cost accuracy. Two of those 7, dahlquist and rc_thermal, are flagged degenerate over the 9 methods finding 1's counterfactual scores on the analytic basis, a field that holds both rk4 and the champion: dahlquist is among the problems rk4 keeps and rc_thermal among the champion's. Python timings check the model rather than stand in for the chip; the protocol and its caveats are on the findings validation page.
Code assigns each record's tier when it is written, by comparison with its cell's incumbent: heldout_verified when it improved on the incumbent in both search-set and held-out error across at least two problem families, search_only when its search error improved and its held-out error did not, no_incumbent when the cell was empty, and no_improvement when there was an incumbent and neither of those rules applied, which includes a held-out improvement without a search improvement. Before the split, no_incumbent and no_improvement were stored as one word, unreplicated, which is still a legal stored tier. Of the 141,364 archive records this page was built from, 40 are heldout_verified, 59,125 search_only, 21 no_incumbent, 82,178 no_improvement and 0 unreplicated. The model proposes hypotheses only as predicates in a closed grammar, and code writes the verdict: no data or an effect below Cohen's d of 0.2 is inconclusive, and refuted hypotheses are fed back into later prompts. The premise got the same treatment, a falsification run with thresholds fixed in advance and its verdict published either way (finding 3).
The cycle model is analytic and has never been checked against a physical chip inside the loop. Measured order comes from one scalar problem in float64. Error is judged at the final time, not along the trajectory, and floor is one hardware rounding convention among several. Optimality claims from the enumerated phases hold only within their lattices, and four held-out problems are a strong filter but not a statistical guarantee. The full list, with the reasoning, is on the findings methodology page.
Everything on this page is the explicit class. The implicit and adaptive classes are measured in float64, outside the scored archive, and their results are on the findings site's implicit and adaptive pages.