Problems no search saw, measured speed, and the premise test.
The practical validation suite runs 8 problems from embedded application domains that no optimizer saw and no archive statistic includes. Discovered methods from the live archive and the classical anchors integrate each one in Q15 at the same cycle budget, 65536 cycles under m0plus_fast with floor rounding, and each run is scored by its final-state error against an independent reference solution.
The suite below reads validation/results.json, built at 141,364 archive records, verifier hash de5bec22.
Python wall clock on a desktop preserves the cycle model's ordering, and the cycle counts are modeled rather than measured on a Cortex-M0+.
Source: rk-work/validation/results.json, results. Arithmetic: Q15 with floor rounding under m0plus_fast; the table below this chart carries each run's float64 error over the same steps.
| problem | method | kind | Q15 error | float64 error | steps | cycles/step |
|---|---|---|---|---|---|---|
| buck_converter | euler | classical | 0.0253608 | 7.5204e-05 | 6553 | 10 |
| buck_converter | heun2 | classical | 0.0434966 | 4.99565e-07 | 2520 | 26 |
| buck_converter | midpoint | classical | 0.022902 | 3.57687e-07 | 2978 | 22 |
| buck_converter | rk4 | classical | 0.03238 | 1.14781e-10 | 992 | 66 |
| buck_converter | rk38 | classical | 0.0208238 | 1.62089e-10 | 910 | 72 |
| buck_converter | 11e898cb | discovered | 0.00198385 | 9.69071e-07 | 1489 | 44 |
| buck_converter | 42863b93 | discovered | 0.0147768 | 3.04888e-07 | 436 | 150 |
| buck_converter | 196b1d17 | discovered | 0.00432324 | 1.52023e-09 | 520 | 126 |
| battery_2rc | euler | classical | 0.0986711 | 4.57811e-05 | 4369 | 15 |
| battery_2rc | heun2 | classical | 0.0776365 | 7.68948e-08 | 1680 | 39 |
| battery_2rc | midpoint | classical | 0.0905505 | 5.50663e-08 | 1985 | 33 |
| battery_2rc | rk4 | classical | 0.117303 | 7.60152e-13 | 661 | 99 |
| battery_2rc | rk38 | classical | 0.105182 | 1.07549e-12 | 606 | 108 |
| battery_2rc | 11e898cb | discovered | 0.0368753 | 1.49545e-07 | 992 | 66 |
| battery_2rc | 42863b93 | discovered | 0.056326 | 8.3658e-09 | 291 | 225 |
| battery_2rc | 196b1d17 | discovered | 0.0442465 | 1.01584e-11 | 346 | 189 |
| bicycle_lateral | euler | classical | 0.105013 | 5.45857e-07 | 6553 | 10 |
| bicycle_lateral | heun2 | classical | 0.0561341 | 1.42198e-09 | 2520 | 26 |
| bicycle_lateral | midpoint | classical | 0.000394556 | 1.01808e-09 | 2978 | 22 |
| bicycle_lateral | rk4 | classical | 0.0439865 | 3.64986e-14 | 992 | 66 |
| bicycle_lateral | rk38 | classical | 0.00394005 | 5.27405e-14 | 910 | 72 |
| bicycle_lateral | 11e898cb | discovered | 0.000539394 | 2.75832e-09 | 1489 | 44 |
| bicycle_lateral | 42863b93 | discovered | 0.0459941 | 1.05546e-10 | 436 | 150 |
| bicycle_lateral | 196b1d17 | discovered | 0.0001774 | 5.01308e-13 | 520 | 126 |
| pll_lock | euler | classical | 0.0481865 | 6.31115e-06 | 6553 | 10 |
| pll_lock | heun2 | classical | 0.0325573 | 3.90381e-08 | 2520 | 26 |
| pll_lock | midpoint | classical | 0.0334146 | 2.80062e-08 | 2978 | 22 |
| pll_lock | rk4 | classical | 0.0386035 | 7.64901e-13 | 992 | 66 |
| pll_lock | rk38 | classical | 0.0128489 | 1.08125e-12 | 910 | 72 |
| pll_lock | 11e898cb | discovered | 0.0159381 | 7.60478e-08 | 1489 | 44 |
| pll_lock | 42863b93 | discovered | 0.0244659 | 7.98545e-09 | 436 | 150 |
| pll_lock | 196b1d17 | discovered | 0.0243841 | 1.00502e-11 | 520 | 126 |
| glucose_minimal | euler | classical | 0.0840464 | 3.9061e-05 | 6553 | 10 |
| glucose_minimal | heun2 | classical | 0.0738776 | 2.96109e-08 | 2520 | 26 |
| glucose_minimal | midpoint | classical | 0.0372548 | 4.36692e-08 | 2978 | 22 |
| glucose_minimal | rk4 | classical | 0.0542891 | 1.23695e-13 | 992 | 66 |
| glucose_minimal | rk38 | classical | 0.0610666 | 2.42427e-13 | 910 | 72 |
| glucose_minimal | 11e898cb | discovered | 0.0207014 | 5.20727e-08 | 1489 | 44 |
| glucose_minimal | 42863b93 | discovered | 0.0280853 | 2.35809e-09 | 436 | 150 |
| glucose_minimal | 196b1d17 | discovered | 0.0279905 | 2.04531e-12 | 520 | 126 |
Source: rk-work/validation/results.json, results. Arithmetic: Q15 with floor rounding under m0plus_fast; the table below this chart carries each run's float64 error over the same steps.
| problem | method | kind | Q15 error | float64 error | steps | cycles/step |
|---|---|---|---|---|---|---|
| servo_load_step | euler | classical | 0.0118808 | 2.3216e-05 | 6553 | 10 |
| servo_load_step | heun2 | classical | 0.0118296 | 3.67262e-08 | 2520 | 26 |
| servo_load_step | midpoint | classical | 0.0115746 | 2.62928e-08 | 2978 | 22 |
| servo_load_step | rk4 | classical | 0.0124489 | 2.62516e-13 | 992 | 66 |
| servo_load_step | rk38 | classical | 0.0148399 | 3.68864e-13 | 910 | 72 |
| servo_load_step | 11e898cb | discovered | 0.0115754 | 7.12907e-08 | 1489 | 44 |
| enzyme_qssa | euler | classical | 0.00974111 | 1.89045e-05 | 6553 | 10 |
| enzyme_qssa | heun2 | classical | 0.00974111 | 3.42268e-08 | 2520 | 26 |
| enzyme_qssa | midpoint | classical | 0.00969807 | 2.09683e-08 | 2978 | 22 |
| enzyme_qssa | rk4 | classical | 0.00991375 | 3.24934e-11 | 992 | 66 |
| enzyme_qssa | rk38 | classical | 0.00982743 | 3.20492e-11 | 910 | 72 |
| enzyme_qssa | 11e898cb | discovered | 0.00965521 | 6.47158e-08 | 1489 | 44 |
| robertson_scaled | euler | classical | 0.194335 | 8.80946e-05 | 4369 | 15 |
| robertson_scaled | heun2 | classical | 0.154215 | 3.81289e-07 | 1680 | 39 |
| robertson_scaled | midpoint | classical | 0.0894046 | 3.16523e-07 | 1985 | 33 |
This suite is compared problem by problem and summarized by medians of those per-problem ratios and by win counts, not by a root mean square, so no problem carries a weight here and there is no share to report. The weight shares that do exist are on the cell pages, where the archive's search error and held-out error are formed.
The comparator here is fixed: the archive champion 11e898cbab59, the elite with the lowest held-out error when the suite was built, with its coefficients unchanged. It was picked at cycle 33 out of 141,364 archived records, before this suite ran. It runs against each of the 5 classical anchors on all 8 problems. Each cell gives that anchor's Q15 error and one word for the champion against it: lower, higher, tie when the two sit inside the 2 percent band, or flagged when the degeneracy filter flagged the problem over the 8 methods this suite ran.
The problems on this page are not the held-out set. Those four select the archive's elites, so a champion's held-out error carries selection bias; these select nothing, which is what makes them the harder test.
| problem | champion Q15 error | euler | heun2 | midpoint | rk4 | rk38 |
|---|---|---|---|---|---|---|
| buck_converter | 0.00198385 | 0.0253608 (lower) | 0.0434966 (lower) | 0.022902 (lower) | 0.03238 (lower) | 0.0208238 (lower) |
| battery_2rc | 0.0368753 | 0.0986711 (lower) | 0.0776365 (lower) | 0.0905505 (lower) | 0.117303 (lower) | 0.105182 (lower) |
| bicycle_lateral | 0.000539394 | 0.105013 (lower) | 0.0561341 (lower) | 0.000394556 (higher) | 0.0439865 (lower) | 0.00394005 (lower) |
| pll_lock | 0.0159381 | 0.0481865 (lower) | 0.0325573 (lower) | 0.0334146 (lower) | 0.0386035 (lower) | 0.0128489 (higher) |
| glucose_minimal | 0.0207014 | 0.0840464 (lower) | 0.0738776 (lower) | 0.0372548 (lower) | 0.0542891 (lower) | 0.0610666 (lower) |
| servo_load_step | 0.0115754 | 0.0118808 (lower) | 0.0118296 (lower) | 0.0115746 (tie) | 0.0124489 (lower) | 0.0148399 (lower) |
| enzyme_qssa | 0.00965521 | 0.00974111 (flagged) | 0.00974111 (flagged) | 0.00969807 (flagged) | 0.00991375 (flagged) | 0.00982743 (flagged) |
| robertson_scaled | no finish | 0.194335 | 0.154215 | 0.0894046 | n/a | n/a |
Of the 30 cells in this table where both methods finish and the problem is not flagged, the champion has the lower Q15 error in 27, the higher error in 2, and 1 is a tie. Against the best of the 5 anchors on each problem it is lower on 3 of 5 non-stiff problems. On the stiff problems where both sides finish it is lower on 0 of 1, with 1 tie. That is 1 discovered method against 5 classical anchors, on every row, with no maximum taken on either side.
On bicycle_lateral the champion is not the discovered method with the lowest error: 196b1d17d0b7 reaches 0.0001774 while the champion reaches 0.000539394. A row like that counts for the discovered side in the maximum-over-both view below and against the champion here, which is the whole difference between the two tallies.
| problem | finishers | verdict | why |
|---|---|---|---|
| battery_2rc | 8 | clean | the 8 finishers' Q15 errors span 218.11 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.39672, so the comparison is about the methods |
| bicycle_lateral | 8 | clean | the 8 finishers' Q15 errors span 59095.95 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.03867, so the comparison is about the methods |
| buck_converter | 8 | clean | the 8 finishers' Q15 errors span 2092.53 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.772995, so the comparison is about the methods |
| enzyme_qssa | 6 | flagged | the 6 finishers' Q15 errors span 2.68 percent from best to worst, under the 5 percent threshold, so the comparison is reporting the problem rather than the method |
| glucose_minimal | 8 | clean | the 8 finishers' Q15 errors span 305.99 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.274207, so the comparison is about the methods |
| pll_lock | 8 | clean | the 8 finishers' Q15 errors span 275.02 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.954011, so the comparison is about the methods |
| robertson_scaled | 3 | clean | the 3 finishers' Q15 errors span 117.37 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 0.708764, so the comparison is about the methods |
| servo_load_step | 6 | clean | the 6 finishers' Q15 errors span 28.21 percent from best to worst, over the 5 percent threshold, and none of them sits within 5 percent of the reference solution's norm of 1.4082, so the comparison is about the methods |
Every problem in this suite carries a verdict from the same rule, computed over the 8 methods the suite ran on each of them. The finishers column is how many of those returned a finite Q15 error, which is what the rule reads. A flagged problem stays in the tables above and out of every count, and the thresholds behind the flag are on the methodology page.
| problem | winner | best classical | Q15 error | best discovered | Q15 error | ratio |
|---|---|---|---|---|---|---|
| buck_converter | 11e898cb (discovered) | rk38 | 0.0208238 | 11e898cb | 0.00198385 | 0.0953 |
| battery_2rc | 11e898cb (discovered) | heun2 | 0.0776365 | 11e898cb | 0.0368753 | 0.475 |
| bicycle_lateral | 196b1d17 (discovered) | midpoint | 0.000394556 | 196b1d17 | 0.0001774 | 0.45 |
| pll_lock | rk38 (classical) | rk38 | 0.0128489 | 11e898cb | 0.0159381 | 1.24 |
| glucose_minimal | 11e898cb (discovered) | midpoint | 0.0372548 | 11e898cb | 0.0207014 | 0.556 |
| problem | winner | best classical | Q15 error | best discovered | Q15 error | ratio | finishers (classical / discovered) |
|---|---|---|---|---|---|---|---|
| servo_load_step | tie (inside the 2 percent band) | midpoint | 0.0115746 | 11e898cb | 0.0115754 | 1 | 5 / 1 |
| enzyme_qssa | flagged (left out of the tallies) | midpoint | 0.00969807 | 11e898cb | 0.00965521 | 0.996 | 5 / 1 |
| robertson_scaled | midpoint (classical) | midpoint | 0.0894046 | none finished | n/a | n/a | 3 / 0 |
ratio is best-discovered over best-classical Q15 error; below 1.0 the discovered method has the lower error. Each row takes a maximum over both sides, 3 discovered methods against 5 classical anchors, so the discovered column has more chances to produce the row's best number and this view reads better for it than the fixed comparison above.
Wall clock per Q15 step, next to the analytic cycle model the rest of this site ranks by. Every fixed-step row runs the same pinned Q15 path at the shared cycle budget of 65536 cycles.
Source: rk-work/benchmark/results.json, speedup.per_method_us_per_step. Python wall clock of the pinned solve_q15 path in Q15 (time.perf_counter, median of 15 repeats after 3 warmups).
| method | problem | microseconds per Q15 step | problems in the median |
|---|---|---|---|
| euler | median over its problems | 6.38 | 7 |
| euler | dahlquist | 4.37 | |
| euler | damped_osc | 7.57 | |
| euler | dc_motor | 5.73 | |
| euler | pendulum | 6.15 | |
| euler | quaternion | 8.89 | |
| euler | rc_thermal | 9.46 | |
| euler | vanderpol_mild | 6.38 | |
| heun2 | median over its problems | 13.96 | 7 |
| heun2 | dahlquist | 8.86 | |
| heun2 | damped_osc | 13.23 | |
| heun2 | dc_motor | 13.96 | |
| heun2 | pendulum | 13.42 | |
| heun2 | quaternion | 18.56 | |
| heun2 | rc_thermal | 19.01 | |
| heun2 | vanderpol_mild | 15.66 | |
| midpoint | median over its problems | 12.46 | 7 |
| midpoint | dahlquist | 8.45 | |
| midpoint | damped_osc | 11.33 | |
| midpoint | dc_motor | 11.55 | |
| midpoint | pendulum | 18.36 | |
| midpoint | quaternion | 18.77 | |
| midpoint | rc_thermal | 21.46 | |
| midpoint | vanderpol_mild | 12.46 | |
| rk4 | median over its problems | 31.99 | 7 |
| rk4 | dahlquist | 21.09 | |
| rk4 | damped_osc | 34.75 | |
| rk4 | dc_motor | 30.88 | |
| rk4 | pendulum | 29.08 | |
| rk4 | quaternion | 45.37 | |
| rk4 | rc_thermal | 38.58 | |
| rk4 | vanderpol_mild | 31.99 | |
| rk38 | median over its problems | 33.23 | 7 |
| rk38 | dahlquist | 21.11 | |
| rk38 | damped_osc | 32.29 | |
| rk38 | dc_motor | 38.23 | |
| rk38 | pendulum | 30.24 | |
| rk38 | quaternion | 57.42 | |
| rk38 | rc_thermal | 50.46 | |
| rk38 | vanderpol_mild | 33.23 | |
| 11e898cb | median over its problems | 20.47 | 7 |
| 11e898cb | dahlquist | 13.17 | |
| 11e898cb | damped_osc | 20.47 | |
| 11e898cb | dc_motor | 18.61 | |
| 11e898cb | pendulum | 21.84 | |
| 11e898cb | quaternion | 38.10 | |
| 11e898cb | rc_thermal | 32.83 | |
| 11e898cb | vanderpol_mild | 19.81 | |
| 42863b93 | median over its problems | 66.37 | 7 |
| 42863b93 | dahlquist | 35.54 | |
| 42863b93 | damped_osc | 55.93 | |
| 42863b93 | dc_motor | 66.37 | |
| 42863b93 | pendulum | 71.88 | |
| 42863b93 | quaternion | 124.06 | |
| 42863b93 | rc_thermal | 71.26 | |
| 42863b93 | vanderpol_mild | 59.33 | |
| 196b1d17 | median over its problems | 71.89 | 7 |
| 196b1d17 | dahlquist | 43.76 | |
| 196b1d17 | damped_osc | 62.60 | |
| 196b1d17 | dc_motor | 63.00 | |
| 196b1d17 | pendulum | 72.50 | |
| 196b1d17 | quaternion | 93.43 | |
| 196b1d17 | rc_thermal | 84.98 | |
| 196b1d17 | vanderpol_mild | 71.89 |
| problem | champion µs/step | rk4 µs/step | predicted ratio | measured ratio | champion Q15 error | rk4 Q15 error | lower error |
|---|---|---|---|---|---|---|---|
| dahlquist | 13.17 | 21.09 | 1.500 | 1.602 | 0.000198741 | 0.00016747 | rk4 |
| damped_osc | 20.47 | 34.75 | 1.500 | 1.698 | 0.00393948 | 0.0159248 | 11e898cb |
| vanderpol_mild | 19.81 | 31.99 | 1.500 | 1.615 | 0.0194682 | 0.100973 | 11e898cb |
| pendulum | 21.84 | 29.08 | 1.500 | 1.332 | 0.00474438 | 0.00403234 | rk4 |
| dc_motor | 18.61 | 30.88 | 1.500 | 1.660 | 0.0276476 | 0.0763589 | 11e898cb |
| rc_thermal | 32.83 | 38.58 | 1.500 | 1.175 | 0.0460875 | 0.158159 | 11e898cb |
| quaternion | 38.10 | 45.37 | 1.500 | 1.191 | 0.0191055 | 0.0540564 | 11e898cb |
Geometric mean over the problems: measured 1.452, predicted 1.500. A ratio above 1.0 means the champion needs less time per step. The measured mean is taken over the per-problem ratios in this table, and the chart's two medians are each taken over that method's own problems, so their quotient is a different number again. At the same budget the champion reaches the lower Q15 error on 5 of 7 problems, 1 discovered method against 1 classical baseline (rk4), with a median error ratio of 0.353.
Across 56 fixed-step Q15 runs the Pearson correlation between analytic cycles per step and measured seconds per step is 0.970. The analytic model counts Cortex-M0+ arithmetic while the measurement is Python interpreter time, so the correlation speaks to ordering, not to absolute scale.
This head-to-head is valid because both sides run the identical solve_q15 path in the same interpreter: same arithmetic, same per-step machinery, only the tableau differs, so the per-step ratio compares like against like. Budget wall-clock totals also carry constant Python per-step overhead, which favors methods that take fewer, larger steps inside the budget; on the Cortex-M0+ target the analytic cycle count is the time, and the predicted ratio column states what that model expects of the measured ratio beside it.
Environment: CPython 3.13.5; Windows 11 (AMD64); scipy 1.14.1; numpy 2.1.3; CPU Intel64 Family 6 Model 141 Stepping 1, GenuineIntel.
All timings are Python-level wall clock on one desktop machine: the Q15 runs pay Python interpreter overhead per primitive while scipy runs compiled internals, so timings compare like against like only within a regime and do not transfer to microcontroller cycle counts.
The cost model is analytic. It derives an instruction sequence from a tableau's sparsity pattern and each coefficient's bit pattern and prices it, and no compiler is in that loop. This section is what happened when the Q15 step it describes was compiled for cortex-m0plus with arm-none-eabi-gcc (15:13.2.rel1-2) 13.2.1 20231009 at -O2, executed under unicorn 2.1.4, and the executed instructions were priced against the Cortex-M0+ technical reference manual's cycle table.
The emulator is instruction accurate and NOT cycle accurate. It reports which instructions execute and in what order. Every cycle count here comes from applying the Cortex-M0+ TRM table to that instruction stream offline, under the assumptions below. No number here was measured on any physical part.
Correctness came before any timing number. The compiled step ran against the pinned Python evaluator over 144 cases, compared on the exact final int16 state and on every stage input, and all 144 agree. 25 of the cases push the Q15 primitives past the range they check, where they raise instead of wrapping, and the checked build traps at the same operation in all of them.
The analytic model prices the stage and b combinations only. It prices no derivative call, no h times k product, no loop control and no stack frame, so the whole-step traced number is larger by construction. Both are published: ratio is whole step, ratio_model_scope is the matched scope.
| method | stages | cycles_analytic | cycles_traced, matched scope | ratio | MULS per step | outside the band |
|---|---|---|---|---|---|---|
| euler | 1 | 5 | 5 | 1.000 | 1 | no |
| midpoint | 2 | 11 | 14 | 1.273 | 2 | no |
| heun2 | 2 | 13 | 17 | 1.308 | 2 | no |
| ralston2 | 2 | 16 | 27 | 1.688 | 2 | yes, gap 0.407 |
| 11e898cb | 3 | 22 | 35 | 1.591 | 3 | yes, gap 0.371 |
| heun3 | 3 | 23 | 42 | 1.826 | 3 | yes, gap 0.452 |
| kutta3 | 3 | 26 | 49 | 1.885 | 3 | yes, gap 0.469 |
| rk4 | 4 | 33 | 67 | 2.030 | 4 | yes, gap 0.507 |
| rk38 | 4 | 36 | 59 | 1.639 | 4 | yes, gap 0.390 |
Both columns are at n_states = 1 under m0plus_fast. The traced column is restricted to what cycle_count prices, which is the stage and b combinations: it excludes the derivative call, the h times k product, loop control and the stack frame. The whole-step figures, and the same counts at two, three and four states, are in the fold below and do not compare with cycles_analytic.
6 of 9 methods sit outside the 0.25 band this comparison chose for itself, measured as |cycles_analytic minus cycles_traced| over cycles_traced at matched scope: ralston2 at 0.407, 11e898cb at 0.371, heun3 at 0.452, kutta3 at 0.469, rk4 at 0.507, rk38 at 0.390. That is published rather than tuned, and the cause is in the next paragraph.
Ranking the 9 methods in this table by cost, the analytic order and the traced order agree to a Spearman correlation of 0.9833 under m0plus_fast. One pair comes out the other way round: the model prices rk4 at 33 cycles against 36 for rk38, while the trace gives rk38 59 and rk4 67. Under the small-multiplier model the two orders do not match: 2 pairs rank differently (ralston2 against 11e898cb and kutta3 against rk38), at a Spearman correlation of 0.9667.
Two results, and the load-bearing one is good news. Every MULS in every trace is the h times k product, one per stage, and not one of them applies a tableau coefficient: the compiler turned the coefficients, dyadic and otherwise, into shifts and adds. The other result is the gap above. Because the analytic model charges each coefficient the cheaper of a shift-add chain and a hardware multiply, and a multiply costs one cycle on the fast multiplier variant, it prices a multiply the compiler does not emit in these nine traces, so it counts low for coefficient-heavy tableaus. The widest gap above is rk4 at 0.507.
For the champion tableau (11e898cb) the trace contains 3 MULS per step, one per stage at 3 stages, and 0 of them apply a tableau coefficient. The dyadic-coefficient assumption behind its cost holds, and holds more strongly than it was stated.
Correcting the cost model would change every archived cycle count, so it moves VERIFIER_HASH and invalidates all scores in the archive. That is an epoch decision for the run's owner, not a fix inside a cycle, and this page publishes the disagreement in the meantime.
State dimension is where the two prices part company. Every count in this paragraph and in the ladder below is one whole rk_step with the derivative routine's own body excluded, priced from the same instruction-accurate emulator. The analytic model charges a fixed amount per state, so its count at four states is four times its count at one for every method above. The compiled step does not keep that shape: at four states it costs 4.03 to 6.04 times its own one-state cost, and a straight line through the one-state and two-state traced costs, read at four states, underprices the measured four-state cost by 1.46x for rk4 up to 2.43x for midpoint. That is why nothing here is priced at a state count the tracer did not run.
| method | instructions per step | cycles traced, whole step | cycles traced, matched scope | cycles_analytic, small multiplier | ratio at matched scope, small multiplier |
|---|---|---|---|---|---|
| euler | 16 | 32 | 5 | 5 | 1.000 |
| midpoint | 30 | 50 | 14 | 11 | 1.273 |
| heun2 | 33 | 55 | 17 | 13 | 1.308 |
| ralston2 | 44 | 66 | 27 | 30 | 0.900 |
| 11e898cb | 58 | 86 | 35 | 29 | 1.207 |
| heun3 | 65 | 93 | 42 | 50 | 0.840 |
| kutta3 | 72 | 102 | 49 | 65 | 0.754 |
| rk4 | 99 | 139 | 67 | 85 | 0.788 |
| rk38 | 90 | 130 | 59 | 64 | 0.922 |
| method | 1 state | 2 states | 3 states | 4 states |
|---|---|---|---|---|
| euler | 32 | 41 | 52 | 129 |
| midpoint | 50 | 74 | 144 | 296 |
| heun2 | 55 | 92 | 160 | 305 |
| ralston2 | 66 | 116 | 264 | 362 |
| 11e898cb | 86 | 144 | 297 | 490 |
| heun3 | 93 | 164 | 363 | 543 |
| kutta3 | 102 | 187 | 419 | 582 |
| rk4 | 139 | 256 | 478 | 716 |
| rk38 | 130 | 235 | 532 | 785 |
Cycles for one whole rk_step under m0plus_fast, at each state count the seven scored problems use. The derivative routine's own body is excluded, and the emulator is instruction accurate rather than cycle accurate, so every entry is a priced instruction stream and not a measurement on any part. These counts do not compare with cycles_analytic, which prices a narrower scope. What they are is the denominator a cycle budget is divided by to get a step count.
Source: rk-work/trace/results.json, written on the host by rk_harness.tracecheck and pinned by TRACE_HASH 61f3a1bc. Arithmetic: Q15 with floor rounding, the same pinned step the archive scores.
Before any search ran, the project tested its own premise on two fixed classical methods, heun2 and rk4: is coefficient arithmetic a real share of step cost, and does Q15 roundoff matter at practical step sizes? Code computed the verdict from the measurements: mixed.
Left of a crossover step size, floor rounding loses more per extra step than the smaller step recovers, so the Q15 line turns back up while float64 keeps falling. Each chart carries its step sizes and both errors in the table beneath it.
Source: rk-work/falsification.json; Q15 and float64 as labeled.
| step size h | steps | Q15 error | float64 error |
|---|---|---|---|
| 0.625 | 64 | 0.0466968 | 0.0467395 |
| 0.3125 | 128 | 0.0127543 | 0.0123373 |
| 0.15625 | 256 | 0.00400537 | 0.00312695 |
| 0.078125 | 512 | 0.00297096 | 0.000786061 |
| 0.0390625 | 1024 | 0.00642073 | 0.00019668 |
| 0.0195312 | 2048 | 0.0126341 | 4.91511e-05 |
| 0.00976562 | 4096 | 0.0276052 | 1.22823e-05 |
Source: rk-work/falsification.json; Q15 and float64 as labeled.
| step size h | steps | Q15 error | float64 error |
|---|---|---|---|
| 2.5 | 16 | inf | 0.0176384 |
| 1.25 | 32 | inf | 0.0124722 |
| 0.625 | 64 | 0.00133582 | 0.00101889 |
| 0.3125 | 128 | 0.00112763 | 6.29326e-05 |
| 0.15625 | 256 | 0.00253715 | 3.88382e-06 |
| 0.078125 | 512 | 0.00479333 | 2.40922e-07 |
| 0.0390625 | 1024 | 0.0156265 | 1.49965e-08 |
| 0.0195312 | 2048 | 0.0398531 | 9.35305e-10 |
| 0.00976562 | 4096 | 0.0258308 | 5.83937e-11 |
| method | coefficient share, m0plus_fast | coefficient share, m0plus_slow | crossover h | crossover in the practical range |
|---|---|---|---|---|
| heun2 | 0.5 | 0.147727 | 0.0390625 | yes |
| rk4 | 0.559322 | 0.361702 | 0.15625 | yes |
The protocol fixes two classical methods, rk4 and heun2, on the damped oscillator and measures two things. One is the share of a step's cycle count spent on coefficient arithmetic rather than derivative evaluation, under both primary cost models. The other is a sweep over n = 8, 16, ..., 4096 steps that compares the Q15 integrator with float64 on identical steps, looking for the step size where roundoff overtakes truncation error.
The verdict is mechanical. proceed needs every coefficient share at 0.30 or more and at least one crossover inside the practical range 1e-3 ≤ h ≤ 1.0. kill needs every share below 0.15 and no practical crossover. Anything else reads mixed. A proceed verdict says that searching over coefficients can plausibly matter; it does not show that any searched method is good.
On the 5 non-stiff practical problems at a 65536-cycle budget under m0plus_fast with floor rounding, the best discovered method has lower Q15 error than the best classical anchor on 4 of 5; the median ratio of best-discovered to best-classical Q15 error is 0.475 (below 1.0 favors the discovered methods). On the 3 moderately stiff problems (stiffness ratios 292 to 1030): where both sides finish, the best discovered method is ahead on 1 of 2 with a median error ratio of 0.998; on 1 of 3 no discovered method finishes at all; every one overflows in Q15 while cheap low-stage classical methods complete the run. Explicit methods pay for stability: at a fixed cycle budget an expensive high-order tableau takes larger steps, and once the step size times the fast eigenvalue leaves the stability interval the Q15 run overflows, whoever found the tableau. This is why the implicit (SDIRK) class is measured.
That paragraph is quoted from validation/results.json, which counted its wins and losses before the tie band and the degeneracy filter existed and over a maximum of 3 discovered methods against a maximum of 5 classical anchors. Where it differs from the tallies above, the tallies above are this page applying both filters; rewriting the stored sentence needs a host validation run, not a site build.
Float64 runs of the same tableaus over the same steps land between 3.64986e-14 and 0.0106122, and the largest raw Q15 magnitude seen anywhere is 20130 of the int16 limit 32767. Where a row's float64 error sits far below its Q15 error, the Q15 number measures quantization, not truncation.
The cycles/step column is one method's analytic cost for one step of that problem, so a single method shows more than one number here: the cost model prices a step at one state and multiplies by the problem's state count. The champion costs 22 cycles per step at one state, which is the figure the archive and the frontier chart carry, and 44 on the 2-state problems and 66 on the 3-state problems.
| problem | method | kind | steps | cycles/step | Q15 error | float64 error | max |q| | note |
|---|---|---|---|---|---|---|---|---|
| buck_converter | euler | classical | 6553 | 10 | 0.0253608 | 7.5204e-05 | 4743 | |
| battery_2rc | euler | classical | 4369 | 15 | 0.0986711 | 4.57811e-05 | 19616 | |
| bicycle_lateral | euler | classical | 6553 | 10 | 0.105013 | 5.45857e-07 | 8221 | |
| pll_lock | euler | classical | 6553 | 10 | 0.0481865 | 6.31115e-06 | 7845 | |
| glucose_minimal | euler | classical | 6553 | 10 | 0.0840464 | 3.9061e-05 | 11428 | |
| servo_load_step | euler | classical | 6553 | 10 | 0.0118808 | 2.3216e-05 | 13977 | |
| enzyme_qssa | euler | classical | 6553 | 10 | 0.00974111 | 1.89045e-05 | 16384 | |
| robertson_scaled | euler | classical | 4369 | 15 | 0.194335 | 8.80946e-05 | 16384 | |
| buck_converter | heun2 | classical | 2520 | 26 | 0.0434966 | 4.99565e-07 | 4777 | |
| battery_2rc | heun2 | classical | 1680 | 39 | 0.0776365 | 7.68948e-08 | 19875 | |
| bicycle_lateral | heun2 | classical | 2520 | 26 | 0.0561341 | 1.42198e-09 | 8663 | |
| pll_lock | heun2 | classical | 2520 | 26 | 0.0325573 | 3.90381e-08 | 7955 | |
| glucose_minimal | heun2 | classical | 2520 | 26 | 0.0738776 | 2.96109e-08 | 11428 | |
| servo_load_step | heun2 | classical | 2520 | 26 | 0.0118296 | 3.67262e-08 | 13978 | |
| enzyme_qssa | heun2 | classical | 2520 | 26 | 0.00974111 | 3.42268e-08 | 16384 | |
| robertson_scaled | heun2 | classical | 1680 | 39 | 0.154215 | 3.81289e-07 | 16384 | |
| buck_converter | midpoint | classical | 2978 | 22 | 0.022902 | 3.57687e-07 | 4840 | |
| battery_2rc | midpoint | classical | 1985 | 33 | 0.0905505 | 5.50663e-08 | 19967 | |
| bicycle_lateral | midpoint | classical | 2978 | 22 | 0.000394556 | 1.01808e-09 | 9168 | |
| pll_lock | midpoint | classical | 2978 | 22 | 0.0334146 | 2.80062e-08 | 8217 | |
| glucose_minimal | midpoint | classical | 2978 | 22 | 0.0372548 | 4.36692e-08 | 11428 | |
| servo_load_step | midpoint | classical | 2978 | 22 | 0.0115746 | 2.62928e-08 | 13975 | |
| enzyme_qssa | midpoint | classical | 2978 | 22 | 0.00969807 | 2.09683e-08 | 16384 | |
| robertson_scaled | midpoint | classical | 1985 | 33 | 0.0894046 | 3.16523e-07 | 16384 | |
| buck_converter | rk4 | classical | 992 | 66 | 0.03238 | 1.14781e-10 | 4776 | |
| battery_2rc | rk4 | classical | 661 | 99 | 0.117303 | 7.60152e-13 | 19681 | |
| bicycle_lateral | rk4 | classical | 992 | 66 | 0.0439865 | 3.64986e-14 | 8773 | |
| pll_lock | rk4 | classical | 992 | 66 | 0.0386035 | 7.64901e-13 | 7914 | |
| glucose_minimal | rk4 | classical | 992 | 66 | 0.0542891 | 1.23695e-13 | 11428 | |
| servo_load_step | rk4 | classical | 992 | 66 | 0.0124489 | 2.62516e-13 | 14013 | |
| enzyme_qssa | rk4 | classical | 992 | 66 | 0.00991375 | 3.24934e-11 | 16384 | |
| robertson_scaled | rk4 | classical | 661 | 99 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| buck_converter | rk38 | classical | 910 | 72 | 0.0208238 | 1.62089e-10 | 4810 | |
| battery_2rc | rk38 | classical | 606 | 108 | 0.105182 | 1.07549e-12 | 19860 | |
| bicycle_lateral | rk38 | classical | 910 | 72 | 0.00394005 | 5.27405e-14 | 9135 | |
| pll_lock | rk38 | classical | 910 | 72 | 0.0128489 | 1.08125e-12 | 8093 | |
| glucose_minimal | rk38 | classical | 910 | 72 | 0.0610666 | 2.42427e-13 | 11428 | |
| servo_load_step | rk38 | classical | 910 | 72 | 0.0148399 | 3.68864e-13 | 14184 | |
| enzyme_qssa | rk38 | classical | 910 | 72 | 0.00982743 | 3.20492e-11 | 16384 | |
| robertson_scaled | rk38 | classical | 606 | 108 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| buck_converter | 11e898cb | discovered | 1489 | 44 | 0.00198385 | 9.69071e-07 | 4889 | |
| battery_2rc | 11e898cb | discovered | 992 | 66 | 0.0368753 | 1.49545e-07 | 20130 | |
| bicycle_lateral | 11e898cb | discovered | 1489 | 44 | 0.000539394 | 2.75832e-09 | 9168 | |
| pll_lock | 11e898cb | discovered | 1489 | 44 | 0.0159381 | 7.60478e-08 | 8396 | |
| glucose_minimal | 11e898cb | discovered | 1489 | 44 | 0.0207014 | 5.20727e-08 | 11428 | |
| servo_load_step | 11e898cb | discovered | 1489 | 44 | 0.0115754 | 7.12907e-08 | 13976 | |
| enzyme_qssa | 11e898cb | discovered | 1489 | 44 | 0.00965521 | 6.47158e-08 | 16384 | |
| robertson_scaled | 11e898cb | discovered | 992 | 66 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| buck_converter | 42863b93 | discovered | 436 | 150 | 0.0147768 | 3.04888e-07 | 4879 | |
| battery_2rc | 42863b93 | discovered | 291 | 225 | 0.056326 | 8.3658e-09 | 20038 | |
| bicycle_lateral | 42863b93 | discovered | 436 | 150 | 0.0459941 | 1.05546e-10 | 8757 | |
| pll_lock | 42863b93 | discovered | 436 | 150 | 0.0244659 | 7.98545e-09 | 8291 | |
| glucose_minimal | 42863b93 | discovered | 436 | 150 | 0.0280853 | 2.35809e-09 | 11428 | |
| servo_load_step | 42863b93 | discovered | 436 | 150 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| enzyme_qssa | 42863b93 | discovered | 436 | 150 | n/a | 0.0106122 | n/a | q15 run failed: Q15OverflowError |
| robertson_scaled | 42863b93 | discovered | 291 | 225 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| buck_converter | 196b1d17 | discovered | 520 | 126 | 0.00432324 | 1.52023e-09 | 4860 | |
| battery_2rc | 196b1d17 | discovered | 346 | 189 | 0.0442465 | 1.01584e-11 | 20045 | |
| bicycle_lateral | 196b1d17 | discovered | 520 | 126 | 0.0001774 | 5.01308e-13 | 9169 | |
| pll_lock | 196b1d17 | discovered | 520 | 126 | 0.0243841 | 1.00502e-11 | 8291 | |
| glucose_minimal | 196b1d17 | discovered | 520 | 126 | 0.0279905 | 2.04531e-12 | 11428 | |
| servo_load_step | 196b1d17 | discovered | 520 | 126 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| enzyme_qssa | 196b1d17 | discovered | 520 | 126 | n/a | 7.18161e-09 | n/a | q15 run failed: Q15OverflowError |
| robertson_scaled | 196b1d17 | discovered | 346 | 189 | n/a | n/a | n/a | q15 run failed: Q15OverflowError |
| kind | name / tableau_hash | order | stages | roles | archive provenance |
|---|---|---|---|---|---|
| classical | euler | 1 | 1 | anchor | classical fixture |
| classical | heun2 | 2 | 2 | anchor | classical fixture |
| classical | midpoint | 2 | 2 | anchor | classical fixture |
| classical | rk4 | 4 | 4 | anchor | classical fixture |
| classical | rk38 | 4 | 4 | anchor | classical fixture |
| discovered | 11e898cbab591a7f432670d4040b788f4c00cc646a005d4f967505578e2c0033 | 2 | 3 | champion, best_elite_order_2 | cycle 33, unreplicated, held-out 0.0286181 |
| discovered | 42863b93bf7d570f7f6fd94033452451bf70bb8adf803e7f2c965f612914472e | 3 | 6 | best_elite_order_3 | cycle 632, heldout_verified, held-out 0.0376551 |
| discovered | 196b1d17d0b708c06c798239ed34bd466fa7e9b6ba7e59797ec5d22645b2ab03 | 4 | 6 | best_elite_order_4 | cycle 358, unreplicated, held-out 0.0603389 |
power electronics; oscillatory, 2 states, integrated to t = 25.
Non-stiff; stiffness ratio 1 (single complex pole pair; both eigenvalues share Re = -1/(2*Qf)).
x1' = D - x2; x2' = x1 - x2/Qf (D = 5/12, Qf = 2)
Reference: closed form via matrix exponential of the augmented affine system.
Source: Averaged CCM buck converter model, R. W. Erickson and D. Maksimovic, Fundamentals of Power Electronics, 2nd ed., Kluwer, 2001; the same averaged equations appear in arXiv:2002.02544. Per-unit normalization with Vg = 12 V, D = 5/12, L = 100 uH, C = 100 uF, R = 2 ohm (Z0 = 1 ohm, Qf = 2, omega0 = 1e4 rad/s). Startup transient from rest.
battery management; linear, 3 states, integrated to t = 6.
Non-stiff; stiffness ratio 6.8 (tau2 / tau1 = 222.9 s / 32.8 s for the two RC branches).
dod' = T*I/(3600*Cn); vk' = T*(-vk/(Rk*Ck) + I/Ck), k = 1, 2
Reference: exact closed form: linear ramp plus two RC charge curves.
Source: Li-ion Thevenin 2RC equivalent circuit, M. Chen and G. A. Rincon-Mora, IEEE Trans. Energy Conversion 21(2):504-511, 2006, parameters for the 850 mAh TCL PL-383562 cell at SOC 0.5: R1 = 0.0467 ohm, C1 = 703 F, R2 = 0.0498 ohm, C2 = 4475 F. Constant 1.7 A (2C) discharge pulse is our scenario choice. States (dod, v1, v2), time unit 20 s.
vehicle dynamics / ADAS; linear, 2 states, integrated to t = 15.
Non-stiff; stiffness ratio 1 (single complex pole pair (sideslip and yaw share their real part)).
v' = T*(-u*w + (Fy1+Fy2)/m); w' = T*(lf*Fy1 - lr*Fy2)/Iz; Fy1 = kf*((v+lf*w)/u - delta), Fy2 = kr*(v-lr*w)/u
Reference: closed form via matrix exponential of the augmented affine system.
Source: Linear-tire dynamic single-track model, Q. Ge, S. E. Li, Q. Sun, S. Zheng, arXiv:2011.09612 (2020), eqs. (1) and (3), model attributed to R. Rajamani, Vehicle Dynamics and Control, Springer 2011; Table I parameters (CarSim C-Class Hatchback 2017): m = 1412 kg, Iz = 1536.7 kg m^2, lf = 1.06 m, lr = 1.85 m, kf = -128916 N/rad, kr = -85944 N/rad. Constant 30 m/s, step steer 0.01 rad. States (v, w), time unit 0.1 s.
communications / clocking; nonlinear, 2 states, integrated to t = 10.
Non-stiff; stiffness ratio 1 (complex pole pair near the lock point (zeta = 0.7)).
theta_e' = w_free - Kvco*(x + (tau2/tau1)*sin(theta_e)); x' = sin(theta_e)/tau1
Reference: mpmath odefun at 30 digits (no closed form).
Source: Second-order type-2 analog PLL with active PI filter and sinusoidal phase detector, N. V. Kuznetsov, G. A. Leonov, M. V. Yuldashev, R. V. Yuldashev, arXiv:1705.05013, model (15). Kvco = 1, tau1 = 1, tau2 = 1.4 (wn = 1 rad/s, zeta = 0.7), frequency step w_free = 0.5 rad/s inside the lock-in range (no cycle slip). States (theta_e, x).
biomedical devices; nonlinear, 2 states, integrated to t = 12.
Non-stiff; stiffness ratio 1.5 (ratio of the two decay rates, 10*(0.01*y2 + p1) over 10*p2, at the y2 peak).
y1' = 10*(-0.01*y1*y2 + 0.0265*(0.82 - y1)); y2' = 10*(-0.0254*y2 + 0.1282*exp(-0.5*t))
Reference: mpmath odefun at 30 digits (no closed form).
Source: Bergman glucose minimal model (MINMOD) for an IVGTT, Pacini and Bergman, Comput. Methods Programs Biomed. 1986, via the MLAB worked example (civilized.com/mlabexamples/glucose.htmld/): p1 = 0.0265, p2 = 0.0254, p3 = 1.282e-5, Gb = 82 mg/dl, Ib = 8 uU/ml, G0 = 279 mg/dl. The insulin input is idealized as I(t) = Ib + 100*exp(-0.05*t_min) uU/ml (our modeling choice; the source drives it from measured data). Normalized y1 = G/100, y2 = 100*X, time unit 10 min. Only time-dependent RHS in the suite.
motor control; linear, 2 states, integrated to t = 260.
Stiff; stiffness ratio 546.4 (electrical pole 9.649/ms over electromechanical pole 0.01766/ms).
di' = (-R*di - ke*dw)/L; dw' = (kt*di - dT_L)/J (deviations; per-ms coefficients -9.667, -9.75, 0.017476, -0.0074683)
Reference: closed form via matrix exponential of the augmented affine system.
Source: Armature-controlled DC servo, the standard two-state model (electrical and mechanical equations coupled by the back-EMF and torque constants; see e.g. Franklin, Powell and Emami-Naeini, Feedback Control of Dynamic Systems, ch. 2). Motor parameters from the maxon RE 25 catalog page, part 118752 (24 V, graphite brushes, 20 W): R = 2.32 ohm, L = 0.24 mH, kt = ke = 23.4 mNm/A, rotor inertia 10.3 g cm^2. Our scenario choice: a 12x reflected load inertia (total J = 13.39e-6 kg m^2) and a 10 mNm load-torque step from steady spin. States are deviations from the pre-step equilibrium, (delta_i / 1 A, delta_omega / 100 rad/s).
biochemical kinetics; nonlinear, 2 states, integrated to t = 12.
Stiff; stiffness ratio 1030 (Jacobian eigenvalue ratio 128.4 over 0.1246 at t = 0; the separation narrows to about 130 as the substrate empties).
u' = -u + (u + 1/2)*v; v' = 64*(u - (u + 1)*v)
Reference: mpmath odefun at 30 digits (no closed form).
Source: Michaelis-Menten enzyme kinetics in the standard dimensionless form of the quasi-steady-state literature: Segel and Slemrod, SIAM Review 31(3):446-477, 1989; Murray, Mathematical Biology I, 3rd ed., Springer 2002, ch. 6. u is substrate, v the enzyme-substrate complex scaled by e0. Dimensionless parameters K = 1, lambda = 1/2, eps = e0/s0 = 1/64 are our scenario choice, placing the problem in the moderately stiff regime. The run starts on the QSS manifold v(0) = u(0)/(u(0)+K), the state reached after the fast initial burst.
chemical kinetics; nonlinear, 3 states, integrated to t = 200.
Stiff; stiffness ratio 292 (largest fast eigenvalue over the window (11.7, from the y2 equation) over the slow consumption rate a = 0.04; the third eigenvalue is 0 because y1 + y2/10 + y3 is conserved).
y1' = -a*y1 + (b/10)*y2*y3; y2' = 10*a*y1 - b*y2*y3 - (c/10)*y2^2; y3' = (c/100)*y2^2 (y2 stored 10x physical; y1 + y2/10 + y3 conserved)
Reference: mpmath odefun at 30 digits (no closed form).
Source: Reaction structure of the Robertson problem: H. H. Robertson, The solution of a set of reaction rate equations, in J. Walsh (ed.), Numerical Analysis: An Introduction, Academic Press, 1966, pp. 178-182 (original rates 0.04, 1e4, 3e7). Our scaling choices: rate constants cut to (a, b, c) = (0.04, 20, 250) and the intermediate stored as y2 = 10x its physical concentration, so the quasi-steady intermediate stays resolvable in Q15 while a clear fast/slow separation remains.