The 2025 ML project

Novel-Numerical-Integration-Methods was a 2025 attempt to find new explicit Runge-Kutta tables with machine learning: a neural generator, a surrogate model and an evolutionary search, scored on generated ODEs. It built an evaluation harness that runs end to end for explicit tables, and its benchmark of 12 methods on 10,000 ODEs ran to completion. Its main conclusions did not hold. The evolutionary search saved the textbook tables it was seeded with, because a clipped score left every candidate tied with the seed, and the neural generator was never trained.

What it set out to do

The plan was to generate Butcher tables for explicit Runge-Kutta methods with a neural network, refine them with a surrogate model and an evolutionary search, and score them on generated ordinary differential equations against four textbook baselines: RK4, Dormand-Prince and Gauss-Legendre 2 and 3. Trials weighted accuracy, efficiency and stability differently, and some added a reward for tables unlike RK4.

The repository has 22 commits. The earliest is dated 2025-08-17, the work itself ran from 2025-09-13 to 2025-09-21, and the last commit, on 2025-12-02, touches only comments and docstrings.

What it built

The evaluation harness runs end to end for explicit tables. It has a generic Butcher-table stepper for explicit methods, a SciPy reference solver, parallel scoring, JSON checkpoints and a training generator with 11 ODE families, including stiff ones such as Robertson, Oregonator and Van der Pol. The reference solver covers 5 of those 11 families, Van der Pol among them but not Robertson or Oregonator. An SQLite results module is in the tree, but nothing in the pipeline calls it.

The final test set came from a separate generator with 6 families: linear_system, van_der_pol, lotka_volterra, brusselator, polynomial and robertson. The reference solver has a branch for 5 of them; robertson is the one it lacks.

The project set up 16 trial configurations, and each saved table records the settings its trial ran with. The whole src/ tree, which holds the generator, the training scripts and the analysis code as well as the harness, has 10,279 lines of Python in 47 files.

The benchmark

The final evaluation ran 12 methods, the four baselines and eight trial tables, over 10,000 generated ODEs in float64, with a fixed step h = 0.01 on t in [0, 1]. Each ODE's reference solution came from Dormand-Prince (SciPy RK45), an adaptive solver. Of the 10,000 ODEs, 8,113 could be scored, the same 8,113 for every method.

For each scorable ODE the benchmark takes the largest absolute error against the reference, then averages those maxima. The lowest mean error, 0.0791, belongs to the Dormand-Prince table and to trial 13, which saved that same table. The next, 0.14, belongs to RK4 and to trials 12, 14 and 15, which saved RK4.

The ranking comes with limits. A few outlier ODEs dominate a mean of per-ODE maxima. The trial entries were copies of RK4 and Dormand-Prince or random tables, and the Gauss-Legendre entries ran through a stepper that ignores the implicit part of a table. Because the reference solutions came from an adaptive Dormand-Prince solver, the Dormand-Prince entry is measured against a reference built on its own tableau. In effect the benchmark ranks two textbook methods against copies of themselves, random tables and truncated methods.

Mean of per-ODE max error for each benchmark entryDot plot on a log scale with one mark for each of 12 benchmark entries, baselines and trial entries as two series. The lowest mean error is 0.0791. Gauss-Legendre 3 is marked off scale at 8.31e+211 because that number comes from a stepper bug.BaselinesTrial entries0.11101001,00010,000off scaleMean of per-ODE max error, float64 (log scale)RK4RK4: 0.14Dormand-Prince (RK45)Dormand-Prince (RK45): 0.0791Gauss-Legendre 2Gauss-Legendre 2: 0.33Gauss-Legendre 3Gauss-Legendre 3: 8.31e+211, an artifact of a stepper bug, drawn off scale8.31e+211: artifact, off scaleTrial 8Trial 8: 0.377Trial 9Trial 9: 0.377Trial 10Trial 10: 0.374Trial 12Trial 12: 0.14Trial 13Trial 13: 0.0791Trial 14Trial 14: 0.14Trial 15Trial 15: 0.14Trial 16Trial 16: 5,577
Mean of per-ODE max error for each of the 12 benchmark entries, float64, fixed step h = 0.01, over 8,113 scorable ODEs of 10,000, log scale. Gauss-Legendre 3 is drawn off scale: its 8.31e+211 is an artifact of a stepper bug, not a measurement of the method. Source: results/real_trial_results.csv at d89c2c809ee0.
Data table
EntryKindStagesMean of per-ODE max errorRuntime (s)Success rateNote
RK4baseline40.1415581.1%
Dormand-Prince (RK45)baseline70.079140381.1%
Gauss-Legendre 2baseline20.3370.481.1%ran as a truncated explicit method
Gauss-Legendre 3baseline38.31e+21111181.1%artifact of a stepper bug, drawn off scale
Trial 8trial40.37715381.1%same A and b as trials 1 to 4, 6 and 9
Trial 9trial40.37715181.1%same A and b as trials 1 to 4, 6 and 8
Trial 10trial70.37436781.1%
Trial 12trial40.1414681.1%same table as RK4
Trial 13trial70.079136481.1%same table as Dormand-Prince
Trial 14trial40.1414681.1%same table as RK4
Trial 15trial40.1414581.1%same table as RK4
Trial 16trial45,57715081.1%the top scorer among random draws

Runtime is wall time for all 10,000 ODEs with the reference solves included, so it measures more than the stepper. RK4 and trials 12, 14 and 15 ran the same table, yet their runtimes span 145 to 155 seconds, so differences of that size are timing noise. Gauss-Legendre 2 had the shortest runtime, 70.4 seconds, because it ran as a truncated explicit method with 2 stages.

Benchmark runtime for each entryHorizontal bar chart of runtime in seconds for 12 benchmark entries, baselines and trial entries as two series. The shortest is Gauss-Legendre 2 at 70.4 s and the longest is Dormand-Prince (RK45) at 403 s.BaselinesTrial entries0100200300400500Wall time for all 10,000 ODEs, secondsRK4RK4: 155 sDormand-Prince (RK45)Dormand-Prince (RK45): 403 sGauss-Legendre 2Gauss-Legendre 2: 70.4 sGauss-Legendre 3Gauss-Legendre 3: 111 sTrial 8Trial 8: 153 sTrial 9Trial 9: 151 sTrial 10Trial 10: 367 sTrial 12Trial 12: 146 sTrial 13Trial 13: 364 sTrial 14Trial 14: 146 sTrial 15Trial 15: 145 sTrial 16Trial 16: 150 s
Wall time in seconds for all 10,000 ODEs, reference solves included, linear scale. Source: results/real_trial_results.csv at d89c2c809ee0.
Data table
EntryStagesRuntime (s)
RK44155
Dormand-Prince (RK45)7403
Gauss-Legendre 2270.4
Gauss-Legendre 33111
Trial 84153
Trial 94151
Trial 107367
Trial 124146
Trial 137364
Trial 144146
Trial 154145
Trial 164150

What did not hold

The code and the committed results contradict the main conclusions in the README and in results/analysis_report.md. The saved table never moved off the seed the evolutionary search started from, the generator that proposed tables was never trained, and nothing in the project could show that a table is the best possible. Two reported numbers, the Gauss-Legendre 3 error and the shared success rate, describe the harness rather than the methods. Trial 16's large error came from a random table, not from forcing variety, and no training set came near 10,000 ODEs. The project also had no working order check and no automated test suite.

The saved table never left the seed

The project claimed:

Evolutionary search, by contrast, independently rediscovered RK4 and Dormand-Prince with exact performance match (README).

RK4 was placed in the starting population of the 4-stage runs and Dormand-Prince in that of the 7-stage run. The genetic operators did run and the population moved off the seed, but the composite score is clipped at 1: every one of the 10,000 rows of the one training log scores 1, and the checkpoints of trials 12 to 15 record a best and a mean score of 1 in every one of their 100 epochs. The best-table tracker replaces the saved table only on a strictly higher score, so the seed stayed. Trials 12, 14 and 15 saved RK4 exactly and trial 13 saved Dormand-Prince exactly.

Those checkpoints are in trials/trial_01{2,3,4,5}_*/checkpoints/checkpoint_epoch_100.json at d89c2c809ee0. Matching tables the search was seeded with is not a blind result, and neither project ran a blind test.

The score could not tell RK4 from a random table

The project claimed:

validating them as true mathematical optima (README); RK4 and Dormand-Prince are not just historically successful methods, but mathematically optimal solutions (results/analysis_report.md).

Optimality would need a stated objective and a proof or a search that could have found something better. Here the score could not tell RK4 from a random table: trial 8's saved file gives its seeded random table a score ratio of 1 to RK4 at 69.4 times RK4's error on the same sample.

Each trial's saved file compares its table with RK4 on one sample, and trial 8 is not alone. Trials 9 and 10 also score 1 against RK4, at 34.2 and 2,039 times RK4's error, and trial 13, which saved Dormand-Prince, scores 1 at 0.172 times RK4's error. Tables with far higher error than RK4, and one with lower error, all scored the same as RK4.

Every candidate in the training log has the same composite score as well.

The generator was never trained

The project claimed:

Empirical benchmarks show gradient descent consistently collapsed into local minima, producing near-identical low-accuracy schemes (README).

The generator that proposes tables was never trained: its optimizer is created and never stepped, its outputs never pass validation, and every empty slot is filled by a random generator that resets the seed to 42. The only gradient descent in the project trained a surrogate model, and nothing in the training loop reads its predictions. Trials 1-4, 6, 8 and 9 saved the same A matrix and weights b (trials 8 and 9 carry a different node vector c, which none of the 6 test families depends on). Trials 8 and 9 score the same because they are the same table.

The surrogate did train. The checkpoints of trials 12 to 15 show it: each records a surrogate loss in 98 of its 100 epochs (trials/trial_01{2,3,4,5}_*/checkpoints/checkpoint_epoch_100.json at d89c2c809ee0).

What each trial saved: its training set size, the largest absolute difference of its A and b from the textbook table with the same stage count (n/a where there is none), and the trials that saved the same A and b
TrialStagesTraining ODEsLargest difference from RK4Largest difference from Dormand-PrinceSame A and b as
145001.69n/atrials 2 to 4, 6, 8 and 9
241,0001.69n/atrials 1, 3, 4, 6, 8 and 9
341,2001.69n/atrials 1, 2, 4, 6, 8 and 9
441,2001.69n/atrials 1 to 3, 6, 8 and 9
56800n/an/atrial 7
646001.69n/atrials 1 to 4, 8 and 9
76700n/an/atrial 5
841,0001.69n/atrials 1 to 4, 6 and 9
941,0001.69n/atrials 1 to 4, 6 and 8
1071,000n/a12.3none
11no table saved; not in the final evaluation
1241,0000n/aRK4; trials 14 and 15
1371,000n/a0Dormand-Prince
1441,0000n/aRK4; trials 12 and 15
1541,0000n/aRK4; trials 12 and 14
1641,0001.51n/anone

The Gauss-Legendre 3 error measures a stepper bug

The project reported:

Gauss-Legendre 3: Numerical instability (8.31e+211 max error) (results/analysis_report.md).

The stepper reads the diagonal only while that stage is still zero and never reads the upper triangle, so an implicit table runs as its strictly lower part. Both Gauss-Legendre entries ran as truncated explicit methods of order one; the number measures that bug, not the method.

The success rate belongs to the ODE set

The project reported:

Success Rate 81.13% for every method (results/analysis_report.md).

The rate is a property of the ODE set, not of the methods: the reference solver supports 5 of the 6 families in the final test (Robertson is missing), and some polynomial ODEs blow up before the end of the interval. It is the same number for every method for that reason.

Every benchmark row records the same success rate, 81.13%: 8,113 of 10,000 ODEs.

Trial 16 is the top scorer among random draws

The project claimed:

Novelty reward successfully created different method but at severe accuracy cost (results/analysis_report.md).

Trial 16's table is the top scorer among fresh random draws: the script's evolution branch never ran, so no mutation or crossover happened. Its fitness read an accuracy field that does not exist, so accuracy carried no weight at all. Its mean error of 5,577 describes a random table picked without regard to accuracy, not a price paid for variety.

Trial 16 used its own training loop, and its score is the only one in the project that is not clipped at 1. Over its 100-epoch history the best score peaked at 1.05 in epoch 33, and the saved table is that epoch's random draw. The trial ran for 40,469 seconds.

No training set came near 10,000 ODEs

The project claimed:

Large-scale training on 10k ODEs confirmed the framework could replicate classical integrators (README).

The 10,000-ODE set was used only for the final evaluation. Trials 8 to 16 trained on 1,000 ODEs each and trials 1 to 7 on 500 to 1,200, so no training set came near 10,000. No committed result ran on a GPU, and the author had already removed the CUDA-speedup claim from the README.

In the final code the training generator has 11 families and the reference solver handles 5 of them, so a training set from that generator held ODEs the pipeline could not score. The final test drew from a separate generator with 6 families.

What the project did not check

The project enforced no order conditions: the only hard validation was that the weights sum to one, and its order-4 test uses a condition that RK4 itself fails, so it labels RK4 as order 3. Trials 12, 14 and 15, which saved RK4, and trial 13, which saved Dormand-Prince, are labelled order 3. Every other saved table, trials 1 to 10 and 16, is labelled order 1.

The repository has no automated test suite: tests/ holds two example scripts, and src/evaluation/quick_test.py is a smoke script.

The training log

metrics_log.csv records one training run: 100 epochs of 100 four-stage candidates, 10,000 rows in all. It has no trial column, but only trial 15's saved settings ask for 100 candidates per epoch, so it is most likely trial 15's log. Every logged composite score is 1, the ceiling the score is clipped to, so the score gave the search nothing to select on.

The lowest max_error of any candidate in the log, 2.28e-07, comes at epoch 1, and no later epoch goes below it.

Lowest and median max_error per epoch in the training logScatter plot on a log scale of the lowest and the median max_error in each of 100 epochs, two series. The lowest value in the log, 2.28e-07, is at epoch 1.Lowest max_error in the epochMedian max_error in the epochmax_error (log scale)1e-71e-61e-50.00010.0010.011102030405060708090100Epochlowest in the log: 2.28e-07, epoch 1Epoch 1: median max_error 0.000293Epoch 2: median max_error 0.000702Epoch 3: median max_error 0.001Epoch 4: median max_error 0.000178Epoch 5: median max_error 0.000102Epoch 6: median max_error 0.000638Epoch 7: median max_error 0.00215Epoch 8: median max_error 0.000312Epoch 9: median max_error 0.000198Epoch 10: median max_error 4.94e-05Epoch 11: median max_error 0.000426Epoch 12: median max_error 0.000634Epoch 13: median max_error 0.000216Epoch 14: median max_error 9.74e-05Epoch 15: median max_error 0.000921Epoch 16: median max_error 5.33e-05Epoch 17: median max_error 0.000695Epoch 18: median max_error 0.000949Epoch 19: median max_error 0.000141Epoch 20: median max_error 0.000708Epoch 21: median max_error 9.44e-05Epoch 22: median max_error 0.000851Epoch 23: median max_error 7.41e-05Epoch 24: median max_error 0.00495Epoch 25: median max_error 0.000164Epoch 26: median max_error 0.00065Epoch 27: median max_error 0.000217Epoch 28: median max_error 0.00465Epoch 29: median max_error 0.000121Epoch 30: median max_error 0.00563Epoch 31: median max_error 8.72e-05Epoch 32: median max_error 0.00302Epoch 33: median max_error 0.000307Epoch 34: median max_error 0.00649Epoch 35: median max_error 0.000229Epoch 36: median max_error 0.000691Epoch 37: median max_error 0.000282Epoch 38: median max_error 3.28e-05Epoch 39: median max_error 0.000155Epoch 40: median max_error 0.000567Epoch 41: median max_error 0.000179Epoch 42: median max_error 0.00754Epoch 43: median max_error 0.000634Epoch 44: median max_error 0.000359Epoch 45: median max_error 0.000136Epoch 46: median max_error 0.00019Epoch 47: median max_error 0.000715Epoch 48: median max_error 0.000813Epoch 49: median max_error 0.000645Epoch 50: median max_error 0.00629Epoch 51: median max_error 0.00654Epoch 52: median max_error 0.000153Epoch 53: median max_error 0.000183Epoch 54: median max_error 0.00553Epoch 55: median max_error 8.01e-05Epoch 56: median max_error 0.000167Epoch 57: median max_error 0.000491Epoch 58: median max_error 6.47e-05Epoch 59: median max_error 0.000131Epoch 60: median max_error 0.000167Epoch 61: median max_error 0.000659Epoch 62: median max_error 0.000707Epoch 63: median max_error 0.000318Epoch 64: median max_error 8.08e-05Epoch 65: median max_error 9.22e-05Epoch 66: median max_error 0.00115Epoch 67: median max_error 5.55e-05Epoch 68: median max_error 0.00062Epoch 69: median max_error 0.0122Epoch 70: median max_error 0.000737Epoch 71: median max_error 0.000446Epoch 72: median max_error 0.000163Epoch 73: median max_error 0.000158Epoch 74: median max_error 0.000103Epoch 75: median max_error 0.000438Epoch 76: median max_error 0.000172Epoch 77: median max_error 0.00114Epoch 78: median max_error 0.000759Epoch 79: median max_error 4.71e-05Epoch 80: median max_error 0.000194Epoch 81: median max_error 0.000351Epoch 82: median max_error 0.0139Epoch 83: median max_error 0.000127Epoch 84: median max_error 0.000209Epoch 85: median max_error 0.000759Epoch 86: median max_error 0.000151Epoch 87: median max_error 0.000581Epoch 88: median max_error 0.00127Epoch 89: median max_error 0.000603Epoch 90: median max_error 0.00122Epoch 91: median max_error 0.000188Epoch 92: median max_error 0.000106Epoch 93: median max_error 0.00199Epoch 94: median max_error 0.000347Epoch 95: median max_error 0.000405Epoch 96: median max_error 0.00201Epoch 97: median max_error 0.000322Epoch 98: median max_error 8.20e-05Epoch 99: median max_error 0.00176Epoch 100: median max_error 0.0108Epoch 1: lowest max_error 2.28e-07Epoch 2: lowest max_error 1.51e-05Epoch 3: lowest max_error 1.82e-05Epoch 4: lowest max_error 1.31e-06Epoch 5: lowest max_error 3.21e-07Epoch 6: lowest max_error 1.58e-05Epoch 7: lowest max_error 1.26e-05Epoch 8: lowest max_error 8.55e-06Epoch 9: lowest max_error 6.19e-06Epoch 10: lowest max_error 3.61e-07Epoch 11: lowest max_error 7.61e-06Epoch 12: lowest max_error 2.40e-05Epoch 13: lowest max_error 5.28e-06Epoch 14: lowest max_error 1.58e-06Epoch 15: lowest max_error 3.87e-05Epoch 16: lowest max_error 1.35e-06Epoch 17: lowest max_error 8.00e-06Epoch 18: lowest max_error 3.85e-05Epoch 19: lowest max_error 6.06e-06Epoch 20: lowest max_error 7.02e-05Epoch 21: lowest max_error 3.69e-06Epoch 22: lowest max_error 3.04e-05Epoch 23: lowest max_error 2.50e-06Epoch 24: lowest max_error 6.12e-05Epoch 25: lowest max_error 2.81e-06Epoch 26: lowest max_error 7.90e-05Epoch 27: lowest max_error 7.79e-06Epoch 28: lowest max_error 0.000105Epoch 29: lowest max_error 4.09e-06Epoch 30: lowest max_error 0.000375Epoch 31: lowest max_error 1.24e-06Epoch 32: lowest max_error 6.39e-05Epoch 33: lowest max_error 2.20e-06Epoch 34: lowest max_error 0.000131Epoch 35: lowest max_error 8.19e-06Epoch 36: lowest max_error 2.58e-05Epoch 37: lowest max_error 4.75e-06Epoch 38: lowest max_error 6.52e-07Epoch 39: lowest max_error 8.60e-06Epoch 40: lowest max_error 5.70e-06Epoch 41: lowest max_error 3.12e-06Epoch 42: lowest max_error 0.000146Epoch 43: lowest max_error 1.41e-05Epoch 44: lowest max_error 8.90e-06Epoch 45: lowest max_error 1.43e-06Epoch 46: lowest max_error 1.30e-06Epoch 47: lowest max_error 1.38e-05Epoch 48: lowest max_error 3.45e-05Epoch 49: lowest max_error 1.98e-05Epoch 50: lowest max_error 0.000484Epoch 51: lowest max_error 0.000228Epoch 52: lowest max_error 2.97e-06Epoch 53: lowest max_error 3.37e-06Epoch 54: lowest max_error 0.000352Epoch 55: lowest max_error 1.16e-06Epoch 56: lowest max_error 1.25e-06Epoch 57: lowest max_error 5.97e-06Epoch 58: lowest max_error 2.96e-06Epoch 59: lowest max_error 1.80e-06Epoch 60: lowest max_error 1.19e-06Epoch 61: lowest max_error 1.12e-05Epoch 62: lowest max_error 2.91e-05Epoch 63: lowest max_error 8.53e-06Epoch 64: lowest max_error 3.84e-06Epoch 65: lowest max_error 7.60e-06Epoch 66: lowest max_error 2.60e-05Epoch 67: lowest max_error 1.72e-06Epoch 68: lowest max_error 1.28e-05Epoch 69: lowest max_error 0.000251Epoch 70: lowest max_error 1.31e-05Epoch 71: lowest max_error 4.77e-06Epoch 72: lowest max_error 3.58e-06Epoch 73: lowest max_error 1.31e-05Epoch 74: lowest max_error 3.76e-06Epoch 75: lowest max_error 6.46e-06Epoch 76: lowest max_error 2.66e-06Epoch 77: lowest max_error 4.78e-05Epoch 78: lowest max_error 5.13e-05Epoch 79: lowest max_error 1.45e-06Epoch 80: lowest max_error 8.96e-06Epoch 81: lowest max_error 1.43e-05Epoch 82: lowest max_error 7.10e-05Epoch 83: lowest max_error 1.57e-06Epoch 84: lowest max_error 1.36e-05Epoch 85: lowest max_error 2.95e-05Epoch 86: lowest max_error 6.21e-06Epoch 87: lowest max_error 8.15e-06Epoch 88: lowest max_error 3.24e-05Epoch 89: lowest max_error 4.26e-05Epoch 90: lowest max_error 6.98e-05Epoch 91: lowest max_error 1.63e-06Epoch 92: lowest max_error 9.86e-07Epoch 93: lowest max_error 3.95e-05Epoch 94: lowest max_error 4.49e-06Epoch 95: lowest max_error 4.98e-06Epoch 96: lowest max_error 3.18e-05Epoch 97: lowest max_error 6.39e-06Epoch 98: lowest max_error 3.21e-06Epoch 99: lowest max_error 5.40e-05Epoch 100: lowest max_error 5.95e-05
Lowest and median max_error among the 100 candidates in each epoch of the one run in metrics_log.csv, log scale. Every row has a composite score of 1, the clip ceiling. Source: metrics_log.csv at d89c2c809ee0.
Data table
EpochLowest max_errorMedian max_error
12.28e-070.000293
21.51e-050.000702
31.82e-050.001
41.31e-060.000178
53.21e-070.000102
61.58e-050.000638
71.26e-050.00215
88.55e-060.000312
96.19e-060.000198
103.61e-074.94e-05
117.61e-060.000426
122.40e-050.000634
135.28e-060.000216
141.58e-069.74e-05
153.87e-050.000921
161.35e-065.33e-05
178.00e-060.000695
183.85e-050.000949
196.06e-060.000141
207.02e-050.000708
213.69e-069.44e-05
223.04e-050.000851
232.50e-067.41e-05
246.12e-050.00495
252.81e-060.000164
267.90e-050.00065
277.79e-060.000217
280.0001050.00465
294.09e-060.000121
300.0003750.00563
311.24e-068.72e-05
326.39e-050.00302
332.20e-060.000307
340.0001310.00649
358.19e-060.000229
362.58e-050.000691
374.75e-060.000282
386.52e-073.28e-05
398.60e-060.000155
405.70e-060.000567
413.12e-060.000179
420.0001460.00754
431.41e-050.000634
448.90e-060.000359
451.43e-060.000136
461.30e-060.00019
471.38e-050.000715
483.45e-050.000813
491.98e-050.000645
500.0004840.00629
510.0002280.00654
522.97e-060.000153
533.37e-060.000183
540.0003520.00553
551.16e-068.01e-05
561.25e-060.000167
575.97e-060.000491
582.96e-066.47e-05
591.80e-060.000131
601.19e-060.000167
611.12e-050.000659
622.91e-050.000707
638.53e-060.000318
643.84e-068.08e-05
657.60e-069.22e-05
662.60e-050.00115
671.72e-065.55e-05
681.28e-050.00062
690.0002510.0122
701.31e-050.000737
714.77e-060.000446
723.58e-060.000163
731.31e-050.000158
743.76e-060.000103
756.46e-060.000438
762.66e-060.000172
774.78e-050.00114
785.13e-050.000759
791.45e-064.71e-05
808.96e-060.000194
811.43e-050.000351
827.10e-050.0139
831.57e-060.000127
841.36e-050.000209
852.95e-050.000759
866.21e-060.000151
878.15e-060.000581
883.24e-050.00127
894.26e-050.000603
906.98e-050.00122
911.63e-060.000188
929.86e-070.000106
933.95e-050.00199
944.49e-060.000347
954.98e-060.000405
963.18e-050.00201
976.39e-060.000322
983.21e-068.20e-05
995.40e-050.00176
1005.95e-050.0108

What still holds

A search like this needs a stepper that takes any explicit Butcher table, a generator of stiff and non-stiff test problems with a reference solver, per-trial configurations and saved artifacts. The 2025 project built all of them, and they run end to end for explicit tables. Two parts have known defects: the stepper runs only the explicit part of a table, and the reference solver has no branch for robertson. The benchmark ran 12 methods over 10,000 ODEs to completion. The report kept every entry, the weakest included, and stated trial 16's mean error of 5,577 as a loss.

The benchmark ranking holds with its limits: in float64 on these ODEs, the Dormand-Prince table and RK4 had the lowest errors. The rk run also found float64 rk4 far more accurate than its champion. Of the two, the 2025 benchmark is the weaker evidence, because its other entries were copies of those two tables, random tables and truncated methods. The rk run looked where the 2025 project did not, in Q15 fixed point with floor rounding.

The author caught some of this in 2025. Trials 6 and 7 put the coefficients that earlier trials kept producing on a forbidden list, so the author had noticed the repeated table, though not its cause. Commit d148c43 changed competitive performance to significantly worse accuracy once the 10,000-ODE results came in, though the same commit strengthened near-optimality to mathematical optimality. Commit 8c7cdd7 removed the README's CUDA speedup claim and its outperform claims.