Can a search invent a better optimizer? (run 2, 8 training instances)
With 8 training instances per family the search again failed on held-out families (0, 0 and 1 of 4). The hypothesis that more training instances shrink the train/held-out gap was not supported: it shrank on 2 seeds and grew on 1.
Key numbers
held-out families beaten per seed (need 3 of 4)
0/4, 0/4, 1/4Source file:
Source hash:
rule-search-lab/runs/2026-10-13-run2/RESULTS.jsonSource hash:
a6340037f38c893cCaveats
- Template mutator, not a model
Watch the roundup
What we tested, and what we learned (90 s)
Video hash: 3de55bc330bba5f8
Request the full write-up
Full write-ups are not public yet. You can request one; requests are reviewed by hand.
Requests are stored privately for up to twelve months, reviewed by hand, never shared, and deleted on request. Contact QMAI@quantummindaillc.com.