Can a search invent a better optimizer? (run 1)
A template search over optimizer update rules won on its training tasks but beat tuned Adam on only 0, 1 and 0 of 4 held-out task families across three seeds (3 of 4 needed). The best rules were bounded Adam variants that overfit two training instances.
Key numbers
held-out families beaten per seed (need 3 of 4)
0/4, 1/4, 0/4Source file:
Source hash:
rule-search-lab/runs/2026-10-13/RESULTS.jsonSource hash:
70f64a1fce5db56ef05fd228e2a49d5aa850dabf8230c9f73e5250575c0abd0fCaveats
- Template mutator, not a model
- 5 held-out seeds per family (smallest p = 1/32)
Watch the roundup
What we tested, and what we learned (90 s)
Video hash: 3de55bc330bba5f8
Request the full write-up
Full write-ups are not public yet. You can request one; requests are reviewed by hand.
Requests are stored privately for up to twelve months, reviewed by hand, never shared, and deleted on request. Contact QMAI@quantummindaillc.com.