Memory-light fine-tuning (MeZO) vs AdamW-LoRA, pilot 1
On a randomly initialised tiny model both methods sat near chance; the score ratios were 0.842, 1.059 and 0.957 (0.9 needed on every seed). Inconclusive.
Key numbers
score ratio per seed (need 0.9 each)
0.842, 1.059, 0.957Source file:
Source hash:
mezo-vs-adam/runs/2026-10-13/results.jsonSource hash:
20da7001daf41faf5bbeec1c89fa7bfe3fe57effb380be250ff85f2e6212818bCaveats
- Random-init model; CPU memory only
Watch the roundup
What we tested, and what we learned (90 s)
Video hash: 3de55bc330bba5f8
Request the full write-up
Full write-ups are not public yet. You can request one; requests are reviewed by hand.
Requests are stored privately for up to twelve months, reviewed by hand, never shared, and deleted on request. Contact QMAI@quantummindaillc.com.