Does a server's prompt cache change evaluation verdicts?
Turning a server's prompt cache on changed 39 of 720 replies (5.4%). Strict verdicts changed in 5 of 720 cases, which is not significant (the confidence interval starts at 0).
Key numbers
replies changed
39 of 720 (5.4%)Source file:
Source hash:
cache-flip-study/runs/2026-10-13-droplet/results.jsonSource hash:
f9b7ba116c66772ce7312c330cc0087fb04819c50e506d6790a1d5c226d31511verdicts changed
5 of 720 (not significant)Source file:
Source hash:
cache-flip-study/runs/2026-10-13-droplet/results.jsonSource hash:
f9b7ba116c66772ce7312c330cc0087fb04819c50e506d6790a1d5c226d31511Caveats
- Two small models near the accuracy floor, CPU, one server build
Watch the roundup
What we tested, and what we learned (90 s)
Video hash: 3de55bc330bba5f8
Request the full write-up
Full write-ups are not public yet. You can request one; requests are reviewed by hand.
Requests are stored privately for up to twelve months, reviewed by hand, never shared, and deleted on request. Contact QMAI@quantummindaillc.com.