a faster model's output on average needs more revision before it basically works, and on average you end up spending more energy on that than you save by reducing time to first response
twotwotwo on Hacker News
In the thread “AI agents: Less capability, more reliability, please”
Helped: Detailing requirements and explicitly pulling specific libraries into context.
Got in the way: Faster models required more revision than time saved.
… anything tradeoff, you more or less always have to prefer reliability. For example, even ignoring subtle head-scratcher type bugs, a faster model's output on average needs more revision before it basically works, and on average you end up spending more energy on that than you save by reducing time to first response. Up-front work that decreases the chance of trouble--detailing how you want something done, explicitly pulling into context specific libraries--also tends to be worth it on net, even if the agent might have gotten there by searching (or you could get it there through follow-up requests). …