the challenge is that too much human intervention for the cheaper models drives up the cost
vidarh on Hacker News
aggregate total of about 350 plans
In the thread “No, it doesn't cost Anthropic $5k per Claude Code user”, replying to this comment
Helped: Using Sonnet/Opus for consistent delivery with fewer review interventions.
Got in the way: Cheaper models required excessive human intervention and prompt tweaking.
… and execute an aggregate total of about 350 plans so far, and the quality difference as measured in plans where the agent failed to complete the tasks on the first run is high enough that it comes out several times higher than Anthropics subscription prices, but probably cheaper than the API prices once we have improved the harness further - at present the challenge is that too much human intervention for the cheaper models drives up the cost. My dashboard goes from all green to 50/50 green/red for our agents whenever I switch from Claude to one of the cheaper agents... …