Mixed
Where it fails is on plugging business logic in. Tell it to generate a new piece of UI and it will do that fine
nottorp on Hacker News
maintenance dropped the productivity improvement to 2x
In the thread “Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases”
Helped: Generating new pieces of UI
Got in the way: Plugging business logic in; missing context without explicit instructions
… And that's with months of saved memories. Where it fails is on plugging business logic in. Tell it to generate a new piece of UI and it will do that fine. My guess is the 10x success stories are for from scratch applications where 80% of the code is boilerplate and self contained modules. …