I've found it to be worse than useless at identifying where the real complexities and bug opportunities actually are
minasmorath on Hacker News
30% of the time
In the thread “How we made our AI code review bot stop leaving nitpicky comments”, replying to this comment
Got in the way: Misidentifies critical areas, causing juniors to ignore real bugs and waste time.
To be honest, I tried to work with with GitHub Copilot to see if it could help junior devs focus in on the important parts of a PR and increase their efficiency and such, but... I've found it to be worse than useless at identifying where the real complexities and bug opportunities actually are. When it does manage to identify the areas of a PR with the highest importance to review, other problematic parts of the PR will go effectively unreviewed, because the juniors are trusting that the AI tool was 100% correct in identifying the problematic spots and nothing else is concerning. …