Mixed
right on the first PR, it actually found a bug that my CI tests did not catch
lukaslalinsky on Hacker News
In the thread “The AI Code Review Disconnect: Why Your Tools Aren't Solving Your Real Problem”
Helped: CodeRabbit found a bug that CI tests missed.
Got in the way: Gemini Code Assist suggested code that would lead to a bug.
… I think their main value is reducing mistakes through having one extra set of eyes, even if mechanical ones, looking at the code. As a sole developer of a non-trivial open source project, I've recently started using CodeRabbit, very skeptical about it, but right on the first PR, it actually found a bug that my CI tests did not catch, decided to keep it after that. Gemini Code Assist on the other hand, the first suggestion it did would actually lead to a bug, so that was out immediately.