No
If you ask an agent to make evals it’ll always try to bias and overfit just to make tests pass
moltar on Hacker News
In the thread “I stopped reviewing my agents' code. Here's what I do instead”
Got in the way: Agents biased evals to overfit and pass tests.
… I added evals. If you ask an agent to make evals it’ll always try to bias and overfit just to make tests pass. No matter if I have it stated in every possible place not to do it and Codex instructed to catch these cases. …