Yes
We got to a solution with both LLMs, but it required us to be quite explicit in our prompting
jumploops on Hacker News
In the thread “Ask HN: What is your experience with Claude 3.5?”
Helped: Being quite explicit in prompting to resolve the issue.
Got in the way: LLMs kept logging the diff, causing YAML parse errors.
… Both LLMs struggled to “understand” the issue, even when they were directly told what to do (they kept logging the diff for debug, which would continue to break it). We got to a solution with both LLMs, but it required us to be quite explicit in our prompting. Caveat: both GPT-4o and Claude 3.5 Sonnet were accessed via a self-hosted interface that hits the API directly, so your experience through the UIs may be different.