How this was made

Each report here is a Hacker News comment in which someone says what happened when they used an AI coding tool or model at their own work. This page shows how we find them, what we show, and how far to trust the numbers.

From comments to reports

1. 431,582Hacker News comments from 1 Jan 2023 to 3 Nov 2026 that name a tool or model we search for.
2. 77,054are long enough to describe something, and say “I” or “we” about work. A simple text filter set aside the other 354,528.
3. 18,511describe what happened at the writer’s own work, according to an open AI model running on our own computer. It set aside opinions, questions, hobby projects and 7,892 sales pitches.
4. 7,736were accepted by Anthropic’s Claude Sonnet 5.5 as a first-hand report from work.
5. 4,888were also accepted by OpenAI’s GPT-6.1 Sol. A report needs both.
6. 4,799are on this site. We left out 22 where the two models gave opposite answers, 66 whose quote did not match the comment word for word and 1 that we removed by hand.

1 more are still being checked and will appear when both models have read them.

What each report shows

• One answer: yes, it worked; mixed, meaning it partly worked or it is too early to tell; or no.
• The words that give the answer, copied exactly, with a sentence either side.
• Who wrote it and when, with a link to the whole comment on Hacker News. We do not republish whole comments.
• The tools it names and, where the writer says, what helped or what got in the way.

How far to trust the numbers

• These are people who chose to post on Hacker News, not a random sample of developers. The numbers say what this group reported, not what happens everywhere.
• We only find comments that name something we search for. Someone who writes “the AI” without naming it is not counted.
• The answers come from AI models reading each comment. The two models gave the same answer on 81% of reports, and where one said yes and the other said no, the report is left out.
• The kind of work and the amount of back-and-forth are tagged by the same model on our own computer. On a random sample of 399, Anthropic’s Claude Sonnet 5.5 tagged them again without seeing the first answer and agreed on the kind of work 73% of the time, and on the amount of back-and-forth 85% of the time when both found it described.
• Each percentage shows how far it could be off by chance, given how many reports it rests on. We only call a gap between two tools real when it is bigger than that.
• No one has checked a large sample by hand yet. If you find a wrong answer, tell us and we will fix it.

What we search for

Coding tools: Claude Code, Cursor, Copilot, Codex, Windsurf, Aider, Cline, Devin, Trae, Qoder, Lingma, CodeGeeX, MarsCode, CodeBuddy, iFlow, Gemini CLI, Antigravity, Kiro, OpenCode, Roo Code, Kilo Code, Augment Code, Jules, Junie, Tabnine, Replit, Firebase Studio.

Models: Qwen, DeepSeek, GLM, Z.ai, Kimi, MiniMax, Doubao, Hunyuan, ERNIE, Baichuan, StepFun, Zhipu, Moonshot AI, EXAONE, HyperCLOVA, Sarvam, ChatGPT, GPT, Claude, Gemini, Grok, Llama, Mistral, Opus, Sonnet.

Phrases: “vibe coding”, “coding agent”, “coding agents”, “agentic coding”.

Tool logos come from Simple Icons (CC0) and Lobe Icons by LobeHub (MIT licence). Each logo belongs to its owner and is shown only to name the tool.

Privacy and corrections

No login. Visits are counted with Google Analytics only if you allow it. Privacy

Spotted a wrong answer, a missing report, or a quote taken out of context? Email ai@diditwork.ai and we will fix it.

Which models and tools people talk about · Back to the answer