I had to deal with product that work great in the field...at least at first, before it start literally catching fire
0xEF on Hacker News
In the thread “Don't be the out of touch Kung Fu master”, replying to this comment
Got in the way: Tests were altered to match results; code too complex to review.
… This is not necessarily the fault of the agent, either; it's just interpretting the prompt(s) - written by a flawed human, btw - with stochastic mechinations that seem to make a great deal of sense on the surface, but remain unable to be followed or repeated by the brains of (most of) its users. As a rresult, I had to deal with product that work great in the field...at least at first, before it start literally catching fire, ruining its own powertrain because everything the agents touched became too complex with too many subtle cracks in the veneer to review properly. The system (read; capitalism) demanded viable product quickly to please investors, and the burnt-out humans who decided to try this AI thing ended up trusting it nearly completely, so any ideas of repeatable and …