about 10-20% of the time, it will completely make up fields or values that don't exist or work but look plausible enough to be right
JohnMakin on Hacker News
10-20% of the time
In the thread “GPTs and Hallucination”
Helped: Parsing and generating HCL for common providers
Got in the way: Making up fields, values, and mixing up versions
… It is pretty good at parsing/generating HCL/terraform for most common providers. However, about 10-20% of the time, it will completely make up fields or values that don't exist or work but look plausible enough to be right - e.g., mixing up a resource ARN with an resource id, or things like "ssl_config" may become something like "ssl_configuration" and leave you puzzling for 20 minutes what's wrong with it. Another thing it will constantly do is mix up versions - terraform providers change often, deprecate things all the time, and there are a lot of differences in how to do …