In the end, just asking LLMs to simply extract the names from the grafana dashboard worked better
raducu on Hacker News
22 used metrics
In the thread “Classic Data science pipelines built with LLMs”, replying to this comment
Helped: Asking LLMs to directly extract names from the dashboard file.
Got in the way: ChatGPT generated a script that only printed metrics appearing exactly once.
… What I wanted was a script/one-liner that would print all the metric names just once (de-duplicate) and chat-gpt ad-literam produced a script that only prints metrics that show up exactly once in the whole file. In the end, just asking LLMs to simply extract the names from the grafana dashboard worked better, parsing out expressions, only producing unique metrics names and all that, but there was no way to know for sure, just that given that 3/4 of the LLMs produced the same output meant it was most likely corect. I fixed the programatic approach and got thr same result, but it was a very wiered feeling asking the LLMs to just give me the result of what for …