All 4,799 reports

MixedLlama · 7 Aug 2024

I was able to craft a small grammar file that does just that, in under 5 minutes, and use it successfully with llama.cpp

thrance on Hacker News

In the thread “Structured Outputs in the API”

Helped: Crafting a small grammar file for llama.cpp

Got in the way: APIs lack custom grammar support

… The company I work for desperately need the LLM to consistently generate results in a subset of HTML. I was able to craft a small grammar file that does just that, in under 5 minutes, and use it successfully with llama.cpp. Yet, there are still no API offering this basic feature that could really benefit everyone. …