New updates to the llama.cpp inference engine integrate a /v1/systemone endpoint that permits the local execution of Jev-style inference. This functionality allows developers to ask typed questions and receive a probability for every potential option, a process used to route tickets, moderate content, or determine the next step for an AI agent. The system currently supports 5 open models with sizes ranging from 144M to 27B parameters.

The release focuses on delivering these capabilities locally to ensure efficiency and privacy without relying on cloud infrastructure. The project is building the tool ecosystem for these decision models and will add more supported models in future releases.