AI agents waste substantial computing resources generating text for internal decisions that don't require written answers, according to a new report from The New Stack. The overhead has grown so severe that OpenAI's researchers recently disclosed spending $7,000 daily on agent workloads. Developer Jared Palmer released Kev on Sunday, a family of open decision models built on Qwen 3.5 that eliminates text generation entirely for routing, safety checks, and ranking tasks.
Palmer's release includes three models — 0.8 billion, 4 billion, and 9 billion parameters — all designed as prefill-only systems. The models process state, questions, and candidates in one forward pass, then extract decisions from a pointer head without an autoregressive decoding loop. Kev supports three decision types: Noul for yes/no questions, Choice for selecting among candidates, and Score for ordered levels. For a tool-routing decision, the output delivers probabilities across available options — such as search at 0.82, database at 0.13, and calculator at 0.05. The 4 billion parameter model processed three questions in 277 milliseconds in bf16 on an M5, though Palmer's documentation doesn't include a controlled comparison against Qwen generating equivalent answers on identical hardware. The largest model, Kev-9B, achieved 83.7% accuracy on the project's locked out-of-domain test, according to Palmer's model card.
The report notes that Kev can still select the wrong tool, but because it scores only the candidates it receives, it can't introduce an option that isn't on the list. Palmer documents several limitations alongside the benchmark results. The probabilities Kev returns don't always reflect how confident developers should be in the outcome, and temperature calibration can drift on unseen source distributions — a problem for agents using probability thresholds to decide whether to execute an action or escalate it. Fine-tuning also diminishes some capabilities inherited from the underlying model, with Palmer's evaluations showing declines on general-knowledge and arithmetic tests, particularly among the smaller models.
Why does this matter? Agent loops constantly cycle through routing, ranking, safety checks, tool selection, and escalation before generating user-facing text, and each step currently forces generative models to produce written answers that agents must parse back into structured decisions. That pattern burns tokens and compute time on text nobody reads. Kev handles these bounded decisions directly while leaving open-ended reasoning and final responses to larger generative models, a division of labor that could reduce costs as agent workloads scale. Palmer released Kev under Apache 2.0 with model weights, training code, and evaluation tooling, giving developers the option to run and train it on their own infrastructure. TypeSafe introduced a similar approach earlier this month with Jev as part of its System One platform, using the same Noul, Choice, and Score primitives, but Jev's weights and training data aren't public. For applications making only a handful of bounded decisions, constrained decoding on an already-running model may be simpler than adding another model to the stack, but as agent loops grow more complex, the ability to evaluate several decisions against the same context in a single forward pass could become more valuable. The trade-off isn't just speed versus accuracy — it's whether the infrastructure supporting agent decisions needs to generate prose at all when a probability will do. Organizations deploying multi-step agents may find themselves weighing whether the simplicity of a single generative model justifies the compute expense, or whether splitting decision-making from reasoning creates new points of failure that offset the savings.