Micro-Agent: put the agent loop inside the API, not your app

The vLLM Semantic Router team just published Micro-Agent, and the pitch is sneaky-clever. Instead of building a complicated multi-model orchestration in your application, you hide the whole collaboration behind a single API call inside the serving layer. One call goes in, a bounded committee of models works the problem, one answer comes back.

It ships five looping strategies. Confidence escalates to a stronger model only when the cheap one is unsure. Ratings runs candidates in parallel. ReMoM fans out reasoning attempts and synthesizes them. Fusion treats model disagreement as evidence and lets a judge resolve it. Workflows wires up planner, patcher, verifier and finalizer patterns. All of it under one endpoint, all of it in the serving layer.

Here is why it matters. The closed-model recipe hits 92.6 on LiveCodeBench, edging out Sakana's Fugu Ultra at 92.0, and matches it on GPQA-Diamond and Humanity's Last Exam. You beat the frontier not by training a bigger model but by orchestrating collaboration where the tokens already live. Built with MBZUAI, McGill and Mila.

This is the other half of a thesis showing up everywhere right now: the model is becoming a commodity and the value is moving to the layer that routes and combines them. Micro-Agent just put that layer inside vLLM itself. Link: vllm.ai/blog/2026-06-29-micro-agent-frontier-models