Tagged “wec-inference”

5 posts · View all tags
Split one MCP toolbox between two agents so the scheduler cannot issue refunds
IntermediatePart 4

Split one MCP toolbox between two agents so the scheduler cannot issue refunds

Part 3 handed one agent all five tools and let a human pause guard the dangerous one. That guard depends on the model asking. This splits the same MCP server between a scheduler and a billing agent, so the scheduler cannot refund an invoice for the simplest reason available — the tool was never in its list. Then the supervisor goes back on top, and routing turns out not to be the guarantee.

AIAgentsself-hostingmcp
September 4, 202613 min read
Read more
Book appointments and refund invoices from a LangGraph agent over MCP
IntermediatePart 3

Book appointments and refund invoices from a LangGraph agent over MCP

The scheduler and billing agents from Parts 1 and 2 never actually scheduled or billed anything. This gives them a calendar, a customer list and a refund that no model can issue alone — over MCP, on the stateless spec, with the model on WEC Inference. Includes three things nobody documents: your server still defaults to sessions, ctx.elicit is the old API and its era error goes to the model, not to you, and cache=True does nothing unless the server advertises a TTL.

AIAgentsself-hostingmcp
September 3, 202621 min read
Read more
Load test an LLM gateway and find the stall the median hides
AdvancedPart 5

Load test an LLM gateway and find the stall the median hides

Part 4 measured a blocking call in a callback at 200-350ms and said the real damage only shows under concurrency. This runs it: sixteen requests at once, with a control against the upstream so you can tell your gateway's fault from the model's. The medians of the two versions are almost identical. One version drops requests and silently stops masking.

performancegatewaylitellmobservability
August 27, 202619 min read
Read more
Hand work between LangGraph agents without corrupting shared state
IntermediatePart 2

Hand work between LangGraph agents without corrupting shared state

A supervisor routes work to specialists, an agent hands off mid-task, and two agents write the same state key in the same step. One of those raises an error rather than picking a winner — and the routing decision costs 412 output tokens to say one word.

AIAgentsself-hostinglanggraph
August 26, 202618 min read
Read more
Checkpoint a LangGraph agent on a WEC Instance so crashes cost you nothing
IntermediatePart 1

Checkpoint a LangGraph agent on a WEC Instance so crashes cost you nothing

Persist agent state to Postgres on a WEC Instance, call a model on the WEC Inference API, then kill the process mid-run twice and resume both times from exactly where it stopped — plus the human-approval pause, and the interrupt that silently fires your side effects twice.

AIAgentsself-hostinglanggraph
August 26, 202621 min read
Read more