AI Gateway · Claude Code · Self-Hosted

Reduce Claude Code Cost, Measured on Your Own Traffic.

TheYgent’s AI Gateway sits between Claude Code and the Anthropic API. It shows what every session, subagent, and developer costs, and what routing would save — then, when you choose, sends subagents and cold-cache turns to a cheaper Claude model and a share of the work to local models you run. It runs on your machine or your cluster, and ships with TheYgent 1.0.

Try It on Your Machine ↓ Read the AI Gateway Guide →

Where Claude Code’s Cost Goes

A Claude Code session is more than your prompts. It is a main conversation, the subagents it starts (an Explore subagent, for example), compaction when the context fills, and small side requests such as session titles and quota checks — each priced at the model it asked for. With CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 in its setup, Claude Code labels every request, so the gateway can record each one’s kind, model, tokens (cache reads and writes included), and cost.

Two things move the bill most. The first is which model answers: Sonnet 5.5 is half the list price of Opus 5.5, and a subagent starts with an empty cache, so moving it to a cheaper model costs nothing extra. The second is how much each request carries. Behind a custom base URL, Claude Code turns MCP tool search off, and every request then carries the definition of every connected tool: in one measurement with many connectors loaded, a one-line prompt went from about 28,000 to 184,000 input tokens. The gateway’s setup turns tool search back on with ENABLE_TOOL_SEARCH=true, and its Report warns when your traffic shows it off.

Observe, Route, Then Go Local

Mode	What it does	What reaches Anthropic
Observer	Records every exchange — its cost, its kind, and what routing would have done and saved.	Every request, exactly as Claude Code sent it.
Router: a cheaper Claude	Runs subagents and cold-cache turns on the target you name, such as `claude-sonnet-5-5`.	The same bytes with one change, the model. A target that refuses hands the request back to the model Claude Code asked for, unseen.
Router: your own models	Answers a share of each kind of request — side requests, subagents, the main conversation — with models on your inference plane.	Whatever the local models don’t answer: requests that don’t translate, quota checks, token counting, and every fallback.
Passthrough	Forwards traffic and records nothing.	Everything, unrecorded.

An admin picks the mode on the AI Gateway’s Settings tab, and every save is a new, immutable configuration version. A gateway attached to the control plane forwards in passthrough until someone saves one, so an organization’s traffic is recorded only once someone chose to record it; a personal gateway on its own records from the start.

How the Gateway Decides What Moves

Prompt caches belong to one model, so switching mid-conversation throws a warm cache away and pays to rebuild it. The policy considers a move only where the cache is cold anyway, then keeps that decision until the next cold point. Every decision carries its reason, in the same words the Sessions tab uses.

subagent spawn A subagent starts with an empty cache — the clearest saving. In measurements, the same exploration subagent cost about half as much on Sonnet 5.5 as on Opus 5.5.
cold cache The main conversation at its start, after compaction, or once its cache expired — and only if you name a target for it.
follows cache Every later request stays on the model the cache is on, so a conversation never switches model mid-turn.
never routed Claude Code’s quota check, auto mode’s safety checks, side requests that ride the main conversation’s cache, and kinds of side request nobody has reviewed.

AI Gateway · Session 9b2f… · Observer Observing

request asked → would route cost

main opus-5-5 no target set $0.071

subagentExplore opus-5-5 → sonnet-5-5 subagent spawn $0.094

subagentExplore opus-5-5 → sonnet-5-5 follows cache $0.012

sidetitle haiku-5-5 unreviewed kind $0.001

main opus-5-5 warm cache $0.019

5 requests would save $0.053 Nothing changed

Route Claude Code to a Local LLM, One Share at a Time

In router mode, the gateway can answer part of Claude Code’s work with models on your TheYgent inference plane: models you install from the in-app catalog and run on its built-in engines — llama.cpp on any machine, MLX on Apple Silicon — or a model on your own GPU servers, registered as OpenAI-compatible. A request a local model answers costs no API spend. How much goes local is a dial per kind of request:

Dial	Starts at	What goes local
Side requests	100%	Kinds of side request, such as session titles, that an admin reviewed as local model in the Catalog.
Subagents	0%	A share of subagents, each from its first request to its last — optionally only some agent types, such as `Explore`.
Main conversation	0%	A share of conversations, from their start or their last compaction.

Each dial has a share, served locally, and a trial, measured locally while Anthropic still answers. A trial sends a copy of each real request to the local model after the fact and records whether it picked the same next action as Claude — the same tool first, or answering where Claude answered. Small models follow agent work poorly: in one measurement, a 4-billion-parameter model matched Claude on 1 real exploration turn in 6. So agent shares start at zero, and you raise them on your own evidence. A conversation that outgrows its local model moves to a larger listed one, and to Anthropic only when none can take it.

A local model answers only what translates faithfully: text, tool definitions, tool calls and their results, and images for a model that takes them. Until its first text or its first whole tool call goes out, any failure — an error, no start in time, an unusable answer — sends the original request to Anthropic, and Claude Code never sees it. Documents, Anthropic’s server tools, token counting, and the quota check always stay with Anthropic, so the gateway still needs an Anthropic credential: local models extend it, they don’t replace it.

The local route is verified with llama.cpp. For a team, serve the model on an engine that handles several requests at once, such as a llama.cpp model with parallel slots: a request that waits past the gateway’s time limit goes to Anthropic instead. Local models: setup, trials, and limits.

Claude Code Usage and Cost Tracking, Self-Hosted

Every exchange becomes a record: the model Claude Code asked for and the one that served it, tokens, cost, the request’s kind, and the decision with its reason. The AI Gateway page in TheYgent turns those records into:

Report — spend, what it costs with routing, the saving, and calls, by day, broken down by request kind, client, requested model, served model, decision, reason, developer, or session.
Sessions — every request of a Claude Code session in order, with what the policy did or would have done, and why.
Shadow runs — copies of requests sent to a routing target or a local model after the exchange ends, showing how much the target wrote and how often it picked the same next action. Claude Code never waits for them.
Setup warnings — tool search off, requests not labelled, a model with no known price — each with its fix.
Subscription seats — for developers on a claude.ai plan, how much of the plan’s five-hour and weekly allowance their traffic used.

Prices are Anthropic’s published list prices, checked for changes every week, plus contracted rates where an admin enters them. A personal gateway gives the same totals from one command: theygent-ai-gateway report --since 24h.

Claude Code prices every token at the model it asked for, so once the gateway routes, Claude Code’s own cost display overstates spend. One Explore subagent task cost $0.16 on Opus 5.5 and $0.08 on Sonnet 5.5; routed to Sonnet, it still shows about $0.16 in Claude Code. The token counts agree — the gateway’s Report is the figure to trust for cost.

Your Gateway, Your Trust Domain

Setting	Personal	Team
Serves	You, on your own machine	Every developer who points Claude Code at it
Listens on	Loopback only (`127.0.0.1:8082`)	Any address you choose
Identifies callers by	Nobody — it is yours	Each developer’s gateway key (`tyg_…`); no key, or a revoked one, is refused
Billing	Your own Claude login or API key	Each developer’s own credential, or the organization’s key
Records	Kept locally and read with the `report` command, or attached to the control plane	Sent to the control plane: the AI Gateway page
Runs with	`make up`	`make up`, Docker Compose (`--profile gateway`), or Kubernetes

The gateway relays Claude Code’s prompts, code, and credentials, so it runs where you run it, beside Claude Code. It never writes a prompt or a response anywhere, never stores a credential except an organization key you give it, and never sends a gateway key upstream. What it sends to TheYgent’s control plane is metadata only: timings, model names, token counts, costs, the routing decision and its reasons, and Claude Code’s session and agent ids — never prompt or response text. Developers see their own records in full; admins see everyone’s.

The gateway is a hop you add on purpose, and the Anthropic API stays its upstream: whatever a local model doesn’t answer goes to Anthropic as before. Savings shown before you route are estimates that assume the target writes as much as the model Claude Code asked for; shadow runs check that, and the figures for what actually ran are exact.

Try It on Your Machine

A personal gateway needs no keys. make up starts it with the rest of TheYgent, on 127.0.0.1:8082:

git clone https://github.com/al2m4n/TheYgent.git && cd TheYgent
cp .env.example .env   # ships a personal gateway on 127.0.0.1:8082
make up

Then point Claude Code at it, in the shell you start Claude Code from:

export ANTHROPIC_BASE_URL='http://localhost:8082'
export CLAUDE_CODE_GATEWAY_HINT_HEADERS='1'
export ENABLE_TOOL_SEARCH='true'

Work as usual — your own Claude login or API key goes through unchanged — and read what your traffic cost, and what routing would save:

uv run --package theygent-ai-gateway theygent-ai-gateway report --since 24h

You need uv, pnpm, and a Postgres to point at — make up starts one in Docker if you don’t have your own. To see the gateway on the AI Gateway page and to route, attach it to the control plane with a service key, then save observer or router mode on AI Gateway → Settings. For a team, IDEs, settings.json, and the desktop app, see Set Up Claude Code; for Docker Compose and Kubernetes, see Running the Gateway.

Questions

Asked and Answered.

How do I reduce what Claude Code costs?

Measure first. Run TheYgent’s AI Gateway between Claude Code and the Anthropic API in observer mode: it records what every request cost and what routing would have saved, without changing anything. Then switch to router mode to move subagents and cold-cache turns to a cheaper Claude model — Sonnet 5.5 is half the list price of Opus 5.5 — and, if you choose, a share of the work to models you run yourself. Keep tool search on, too: Claude Code turns it off behind a custom base URL, and every request then carries every connected tool’s definition.

Can Claude Code use a local model?

Yes, through the AI Gateway in router mode. You set a share per kind of request — side requests such as session titles, subagents, and the main conversation — and the gateway answers that share with a model on your TheYgent inference plane, at no API spend. Each model is measured first: a trial sends copies of real requests to it while Anthropic still answers, and records how often it picked the same next action. Whatever a local model can’t take goes to Anthropic before Claude Code sees anything.

Does routing change my prompts or break prompt caching?

No. A request routed to a cheaper Claude model changes in one place, its model; everything else reaches Anthropic exactly as Claude Code sent it. Because a prompt cache belongs to one model, the gateway moves a conversation only where its cache is cold anyway — when a subagent starts, when a conversation starts, or after compaction — and keeps it there until the next cold point.

Does the AI Gateway see my code?

Yes — it relays Claude Code’s prompts, code, and credentials, so it runs where you run it: your machine or your cluster. It never writes a prompt or a response anywhere, and never stores a credential except an organization key you give it. TheYgent’s control plane receives metadata only: timings, model names, token counts, costs, and the reasons behind each routing decision.

Does it work with a Claude subscription?

Yes, from the CLI and IDE extensions: the gateway forwards your own Claude login or API key unchanged. On a claude.ai plan, the Report’s dollar figures are the API-equivalent value of your traffic, not a bill, and it shows how much of the plan’s five-hour and weekly allowance you used. The desktop app reaches a gateway only with an API key, so it works through a team gateway billed to the organization’s key.

How much will the AI Gateway save?

That depends on your traffic, which is why observer mode comes first: it shows what routing would save on your own requests before anything changes. In one measurement, the same exploration subagent cost about half as much on Sonnet 5.5 as on Opus 5.5, and a request a local model answers has no API spend. Savings are estimates that assume the target writes as much as the model Claude Code asked for; shadow runs check that assumption.

Keep Reading

docs