No Monkey
Business
← All notes
Cost control6 min read1216 words

LiteLLM fail-open

How spend tracking can keep working while budget enforcement skips a request path, the matrix to test, and the evidence that proves a limit actually rejects.

On this page

A budget that records spend but does not reject the request is a dashboard, not a control.

This note is the operator’s version: which request paths to exercise, what a safe reproduction looks like, and what evidence to demand before trusting a spend limit in production. Everything about LiteLLM’s behaviour below is attributed to its issue tracker, pull requests or release notes; the testing matrix, ordering and recommendations are our reading.

NMB · FIG.01 LITELLM · SPEND TRACKING VS BUDGET ENFORCEMENT Tracking is not enforcement. one request · three tag locations · two outcomes 01 REQUEST PATHS body tags metadata x-litellm-tags header 02 BUDGET GATE reads the tag representation the path actually carried checks key · customer · router 03 PROVIDER the call that costs money only when the gate allows it checked allowed reject 429 path the gate never sees no gate ran ENFORCED 429 before the provider call recorded spend does not move FAIL-OPEN provider called, spend logged nothing rejected, dashboard green Both outcomes look the same on a spend dashboard.
Same request, two outcomes. When the gate never sees the path, the provider call still happens and the dashboard stays green.

Tracking is not enforcement

LiteLLM does real spend accounting. That is a different subsystem from the code that decides whether a request may proceed.

The failure mode to watch for is quiet: the request succeeds, the provider is called, the spend is logged — and the limit that was supposed to stop the request never ran on that path.

That can happen when the gate and the tracker read different representations of the request. Tags are the obvious example, because they can arrive in more than one place. The same shape appears with customer identifiers, virtual-key scopes, and any configuration where the database or routing layer that enforcement expects is not the one in the request’s path.

The historical example, from LiteLLM issue #27480: tag-budget enforcement was skipped when tags arrived through the x-litellm-tags header path, while body tags enforced correctly. Spend still recorded. Fix #27573 landed on 13 May 2026 and the v1.85.0 release notes list header-based tag-budget enforcement among the fixes.

That is a record of a past bug, not a claim about any current release. The operator’s lesson is version-independent: a green budget dashboard proves recording, not rejection.

A budget that cannot reject on every path that can carry its identifier is a monitoring feature with a limit field.

The paths to test

Pick the paths your deployment actually exposes. Not every row applies to every version or configuration, and the field names move between releases.

Path or mode What the test must prove
Tag budget via body tags The request is rejected once the agreed spend is reached
Tag budget via metadata Same rejection as the body path
Tag budget via x-litellm-tags header Same rejection as the body path
Customer / end-user budget The identifier used for accounting is the one enforcement reads
Virtual-key budget A key over its limit cannot reach the provider
Router or fallback configuration Every deployment in the route is covered
Budget with no spend database configured The failure is explicit, not silent
Streaming, retries and async logging A retry or a stream does not slip past the gate
Direct provider access outside the proxy The bypass route is blocked or accepted as a documented risk

For each row, the pass condition is the same: a rejected request, a clear error, and no provider call.

A safe reproduction

Do this in an isolated environment with sandbox credentials, a throwaway database, and a budget small enough to trip in one request. Record the gateway version and configuration for every run.

# 1. Start an isolated proxy with a throwaway spend database and a sandbox upstream.
# 2. Attach a tag budget with one approved request, then send the same request
#    with the tag in each supported location.
curl -sS -D - -o /dev/null "$LITELLM_BASE/chat/completions" \
  -H "Authorization: Bearer $SANDBOX_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-tags: budget-probe" \
  -d '{"model":"sandbox-model","messages":[{"role":"user","content":"budget probe"}],"metadata":{"tags":["budget-probe"]}}'
# 3. Reset the spend between runs, repeat with the tag only in the body and only
#    in metadata, then repeat after a proxy restart.
# 4. After each run, read the recorded spend for the tag and compare it with the
#    HTTP response. Rejected request, unchanged spend: enforced.

Adapt paths and field names to the documentation for your pinned release; do not copy them between versions.

The useful output is a dated table: request variant, expected result, HTTP status, whether the upstream saw the call, and the spend movement. That table is what an auditor or a buyer can check.

What enforced looks like

  • The request is rejected before the provider call, and the upstream’s own logs show nothing arrived.
  • The error is explicit: a status code and a message that names the limit.
  • Recorded spend does not increase for the rejected request.
  • Body, metadata and header paths behave identically, before and after a restart.

If any of those is false, the budget is advisory on that path.

Limits of this note

The issue and fix links describe a historical, version-specific gap. This note does not claim a current vulnerability, and the reproduction was not re-run for publication. The matrix is a testing method, not a substitute for your own measurements. Gateway behaviour depends on the deployed version and configuration; treat every release note that mentions budgets as a reason to retest, not a reason to relax.

What to do, in order

  1. Pin the proxy version and attach the test evidence to it.
    Why: enforcement paths have changed between releases, and a release note is not a test result.
    Artifact: a dated matrix run next to the deployed version and configuration.

  2. Name one owner per control point: virtual key, team, tag, customer.
    Why: a limit with no owner is a limit nobody rechecks.
    Artifact: an ownership list that matches the budget configuration.

  3. Include the header and metadata tag paths in the regression test.
    Why: the historical fail-open was exactly a header-path gap while body tags enforced.
    Artifact: a test case per tag location, in CI or in a scheduled manual run.

  4. Test the no-database and router configurations.
    Why: these are the paths where the enforcement code is easiest to lose.
    Artifact: one rejected request and one recorded response for each configuration.

  5. Reconcile the provider invoice against gateway spend on a schedule.
    Why: the invoice is the only independent check that the tracker and the gate agree with reality.
    Artifact: a recurring reconciliation with an alert on the gap.

  6. Keep a kill switch that does not depend on the budget code.
    Why: if the gate is what failed, the recovery path must not run through it.
    Artifact: a documented revoke-key / disable-route procedure that has been exercised.

  7. Re-run the matrix after every upgrade.
    Why: the vendor fix for a past gap does not guarantee the next path is covered.
    Artifact: a new dated matrix entry for each deployed version.


If you run an AI gateway in front of production traffic and cannot yet show a rejection at the budget boundary, get in touch. A 30-minute assessment is enough to walk the request path together.