LiteLLM fail-open
How spend tracking can keep working while budget enforcement skips a request path, the matrix to test, and the evidence that proves a limit actually rejects.
On this page
A budget that records spend but does not reject the request is a dashboard, not a control.
This note is the operator’s version: which request paths to exercise, what a safe reproduction looks like, and what evidence to demand before trusting a spend limit in production. Everything about LiteLLM’s behaviour below is attributed to its issue tracker, pull requests or release notes; the testing matrix, ordering and recommendations are our reading.
Tracking is not enforcement
LiteLLM does real spend accounting. That is a different subsystem from the code that decides whether a request may proceed.
The failure mode to watch for is quiet: the request succeeds, the provider is called, the spend is logged — and the limit that was supposed to stop the request never ran on that path.
That can happen when the gate and the tracker read different representations of the request. Tags are the obvious example, because they can arrive in more than one place. The same shape appears with customer identifiers, virtual-key scopes, and any configuration where the database or routing layer that enforcement expects is not the one in the request’s path.
The historical example, from LiteLLM issue #27480: tag-budget enforcement was skipped when tags arrived through the x-litellm-tags header path, while body tags enforced correctly. Spend still recorded. Fix #27573 landed on 13 May 2026 and the v1.85.0 release notes list header-based tag-budget enforcement among the fixes.
That is a record of a past bug, not a claim about any current release. The operator’s lesson is version-independent: a green budget dashboard proves recording, not rejection.
A budget that cannot reject on every path that can carry its identifier is a monitoring feature with a limit field.
The paths to test
Pick the paths your deployment actually exposes. Not every row applies to every version or configuration, and the field names move between releases.
| Path or mode | What the test must prove |
|---|---|
| Tag budget via body tags | The request is rejected once the agreed spend is reached |
| Tag budget via metadata | Same rejection as the body path |
Tag budget via x-litellm-tags header |
Same rejection as the body path |
| Customer / end-user budget | The identifier used for accounting is the one enforcement reads |
| Virtual-key budget | A key over its limit cannot reach the provider |
| Router or fallback configuration | Every deployment in the route is covered |
| Budget with no spend database configured | The failure is explicit, not silent |
| Streaming, retries and async logging | A retry or a stream does not slip past the gate |
| Direct provider access outside the proxy | The bypass route is blocked or accepted as a documented risk |
For each row, the pass condition is the same: a rejected request, a clear error, and no provider call.
A safe reproduction
Do this in an isolated environment with sandbox credentials, a throwaway database, and a budget small enough to trip in one request. Record the gateway version and configuration for every run.
# 1. Start an isolated proxy with a throwaway spend database and a sandbox upstream.
# 2. Attach a tag budget with one approved request, then send the same request
# with the tag in each supported location.
curl -sS -D - -o /dev/null "$LITELLM_BASE/chat/completions" \
-H "Authorization: Bearer $SANDBOX_VIRTUAL_KEY" \
-H "Content-Type: application/json" \
-H "x-litellm-tags: budget-probe" \
-d '{"model":"sandbox-model","messages":[{"role":"user","content":"budget probe"}],"metadata":{"tags":["budget-probe"]}}'
# 3. Reset the spend between runs, repeat with the tag only in the body and only
# in metadata, then repeat after a proxy restart.
# 4. After each run, read the recorded spend for the tag and compare it with the
# HTTP response. Rejected request, unchanged spend: enforced.
Adapt paths and field names to the documentation for your pinned release; do not copy them between versions.
The useful output is a dated table: request variant, expected result, HTTP status, whether the upstream saw the call, and the spend movement. That table is what an auditor or a buyer can check.
What enforced looks like
- The request is rejected before the provider call, and the upstream’s own logs show nothing arrived.
- The error is explicit: a status code and a message that names the limit.
- Recorded spend does not increase for the rejected request.
- Body, metadata and header paths behave identically, before and after a restart.
If any of those is false, the budget is advisory on that path.
Limits of this note
The issue and fix links describe a historical, version-specific gap. This note does not claim a current vulnerability, and the reproduction was not re-run for publication. The matrix is a testing method, not a substitute for your own measurements. Gateway behaviour depends on the deployed version and configuration; treat every release note that mentions budgets as a reason to retest, not a reason to relax.
What to do, in order
-
Pin the proxy version and attach the test evidence to it.
Why: enforcement paths have changed between releases, and a release note is not a test result.
Artifact: a dated matrix run next to the deployed version and configuration. -
Name one owner per control point: virtual key, team, tag, customer.
Why: a limit with no owner is a limit nobody rechecks.
Artifact: an ownership list that matches the budget configuration. -
Include the header and metadata tag paths in the regression test.
Why: the historical fail-open was exactly a header-path gap while body tags enforced.
Artifact: a test case per tag location, in CI or in a scheduled manual run. -
Test the no-database and router configurations.
Why: these are the paths where the enforcement code is easiest to lose.
Artifact: one rejected request and one recorded response for each configuration. -
Reconcile the provider invoice against gateway spend on a schedule.
Why: the invoice is the only independent check that the tracker and the gate agree with reality.
Artifact: a recurring reconciliation with an alert on the gap. -
Keep a kill switch that does not depend on the budget code.
Why: if the gate is what failed, the recovery path must not run through it.
Artifact: a documented revoke-key / disable-route procedure that has been exercised. -
Re-run the matrix after every upgrade.
Why: the vendor fix for a past gap does not guarantee the next path is covered.
Artifact: a new dated matrix entry for each deployed version.
If you run an AI gateway in front of production traffic and cannot yet show a rejection at the budget boundary, get in touch. A 30-minute assessment is enough to walk the request path together.