Soft caps vs hard caps
Every paid plan includes soft caps: spend is tracked as it happens, and calls are blocked the moment a budget is hit. Simple, instant, and works with every model and workload. Because the check happens at the start of each call, a burst of calls firing at the same instant can push you slightly over before the cap catches up.
Hard caps close that gap. They reserve each call's worst-case cost before the call runs, so no matter how many calls arrive at once, they can't collectively exceed the limit. Hard caps are an add-on you can switch on for any paid plan — for the budgets where going over isn't an option.
Reserve, call, settle
Hold the worst case
Before the call, we compute its worst-case cost and atomically hold it against the budget. If there's no headroom, the call is blocked — nothing is spent.
Make the call
You make the AI call exactly as you do today. The reservation protects the headroom while the call is in flight, so the limit can't be breached in that window.
Reconcile to actual
After the call you send the real token counts. We look up the exact cost from our pricing table, replace the hold with that amount, and release any unused budget back to the cap instantly.
Because the reserve step is atomic, concurrent calls are handled one after another — each reservation accounts for the one before it. That's what makes the limit hold under load, where a simple "check then spend" approach would let simultaneous calls all pass and overshoot together.
The integration
Two calls around your existing AI call: one to reserve, one to settle. Both go to POST /v1/events — the same endpoint you already use for monitoring events.
// 1. Reserve the worst-case cost before the call
const reserve = await fetch("https://api.tokencapai.com/v1/events", {
method: "POST",
headers: {
"Authorization": `Bearer ${TOKENCAP_KEY}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
reserve: true,
agent_id: AGENT_ID, // your agent's UUID from the TokenCapAI dashboard
model: "gpt-4o",
input_tokens: 1200,
max_tokens: 800 // the ceiling we reserve against
})
}).then(r => r.json());
if (!reserve.allowed) {
// Budget has no headroom — do not proceed with the LLM call
throw new Error(reserve.reason);
}
// 2. Make your LLM call exactly as you do today
const completion = await openai.chat.completions.create({ /* … */ });
// 3. Settle — send actual token counts, we compute the cost and release unused budget
const settled = await fetch("https://api.tokencapai.com/v1/events", {
method: "POST",
headers: {
"Authorization": `Bearer ${TOKENCAP_KEY}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
agent_id: AGENT_ID,
model: "gpt-4o",
reservation_token: reserve.reservation_token,
input_tokens: completion.usage.prompt_tokens,
output_tokens: completion.usage.completion_tokens
})
}).then(r => r.json());
// → { allowed: true, event_id: "...", enforcement_degraded: false }
// Settle is idempotent — safe to retry on network timeout; we'll never double-count.You send tokens. We compute cost. At settle time we look up the exact rate for your model from our pricing table — the same table we use at reserve time — and apply it to the actual token counts. You never have to calculate a cost figure yourself. If your model isn't in our table, pass cost_usd explicitly as a fallback and we'll use that instead.
Reserve high, settle exact
The reservation is sized on max_tokens — the ceiling you declared. Because most calls use far fewer output tokens than their declared maximum, the settled cost is almost always lower than the reservation. That gap is returned to the cap immediately on settlement, so a conservative reservation doesn't eat into your budget for subsequent calls.
If you don't settle within the reservation window (the TTL — 300 seconds by default, configurable per organisation), the hold is released automatically and the budget returns to the cap. The audit record is always written either way, so the log is always complete.
The allowed: false check on the reserve response is essential — if the budget is exhausted, don't proceed. The response includes a reason string (e.g. global_daily_cap_exceeded) you can surface to your own users.
What it guarantees
That concurrent calls cannot collectively exceed the cap. This is the failure that catches naive spend limits: two calls both see "budget available," both proceed, and together they bust the limit. The atomic reserve serialises that check, so the cap holds under any amount of parallelism. It also reconciles every call to its true cost on settlement, so your remaining budget is always accurate.
Where it's tightest — and where it isn't yet
The guarantee is precise, not absolute, and we'd rather tell you exactly where the edges are than oversell it.
The reservation is sized on the input token count you supply and the maximum output you declare — we trust those numbers directly and don't re-derive them. For standard text generation — the majority of calls — that's a true ceiling on cost, and the hard cap is at its tightest. Two cases are looser today:
- Reasoning models (the o1 family, extended-thinking modes) generate internal reasoning tokens that are billed but aren't bounded by your output limit. The worst-case estimate can under-reserve for these, so a single call can settle slightly over. We recommend a conservative buffer on the cap for reasoning-heavy workloads for now.
- Image and multimodal calls. Reservation cost is computed from the token counts you supply; we trust that number and don't re-inspect the call. For multimodal calls you must include image tokens in the supplied count, and we haven't yet end-to-end validated this path — so for image-heavy workloads we suggest soft caps or a manual buffer until we have.
FAQ
What happens to a call that exceeds its reservation?
What if I forget to settle?
Can I mix soft and hard caps?
reserve: true in your preflight call.Can I use Reservation Mode with the proxy integration?
POST /v1/events. The reserve-before/settle-after protocol requires your code to control exactly when the LLM call is made, which isn't possible when traffic routes through the proxy. The proxy uses a soft-cap model: it checks spend before forwarding the call, but cannot atomically hold budget across the LLM round-trip.Does this add latency?
What does enforcement_degraded: true mean?
Is this good enough for a regulated workload?
On the roadmap
We're actively working to tighten the worst-case estimate so the guarantee strengthens over time. Planned, not yet shipped:
- Thinking-aware reservations — pass a
thinkingobject at reserve time (model type, adaptive vs manual, effort level or token budget) so the held amount reflects your actual reasoning configuration rather than the fullmax_tokensceiling. Covers Claude adaptive thinking, OpenAI o-series reasoning, and Gemini thinking mode. Settlement already prices correctly today —output_tokensfrom the API total includes reasoning tokens — so this is purely a reservation accuracy improvement. - Reasoning-model characterisation — measuring, with live calls, how far reasoning models exceed a max-tokens-based reservation at each effort level, so adaptive reservations can use a tight p99 ceiling rather than the conservative maximum.
- Per-model worst-case estimation — computing reservations differently for reasoning-capable and multimodal models, including model-aware thinking token pricing for providers (e.g. Gemini) that bill reasoning at a different rate than output tokens.
- Strict mode — an optional safety buffer that reserves above the naive worst case for models known to bill beyond their output tokens, trading a little throughput for a tighter ceiling.
- Validated multimodal path — end-to-end testing of image and multimodal reservations, so those workloads reach the same tightness as text.
If one of these matters to your use case, tell us — real demand moves it up the list.