How to Cut Your LLM Bill Without Losing Quality

The bill for an AI feature has an unusual shape. It does not stay flat like a software license. It grows with every user, every message, and every extra paragraph someone adds to the system prompt. A feature that cost a few hundred dollars a month in beta can cost many times that after launch, and the first sign is usually an invoice.

Most of that growth is avoidable. This guide covers where LLM spend actually goes, the five changes that usually cut it the most, and the step most cost advice skips: proving that the cheaper setup still gives your users the right answer.

Where the money goes

LLM providers charge per token, and they charge differently for different kinds of tokens. Input tokens (everything you send) are priced separately from output tokens (everything the model writes back), and output is usually several times more expensive per token. Many providers also price cached input, batch requests, and reasoning tokens on their own lines.

That means the same feature can be cheap or expensive depending on details nobody looks at. Before changing anything, break the bill down three ways:

• By feature. Which part of the product sends the requests? One feature often accounts for most of the spend.
• By model. Which model handles each request, and does that request actually need it?
• By token type. How much is input, how much is output, and how much of the input is the same text sent over and over?

Your provider's usage export or a sample of request logs is usually enough to do this. The breakdown alone often points straight at the fix.

1. Stop using the top model for easy requests

The most common waste is sending every request to the most capable model. Classifying a support ticket, extracting a date from an email, or rewriting a sentence rarely needs it. Smaller models in the same family are much cheaper per token and often just as accurate on narrow tasks.

The fix is routing: send each type of request to the cheapest model that handles it well. That can be as simple as choosing a model per feature, or as involved as a small classifier that decides per request. Either way, the routing rule has to be tested, which we cover below.

2. Use prompt caching

Many features send the same long block of text on every call: a system prompt, a product catalog, a policy document, a set of examples. Without caching, you pay full price for that block every single time.

OpenAI, Anthropic, and Google all offer prompt caching, which bills repeated input at a fraction of the normal rate. The rules differ by provider. Some cache automatically, some need you to mark what to cache, and caches expire after a set time. The general rule holds across all of them: put the stable content first, put the part that changes last, and keep the stable part byte-for-byte identical between calls.

3. Shorten what you send

Prompts tend to grow. Every bug fix adds a sentence, every edge case adds an example, and nobody removes anything. A prompt that started at 300 words ends up at 3,000, and every word is billed on every call.

Go through the prompt and cut instructions the model already follows without being told, examples that duplicate each other, and context the request does not use. Do the same for retrieved documents: sending the three most relevant passages instead of the top twenty can cut input cost sharply and often improves answers, because the model has less noise to sort through.

4. Batch the work that can wait

Not every request needs an answer in two seconds. Nightly report generation, bulk tagging, back-filling summaries, and evaluation runs can all wait hours. OpenAI and Anthropic both offer batch processing at roughly half the normal price in exchange for results within 24 hours, and Google offers a similar discount.

Look at your request logs for work that runs on a schedule or in bulk. If no person is waiting on the result, it is a candidate for batching.

5. Cap and control output

Because output tokens usually cost the most, a model that writes long answers when short ones would do is expensive. Set sensible maximum lengths, ask for structured output when you only need a few fields, and tell the model what length you expect. For reasoning models, check whether a lower reasoning setting gives the same quality on your task.

The step most advice skips: proving quality holds

Every change above can make answers worse. A smaller model may miss a nuance. A shorter prompt may drop the one instruction that prevented a known failure. Routing may send a hard question down the cheap path. Cost dashboards will show you the savings. They will not show you the support tickets that follow.

So before switching anything in production, test it:

• Build a test set from real traffic. Pull 50 to 150 real prompts from your logs, chosen to cover the common cases and the hard ones.
• Write down what a good answer is. For each prompt, or each type of prompt, state what a correct answer must include and must not include. Without a written standard, "it seems fine" becomes the bar.
• Run every candidate setup against the set. Score each one for quality and record its cost per thousand requests. Reject any setup that fails, even if it is cheaper.
• Keep the test set. Rerun it whenever a prompt, model, or provider changes, so a future update does not quietly undo the work.

Test runs like this usually cost a few dollars in API fees. They are the difference between a saving you can trust and one that comes back as churn.

Free can be an option too

For some features, the cheapest model is one with no per-request bill at all. When we built the Reductivist Shop Assistant, it ran on Cloudflare's free Workers AI allocation, with daily limits that kept it there, so the store pays nothing to run it. That works because the assistant answers a narrow set of questions from the store's own data, and the model never has to state a price itself. Not every feature fits that shape, but more do than teams expect.

Where to start

If you only do one thing this week, export last month's usage and break it down by feature and model. You will likely find that one or two features drive most of the bill, and that at least one of them uses a bigger model than it needs.

If you want the full process done for you, our LLM cost optimization service starts with a cost audit that puts a dollar figure on each change, then tests every candidate on your own prompts before anything ships.