What an LLM Feature Really Costs to Run for a Year

Short answer: For most internal LLM features, the model bill is smaller than people fear. In our three examples below, a year of usage costs from about $130 to about $66,000, depending mostly on which model you pick. Model choice moves the bill by roughly 100x. Volume, prompt length and caching move it next.

Prices as of: 25 October 2026 (LiteLLM price file), cross-checked against official provider pricing pages on 26 October 2026. All prices are in US dollars per million tokens. Prices change often, so check again before you budget.

The three models we used

We picked one cheap, one mid and one top model. All prices come from the LiteLLM price file that powers our LLM token cost calculator. We checked each one against the provider's own pricing page. They matched.

A few notes from the official pages:

• Claude Haiku 5.5 has the same rates as GPT-6 luna for prompts up to 100K tokens. Above that, Anthropic charges $0.50 input and $2.50 output.
• GPT-6.1 sol has the same list rates as Claude Sonnet 5.5.
• Anthropic also charges for cache writes ($2.50 per million on Sonnet 5.5, 5-minute cache). We left this out of the examples to keep the arithmetic simple. Writes happen once per cache refresh, so they are small next to reads at steady volume.
• Google's Gemini 4 Argon is $2 input and $10 output until 30 November 2026. From 1 December 2026 it doubles to $4 and $20. This is a good example of why "prices as of" matters.

The formula for every example is the same:

Cost = (uncached input tokens x input price) + (cached input tokens x cached price) + (output tokens x output price), with tokens counted in millions.

Example 1: Internal support chatbot

Staff ask an assistant about HR, IT and policy questions. The system prompt and policy text repeat on every call, so most input can be cached.

Monthly volume: 2,000 x 30 = 60,000 requests.

• Input: 60,000 x 3,000 = 180M tokens. Cached: 108M. Uncached: 72M.
• Output: 60,000 x 400 = 24M tokens.

Notice that output is half or more of the bill on every model. Output tokens cost five times as much as input here.

Example 2: Document Q&A (RAG)

Analysts ask questions about contracts and reports. The system retrieves relevant passages and sends them with each question. Retrieved text changes every time, so we assume no caching.

Monthly volume: 500 x 22 = 11,000 queries.

• Input: 11,000 x 7,200 = 79.2M tokens.
• Output: 11,000 x 500 = 5.5M tokens.

Here input dominates. The retrieved context is the cost driver. Sending 3,000 tokens of context instead of 6,000 would cut the input bill by about 40%.

Example 3: Batch summarisation job

A nightly job summarises incoming documents. Nobody waits for the answer, so it can use the batch APIs. Anthropic, OpenAI and Google all list batch prices at half the standard rate for these models.

Monthly volume: 400M input tokens and 30M output tokens.

Without batch, the yearly figure on GPT-6 astra would be $66,000. Batch halves it for the cost of waiting.

The costs people forget

The token bill is only one line. Budget for these too. We give you a way to estimate each, not a made-up total.

• Embeddings. RAG needs every document turned into vectors, plus each query. Estimate: total tokens in your corpus, plus re-embedding when documents change, times the embedding price. As listed on the official pages, OpenAI text-embedding-3-small is $0.02 per million tokens and Google's Gemini Embedding 2 is $0.20 per million for text.
• Vector database. Hosting cost depends on vector count, dimensions, replicas and query rate. Get a quote from your vendor or price the cloud instance you would run it on.
• Retries and guardrail calls. Timeouts, failed JSON parses and safety checks each add calls. Estimate a multiplier from your logs, for example 1.1x if one in ten requests is repeated. Add any classifier or moderation model as its own line.
• Logging and monitoring. Storing prompts and responses for audit grows fast. Estimate log volume per request times retention period, then price it in your log platform.
• Evaluation runs. Every prompt change or model upgrade should be tested against a fixed question set. Estimate: test set size x tokens per test x runs per month, priced like production. LLM-as-judge scoring adds a second model call per test.
• People time. Someone reviews flagged answers, updates prompts and handles complaints. Estimate hours per week x loaded hourly rate. For many teams this is the largest line.

Our AI project cost estimator helps you put these categories next to build cost.

Cost levers that work

1. Pick the right model, then route. The gap between cheap and top tier is about 100x in our examples. Send easy requests to a small model and only hard ones to a large one.
2. Use prompt caching. In Example 1, cached input costs a tenth of fresh input on GPT-6 luna and a twentieth on Claude Sonnet 5.5. Put stable text (instructions, policies) at the start of the prompt so it can be cached.
3. Shorten prompts. Trim long instructions, old chat history and extra retrieved passages. In RAG, fewer and better chunks often help answers too.
4. Use batch APIs for anything that can wait. The official pages list a 50% discount on Anthropic and Google, and OpenAI's batch prices are half of standard for the models we used.
5. Cap output length. Output tokens cost five times input on all three models. Set a max token limit and ask for short, structured answers.

FAQ

How do I redo these numbers for my own feature?

Count requests per month, average input and output tokens per request, and the cached share. Multiply by the per-million prices. Our token cost calculator does this for you.

Why does the top model cost so much more?

Its list price is 100 times the cheap model for both input and output. The answer quality may justify it for hard tasks. Test before you decide.

Are these prices final?

No. Providers change prices and add tiers often. Gemini 4 Argon, for example, doubles on 1 December 2026. Recheck the official pages before signing off a budget.

Does prompt caching always apply?

No. It only helps when the start of the prompt repeats. Rules on minimum length, cache lifetime and write charges differ by provider.

Is the model bill the biggest cost?

Often not. For many internal tools, review time, evaluation and monitoring cost more than tokens. Budget for all of them.

Sources

1. LiteLLM model prices and context window file (MIT licence), retrieved 25 October 2026
2. OpenAI API pricing, checked 26 October 2026
3. Claude pricing (Anthropic), checked 26 October 2026
4. Gemini Developer API pricing (Google), checked 26 October 2026
5. ZigmaNeural LLM token cost calculator
6. ZigmaNeural AI project cost estimator