LLM token prices: for the same volume, the cost can vary from 1 to 25

Key Takeaways

• For 20,000 monthly conversations, the cost of aKnowledge Agent (RAG) ranges from €120 to €2,987 per month, depending on the model chosen: a ratio of approximately 1 to 25.
• The issue: separate billing for input and output. Generated tokens generally cost 3 to 6 times more than read tokens, and as much as 8 to 9 times more on certain models. “Thinking” models widen the gap even further: their reasoning—which is invisible to the user—is billed at the output rate.
• Without taking the prompt cache into account, a comparison overestimates the actual cost: in our simulation, the cache reduces the cost by about 35% based on the RAG architecture. However, it is still necessary to understand its rules: a minimum size threshold and a limited lifespan.
• The per-token pricing model alone is not enough to choose a solution: compliance with the AI Act, data sovereignty, and integration with the accounting system are just as important.
• There are five ways to keep costs down: routing based on complexity, caching, batch processing, optimized drivers, and regular reviews of rates.

As of June 1, 2026, all GitHub Copilot plans are billed on a pay-as-you-go basis. The flat-rate subscription offering unlimited access to AI is no longer available: each token is now counted and billed separately. Beyond being a mere footnote, this change marks a significant shift.

In 2025, generative AI was still considered an almost free resource that could be deployed without any real financial oversight. In 2026, the reality check was harsh: “AI for developers” budgets were exhausted in a matter of months, and finance departments realized—often too late—that the cost of tokens was rising faster than expected.

In this context, any company rolling out an AI agent or assistant must ask itself a simple question: Does the choice of model actually affect the cost? To answer this question with data rather than intuition, we simulated the actual cost of aKnowledge-Based AI Agent (RAG)across ten models, all with strictly identical usage volumes.

Input, Output, Reasoning: How Is a Token's Price Calculated?

To compare plans, you first need to know what you’re paying. All providers use two separate pricing structures:

• input tokens, that is, everything that is passed to the model: the question, the context, and the retrieved documents;
• output tokens, in other words, the text generated by the model.

Generating a response token by token requires more computational power than simply reading a prompt. The output is therefore always more computationally expensive than the input: generally 3 to 6 times more, and up to 8 or 9 times more for certain models.

For RAG (Retrieval-Augmented Generation) architectures, this asymmetry has a direct impact. Each request sends a large number of tokens as input (question, history, knowledge base excerpts) to obtain a response that is usually brief. The largest cost item is therefore not necessarily the one you might expect.

The Special Case of “Thinking” Models

This logic is reversed with so-called “thinking” or “reasoning” models. When a model reasons before formulating its response, the user sees none of that reasoning, but it is still billed: each reasoning token counts as an output token, at the full rate.

A response expected to be 100 words long may thus be preceded by several minutes of silent “reflection”—the equivalent of 2,000 words of hidden reasoning. The total output is therefore 2,100 words, not 100.

When used in conversation, caution is warranted. In our simulation (detailed below), approximately 2,000 tokens of hidden reasoning per RAG call are enough for the cost of the output to exceed that of the input: the balance shifts, and the input—which had been the primary cost—takes a back seat. In customer service, where an immediate response is expected, this type of in-depth reasoning is rarely justified. Moreover, some providers do not guarantee completely transparent reasoning, even when explicitly requested.

Simulation: from 1 to 25 for the same volume of conversations

To make an objective comparison, we defined a typical usage profile for a knowledge-based AI agent (RAG) and applied the same data volumes to ten models currently on the market:

• via RAG call: 10,000 tokens in and 250 tokens out;
• An average of 2.5 RAG calls per conversation;
• 3 additional requests per conversation, each with 1,000 input tokens and 40 output tokens;
• 20,000 conversations per month.

From one line to the next, only the model's price varies. Here are the results, based on the official prices from August 2026 converted at a rate of $1 = €0.92:

The result is clear: for the same volume, the most expensive model costs nearly 25 times as much as the most affordable one. For an AI agent handling 20,000 conversations per month, the difference amounts to approximately €2,870 per month—solely due to the choice of model. Over twelve months, this difference turns a negligible budget line item into a significant expense.

However, the cheapest model isn’t necessarily the best choice by default. A “nano” or “flash-lite” model is less sophisticated than a flagship model, and an approximate answer to a complex case often costs more in customer dissatisfaction than it saves in tokens. Above all, the simulation shows that choosing a model is by no means a technical detail: it’s a budgetary decision that must be made with full knowledge of the facts rather than simply accepted.

Be aware of follow-up requests

There is a pitfall in this type of simulation: the size of follow-up requests. If a request is short (around 1,000 tokens), it generally does not reach the threshold that makes it eligible for caching (see below). It is therefore billed at the full rate each time it is sent.

The result: in practice, a request for 1,000 tokens can cost as much as a request for several thousand tokens, which benefits from caching. Each scenario must therefore be examined line by line before a reliable price comparison can be made.

The prompt cache—a major omission from the comparisons

Even when conducted rigorously, a comparison of this type almost always overestimates the observed production costs. This is because it applies the full rate to each call, whereas a large portion of the prompt used by a knowledge-based AI agent (RAG) is repeated—identically or nearly so—from one call to the next.

Prompt caching, now offered by most major providers, takes advantage of this repetition. When a prompt is sent for the first time, a block of context is stored in the cache; subsequent calls that reuse that same context are billed at only a fraction of the standard entry rate.

A prefix-based comparison mechanism

The cache is based on a prefix comparison: the model takes the query starting from its first token and compares it, token by token, to what it has already processed. As long as the content matches, that portion is considered “in cache.” At the first mismatched token, the comparison stops: everything that follows is charged at the full rate, even if the rest of the prompt is identical to a previous request.

That is why variable elements (RAG search results, conversation history, contextual data) override the cache as soon as they appear at the beginning of the prompt. Conversely, a stable foundation placed at the very top of the query (system instructions, a knowledge base that changes infrequently) takes full advantage of it.

A minimum threshold of 1,024 tokens at OpenAI

Another rule that is often overlooked: the common portion must be at least a certain size. At OpenAI, for example, it must exceed 1,024 tokens to qualify for the cache rate. If it’s smaller than that, it isn’t taken into account and is charged at the full price, even if it’s an exact duplicate of a previous prompt.

This results in a particularly costly range, between approximately 800 and 1,024 tokens: these requests drive up costs without being able to benefit from the cache. Best practice: when a prompt approaches this threshold, expand it slightly so that it clearly exceeds it, provided that the request is actually reused during the cache’s lifetime.

A runtime of 10 to 15 minutes

The cache is also temporary: with most providers, it expires by default after about 10 to 15 minutes. If no new request reuses the context within that time frame, it disappears and must be recreated on the next call.

Some providers offer extended retention, usually for an additional fee charged at the time of writing: an attractive option for use cases where the context varies very little. The terms and conditions vary from one provider to another.

A cache read is about 10 times cheaper

At both Anthropic and Google, retrieving cached content costs about 10% of the standard input rate—a discount officially documented by both providers; the order of magnitude is comparable at OpenAI. On GPT-5.1, for example, a non-cached input costs about €1.2 to €1.3 per million tokens, compared to €0.13 to €0.14 when cached—a ratio of nearly 10.

When applied to Claude Sonnet 5 in our simulation—with a cache write for only one of the 2.5 RAG calls in the conversation and a cache read for the subsequent ones—the actual cost per conversation decreases by approximately 35% compared to the raw calculation. A comparison that ignores the cache therefore risks exaggerating the actual gap between two solutions and, to the same extent, underestimating the potential for optimizing an existing architecture.

The actual reuse rate depends largely on the architecture. An AI agent whose document context changes very little (FAQs, stable knowledge base) benefits much more from caching than an assistant that draws on different documents for each query. This factor must be measured precisely before making any budget decisions based on a price comparison.

Beyond Price: The Criteria That Really Make a Difference

The per-token pricing model is just one factor among many. An MIT study (the NANDA initiative), which received widespread attention in 2025 and was still cited as a reference in 2026, found that 95% of generative AI pilot projects in the corporate sector did not yield a measurable return on investment.

The cost of the models wasn’t the issue: the main obstacles were unclear objectives, insufficient user adoption, and inadequate integration into business workflows. Optimizing only the price per token—without clarifying these issues beforehand—is like tackling the wrong variable.

When choosing a plan for a business, four criteria are just as important as the price:

• Regulatory Compliance. With the gradual implementation of the European AI Act, requirements for transparency, traceability, and human oversight vary depending on the use case. A less expensive but poorly documented model may expose an organization to compliance risks that far outweigh the cost savings achieved.
• Data sovereignty and hosting. For organizations subject to strict GDPR requirements, the origin and location of data processing (hosting in Europe, no reuse of data for training purposes) are often just as important as price.
• Integration with the information system. A model that connects natively to your tools (CRM, knowledge base, line-of-business applications) minimizes hidden integration costs, which often exceed the cost of the tokens themselves over time.
• Reliability for your specific use cases. A high ranking on a general-purpose benchmark does not guarantee performance in your specific scenario. The best indicator is still a comparative test using a representative sample of your queries.

Managing Your Token Costs: 5 Practical Strategies

Regardless of the model chosen, there are several ways to reduce spending without compromising service quality:

1. Route each request based on its complexity. Flagship models (Opus, GPT-5.5, Gemini Pro) are reserved for complex requests; simple tasks (FAQs, classification, extraction) are handled by cost-effective models such as Haiku, GPT-5.4 nano, or Gemini Flash-Lite.
2. Cache the context as soon as the system prompt or knowledge base varies little from one request to the next. Plan for this during prompt design: include a fixed portion of context that exceeds the provider’s eligibility threshold (1,024 tokens for OpenAI), so you don’t have to pay the full rate for content that’s almost long enough to be cached.
3. Use batch processing (Batch API) for anything that doesn't require a real-time response: most providers offer a 50% discount for this method.
4. Measure before rolling out on a large scale. A limited pilot, with careful tracking of actual token consumption, helps avoid unpleasant surprises once the rollout is extended to the entire organization.
5. Regularly reassess the price-performance ratio. Rates change quickly (all providers revised them several times in 2026), and the plan that’s best today won’t necessarily be the best in six months.

Choosing a model: a strategic decision

The token economy is no longer just a concern for technical teams. It is now a strategic decision, just like choosing a cloud provider or business software. Our simulation shows that, for the same use case, the chosen model—among those studied—can increase the cost by a factor of 25. But the listed price isn’t the whole story: prompt caching, regulatory compliance, and suitability for the use case are just as important.

At Zaion, we support you every step of the way: from designing user journeys to scaling agent-based AI, with clear governance and measurable, long-term results.

Would you like to get an accurate estimate of the cost of an AI agent project for your organization? Request a demo.