Guide

What makes an AI request cost more?

Work out the token bill for a book-swap question box, then compare caching, batching, and sending less.

October 28, 2026 · 6 min read

If a subscription covers your chat, what does each request cost to run? Much of the bill comes from how much text goes in and comes out. Counting that text lets you work through most AI pricing pages.

What a token is

A language model reads and writes text in pieces called tokens. A token can be a whole word, part of a word, or a punctuation mark. The how AI answers work guide shows the model producing an answer one piece at a time.

Each provider splits text its own way, so counts differ between models. Google's token guide gives a rule of thumb for its models: "a token is equivalent to about 4 characters," or roughly 60 to 80 English words per 100 tokens. Images and audio are counted in tokens too. On that page, a small image counts as 258 tokens.

Providers usually sell developers tokens per million at 2 rates. Input tokens include your question, documents, product instructions, and earlier messages. Output tokens are what the model writes back, usually costing several times more than input.

On October 28, 2026, Anthropic's pricing page listed one mid-priced model at $2 per million input tokens and $10 per million output tokens. We'll use those example rates. OpenAI's pricing page and Amazon Bedrock's pricing page use the same per-million units with separate input and output prices. Prices change often. Recheck these dated examples before budgeting.

Why a conversation gets more expensive as it goes

A model does not remember between turns. Most products resend the conversation with your new message, including the model's last answer as input.

A 10-turn conversation therefore does not cost 10 times one turn. Turn 10 pays for turns 1 through 9 again as input. A long document pasted at the start is paid for on every turn after it.

The context window is the most text a model can work with in one request, counting its own reply. Anthropic's context window page lists 1 million tokens for its current models. Filling that window is allowed and billed at the token rate. The page also notes that accuracy can drop as context grows. More room need not mean better answers.

You may never see some billed text. Product instructions and descriptions of tools, such as search, count as input. Anthropic's pricing page says enabling tools adds a few hundred tokens of hidden instructions per request. Models that "think" before answering bill that reasoning as output.

Follow one example

In this fictional example, written for the guide, a neighborhood book swap adds a question box to its page. Each short answer uses a 3,000-token rule sheet covering dates, the book limit, and accepted books, plus the member's question.

Input per question:   3,000 (rules) + 50 (question) = 3,050 tokens
Output per answer:    150 tokens

Input cost:   3,050 / 1,000,000 x $2   = $0.0061
Output cost:    150 / 1,000,000 x $10  = $0.0015
Per answer:                              $0.0076

That is under 1 cent an answer, or about $1.52 for 200 questions a month. Most of the bill pays for resending the rule sheet.

Now take swap day, when 60 questions arrive in 2 hours. Without changes, that day costs about $0.46.

Reduce the bill

Start with the part that repeats. Prompt caching stores a fixed opening, such as the rule sheet, so later requests can reuse it at a discount. Anthropic's caching guide prices the first write at 1.25 times the input rate and later reads at a tenth of it or less, depending on the model. The cache lasts 5 minutes by default, and the clock restarts each time it is used. Amazon Bedrock describes a similar feature in its prompt caching guide.

At the example rates, the dated pricing page lists $2.50 per million for the cache write and $0.10 per million for a cache read. On swap day:

First question:   3,000 x $2.50/M + 50 x $2/M + output $0.0015 = about $0.0091
Each later one:   3,000 x $0.10/M + 50 x $2/M + output $0.0015 = about $0.0019
Day total:        $0.0091 + 59 x $0.0019                        = about $0.12

The day drops from about $0.46 to about $0.12. The cache only helps because the questions arrive close together. Scattered across a month, most would arrive after the cache expired, and each would pay the higher write price instead.

After caching, output is most of the bill. A request for shorter answers saves more than another input trick.

Next, batch what can wait. Batch processing sends many requests together and returns results later. Anthropic's batch guide cuts both input and output prices by 50 percent, with most batches done in under an hour. Google's Gemini pricing page lists a 50 percent batch reduction, and Bedrock offers batch prices on selected models. A live question box cannot wait. A weekly summary can.

Then send less. A plain rule or a search can answer some questions before a model sees them. "When is the next swap?" can be answered from the calendar. Should a model make this decision? and writing an answer or choosing one cover when ordinary software fits better.

Last, pick the cheapest model that passes. Write 20 real questions with answers you have checked. Try the cheapest model first and move up only when it fails that check. Compare with your own token mix: a model with cheap input and expensive output can cost more for long replies. Some list prices also change with size. On the dated Anthropic page, one small model charges more once a single prompt passes 100,000 tokens.

Costs around the model

A retry means paying twice for a failed or rejected answer. An agent making 20 model calls for 1 task resends growing context each time, so limit its steps.

A public question box may also pay per request for the web gateway, server code, and logs. Someone still checks answers that matter. In most small projects, that time costs more than the tokens.

For anything public, price the worst case. Assume someone sends questions as fast as your limit allows for a whole month. If the swap's box allowed 1 question per second, that is 2,592,000 questions in 30 days. At $0.0076 each, about $19,700; even with a warm cache, near $4,900. The fix is a lower limit or a switch that stops the requests themselves. Turning off the model alone may leave the request charges running.

This task is also an illustration. Each week, the book swap turns about 40 member messages into a newsletter. The messages total 6,000 tokens, the instructions add 1,000, and the newsletter comes out at about 1,000 tokens. Use the example rates of $2 and $10 per million.

What does one newsletter cost, and how would you cut that in half with the same model?

Answer: input is 7,000 tokens, or $0.014. Output is 1,000 tokens, or $0.010. Each newsletter costs about $0.024.

Caching the instructions is a reasonable first guess, since they repeat every week. It barely helps here. The instructions are 1,000 of 7,000 input tokens, and a weekly request arrives long after a 5-minute or 1-hour cache has expired.

The newsletter can wait, so send it as a batch. Half off both input and output brings it to about $0.012. Before any real budget, recheck the batch discount and current rates on the provider's pricing page.

Sources

All guides