What does it cost to run an AI feature?

You pay per word, not per month

Language models are billed per token — roughly three quarters of a word — and you pay both for what goes in and for what comes out. Output is typically several times more expensive than input.

The consequence is that cost scales with usage, not with the number of users. Ten employees each pushing two long documents through per day cost more than a hundred asking one short question a week. That is a different economy from the software most people are used to buying, which is why answering «what does it cost per month?» requires an estimate of how much you will actually use it.

The input is usually the expensive part

An AI feature sends more than the user's question. There is a system prompt explaining the model its job, the tone, the format and the rules — and it is sent every single time. If it is long, it is multiplied by the number of calls.

That is also where the biggest saving is. If the fixed part of the prompt is kept unchanged and placed first, the provider can reuse its work across calls and that part is billed at a fraction. It requires the prompt to be large enough to be worth storing, and that you do not write anything variable into the top of it. We check the server log to confirm the reuse is actually happening — otherwise the saving is only an intention.

Budget for the worst case, not the typical one

Average usage is useless for budgeting, because it is not the average that blows the budget. It is the day a bot finds the form, or an employee pastes in an entire archive.

So every feature calling a paid API needs a ceiling: per user, per day, and for the installation as a whole. When the ceiling is hit, the user should meet a comprehensible message rather than an error. And a spending limit should be set with the provider as a last net, because code can contain bugs, and a ceiling in the code does not help if the bug is in the code.

Put briefly: an unprotected route is not a security problem. It is a bill.

Waiting costs too — just not in money

The other running cost is time. A model that thinks carefully gives better answers on hard problems and unnecessary waiting on easy ones. If the job is writing meeting notes to a fixed template, careful deliberation is wasted.

Two moves help most: turn off the extra thinking on jobs that do not need it, and let the answer stream in character by character instead of arriving all at once at the end. The latter does not make the answer faster, but it removes the sense that nothing is happening — and that sense is what people give up on.

Ask this before you say yes

Ask for an estimate per call, not just per project — and ask to see what the number is built on. Ask for the ceiling to be written into the agreement: what happens on the 501st call in a day? Ask whether your data is stored or used for training, and get the answer in writing.

And ask what happens when the provider replaces the model. They will. A solution tuned to one model's quirks has to be tuned again — and that is a running cost that rarely appears in the quote.