The question is not whether AI can
The conversation we have most often starts with a company that has seen something impressive and would like the same thing. And usually it is possible. Language models have become good enough that «can this be done?» is almost always answered yes.
That is simply not the question that decides anything. The right question is whether AI is the cheapest, most predictable and most maintainable way to solve this particular job. Often it is not. A set of rules that always gives the same answer beats a model that almost always does — and «almost» is more expensive to live with than people expect.
Is there one right answer?
If the job has a correct answer, AI is the wrong tool. A price, a staff rota, a VAT calculation, a discount structure: all of those have a right answer that can be written down as rules. A calculator gives the same result every time, can be audited by a bookkeeper, and can be explained to a customer who asks why.
The strength of language models is ambiguity — reading an email that was not written to a template and working out what is actually being asked. Their weakness is arithmetic and fixed rules. Use them to calculate and you pay for the expensive tool and get the unreliable result.
What does a mistake cost?
A language model is a probabilistic system. It gets things wrong sometimes, and you cannot test your way out of that entirely. So what matters is not the error rate but the price of a single error.
If a mistake costs an apology, fine. If it costs a wrong invoice, a missed deadline or a commitment the company is bound by, then either a human has to sign off before anything is sent, or that part of the job has to be solved with rules instead. That is why our email workflows draft rather than send: the mistake becomes two seconds of reading instead of a complaint.
Do you need to explain the answer afterwards?
Some industries have to document why a system reached a decision. A set of rules can be audited line by line: here was the condition, here it was met.
A language model can explain itself, but the explanation is not necessarily the reason — it is a plausible-sounding rationalisation made after the fact. If the decision has to survive an audit or a complaint, that is not good enough.
Does the data even exist?
«Ask our own manuals» is a good idea if the manuals exist. We have watched the job fall apart because the knowledge the system was meant to search lived in one employee's head and in a folder full of files called final_v3_FIXED.docx.
AI cannot invent knowledge the company never wrote down. If the documentation is not there, the first job is getting it out of people's heads — and that is an organisational project, not an IT one. We say so out loud, because otherwise it gets discovered after the meter has started running.
What we do instead
The answer is rarely either-or. What we most often land on is rules at the core and AI at the edge: the calculator calculates, and the language model writes the covering note. The rota is built by an algorithm that respects the collective agreement, and the AI phrases the message to whoever drew an awkward shift.
That gives you predictability where it matters and language where writing it yourself is tedious. And you only pay for the model on the part that actually needs it.
In short
- If the job has one right answer, solve it with rules — not with a model.
- The error rate matters less than the price of a single error. If that price is high, a human signs off before anything is sent.
- If the decision has to be auditable, a model cannot carry it alone.
- AI cannot search knowledge that was never written down.
- The most durable design is usually rules at the core and AI at the edge.