Guy Kurlandski is the CEO & Co-founder of Tokto, a leading AI governance, finops and system of record on-prem solution for enterprises.
getty
Many leaders form their mental model of AI cost from a consumer subscription: a predictable $20 per month per user. But in my experience working in AI infrastructure and governance, that figure is one of the most misleading numbers in the AI conversation, and it is why I believe many community banks are about to be surprised.
Consumer chat plans are typically loss-leaders with hard usage caps. The areas that many bank finance department heads want tools for—loan review, document analysis, credit memo drafting, customer service—don’t run on that kind of plan. They run on commercial API access, priced by the token and uncapped by design. At commercial rates, a single power user doesn’t cost $20 a month. They can cost several hundred.
Where The Money Actually Goes
Frontier models are generally priced per million tokens, with output—and the hidden “reasoning” tokens that premium models generate before answering—billed at a substantial multiple of input. A few dollars per million tokens sounds trivial. The trap is volume, and volume in real banking workflows tends to be enormous.
Consider an analyst running a 200-page loan file through an AI workflow. That isn’t one call. An agentic process retrieves documents, re-reads context on every turn, cross-checks covenants, reconciles financials and iterates often dozens of model calls per task, each resending accumulated context. One document-heavy task can consume millions of tokens and cost anywhere from a few dollars to well over 50, depending on model choice and how tightly the workflow is engineered.
Scale it: 20 tasks a day, 20 days a month, is 400 tasks. If the tasks are priced at $10 each, that’s $4,000 a month for one analyst. Extend the capability to a lending team of eight, and its a six-figure annual line item that didn’t appear in any capital plan, driven entirely by how enthusiastically your people use the AI.
The pressure is internal, and it is legitimate.
Community bank chief financial officers (CFOs) aren’t managing reckless colleagues. Your chief operating officer (COO) asking for AI is likely just trying to hold headcount flat against rising volume. Your chief lending officer (CLO) may be watching a competitor close faster. They’re right that the capability can deliver. What they may not be able to see is that it’s metered, with no natural ceiling. In many cases, this pushes adoption onto expense reports, where it’s both more costly per unit and entirely ungoverned. That is why I believe the CFO’s job isn’t to slow AI down—it’s to make it countable.
Before you begin, answer these questions.
In order to properly track and plan for hidden AI costs, there are a number of questions I recommend CFOs and banking leadership teams consider this quarter:
• What are you paying per unit of work? (Not per seat, per credit memo, per alert reviewed or per document processed.)
• Do you know total AI spend across licenses, embedded vendor features, cloud AI services and expensed departmental tools?
• Can you attribute consumption to a department, a use case, an individual? If AI is just a line inside the IT budget, it is unmanaged.
• What happens if consumption triples? Do you learn about it from a dashboard, from an alert or from an invoice 90 days later?
• Who can stop it? Is there a hard ceiling, or does the meter simply run?
• Who chooses your model? If an AI vendor upgrades your embedded feature to a costlier model, does your spend move without your approval?
• What do you show an examiner asking how AI in a credit or compliance workflow is governed, monitored and documented?
Choose your monitoring systems.
When it comes to choosing software for tracking AI token spend, there are three core elements to consider:
1. Governance: Governance is about ensuring that all policy, model inventory, approvals, acceptable use, output monitoring and audit evidence answer the examiner. Governance platforms are built for risk and compliance; however, they generally don’t answer the CFO’s budget and expense questions at all.
2. FinOps: FinOps focuses on applying cloud-cost rigor to token consumption, including real-time visibility, budget guardrails and active cost “reduction.” This is typically where the leverage sits. Prompt- and context-caching, smart routing to cheaper models while reserving premium for tasks that need it, eliminating redundant calls, and trimming prompts routinely can significantly cut spend without degradation.
3. Systems Of Record: This is often the scarcest category. A true system of record should capture and retain every interaction: what was asked, what was answered, which model, which user, under which policy and at what cost. It should be able to convert AI from un-auditable activity into a governed business process.
Governance alone gives you policy but not economics, FinOps alone gives you economics but not defensibility, and a system of record alone gives you a well-documented archive of spending you never controlled. That’s why I recommend utilizing all three to ensure the most accurate management of your AI token spend.
If you don’t have the internal systems or team needed to set up these three capabilities within your own company, there are numerous options available from third-party providers. Many services offer these platforms individually, although contracting them separately means reconciling three contracts and three data models. Some vendors offer all in one platform, where the policy authorizing a use case, the budget constraining it and the record proving it draw on the same data. (Full Disclosure: My company offers these services, as do others.) Before deciding on a partner, ask each prospective vendor which of the three their platform owns and which ones they merely integrate with.
Start with the number.
You don’t need another AI strategy this month. You need a number, and a unit cost beneath it.
Find your total AI spend. Attribute it. Set a ceiling with an owner and an alert. Require every new use case to arrive with an expected return and an expected cost per transaction. Then evaluate platforms that allow you to govern, optimize and document it.
AI may be the most consequential efficiency lever community banks have had in a generation. Whether it also becomes the least controlled expense on your income statement depends in part on the decisions you make today.
Forbes Business Council is the foremost growth and networking organization for business owners and leaders. Do I qualify?
