What Is the Unit of Value in AI Spend?

Cloud cost has always had a natural unit: a virtual machine running for an hour, a storage bucket holding a gigabyte, a data transfer event moving a measurable number of bytes from one region to another. Finance could read a bill, trace a line item back to a workload, and connect that workload to a team or a product. It was not always clean, but the cost object was recognizable.

AI workloads change the cost object in every way. When an application calls a large language model (LLM), the billing unit is the token, a fragment of text representing a word, part of a word, or a punctuation character. A single user request can consume hundreds or thousands of tokens depending on prompt length, context window size, and how much the model needs to reason before responding. Inference can cost anywhere from fractions of a cent to several dollars depending on the model tier and input complexity. That range does not exist in traditional compute billing, and organizations have not yet built the instrumentation designed for it.

This is where a familiar FinOps problem surfaces in a new form. Finance can see aggregate AI spend climbing. Engineering can see token counts in logs. Neither team can easily answer the question that matters most to the business: what did it cost to produce one unit of output?

The Measurement Gap

The challenge is that the relevant cost object does not map to any existing financial structure most teams have. Consider what happens when a user interacts with an AI-powered feature:

The posted per-call price reflects only a fraction of what a resolved AI task costs the business. Each of those surrounding costs is distributed across services, accounts, and billing categories built for different cost shapes, and none of them shows up on a single line item.

Why the Unit Matters Financially

There is a pricing dynamic that makes this harder to ignore over time. Per-token costs have fallen by more than 280 times for comparable model performance between 2022 and 2024, according to the Stanford HAI 2025 AI Index. Enterprise AI spending still more than tripled between 2024 and 2025, according to Menlo Ventures' 2025 State of Generative AI in the Enterprise report. The unit got cheaper while volume expanded faster, and teams reading falling per-token prices as a sign that AI costs are under control are working from an incomplete signal.

The gross margin implication is more direct than most finance teams currently appreciate. Traditional SaaS companies typically operate with gross margins well above 80 percent. AI-centric products compress that number considerably because inference cost functions as a direct input into cost of goods sold (COGS) in a way that traditional software compute never did. Engineers building AI features are making gross margin decisions whether finance knows it or not. As one FinOps practitioner put it in the State of FinOps 2026 report: "Is your AI providing value? No one can answer that question yet."

Where FinOps Becomes the Connective Layer

FinOps is positioned to close this gap because its function is to translate technical activity into financial context. The same practices that connect compute usage to cost centers can extend to AI workloads. The extension requires deliberate instrumentation decisions that most teams have not yet made, but the organizational pattern is familiar. Some teams are beginning to build toward three meaningful metrics:

Cost per AI-assisted transaction is where FinOps and product economics finally meet. When the cost is justifiable against the outcome it produces, the number becomes a planning input rather than a budget anomaly. When it cannot be explained in those terms, it will eventually surface in a finance conversation with no clean answer.

What often gets missed here is that the instrumentation problem is primarily organizational. Finance, engineering, and product teams are all measuring different things, and FinOps has not yet been consistently asked to bridge them in the context of AI workloads.

Getting to a reliable cost-per-output number requires shared definitions, consistent tagging of AI spend at the feature or product level, and a shared agreement to treat inference spend as a cost of goods rather than an open-ended experiment. That last shift carries more financial weight than it initially sounds.