Tetrate adds token broker to control AI agent spend

Tetrate has added a token-brokering feature to its Agent Router Enterprise product, now generally available to existing subscribers at no extra charge.

The update targets companies trying to control the cost of AI coding and other agent-based workloads, where one request can trigger many model calls before returning a result. It gives AI and platform teams a central way to set budgets, approved models, approved regions, and fallback rules across distributed inference deployments.

Agent Router Enterprise now links each inference request to policy checks before routing it to a model. Those checks cover spending, availability, and sovereignty rules, and the system is designed to switch requests automatically to an approved alternative when a budget limit is reached or a service issue arises.

The token broker sits between developers or software agents and the models they use. It evaluates each request against business, technical, and governance rules, including remaining budget, model approval status, and regional restrictions, then sends the request to a model that meets those conditions.

Cost pressure

The launch comes as companies face rising AI operating costs despite falling unit prices for inference. Tetrate pointed to industry data showing that almost all FinOps practitioners now manage AI spending, while total bills continue to climb because software agents can multiply the number of model calls behind a single user task.

That has made cost control a problem for engineering teams as well as finance departments. Companies often start with open-source AI proxies to manage access to models, but Tetrate argued those tools were built to sit in front of a small number of models rather than enforce a single budget and policy set across multiple regions and providers.

Tetrate said its approach is intended to remove manual review from the process. AI teams set policy through application programming interfaces, while developers continue to request what the company described as the nearest approved model through a software development kit, without adding their own budget controls into code.

If a budget threshold is crossed, the system triggers what Tetrate described as a circuit breaker and redirects traffic to another approved model instead of returning an error. The fallback can be a private model running inside a company's own environment, reducing cost and keeping workloads within required jurisdictions.

Governance layer

Tetrate framed sovereignty as one of several controls applied to each workload rather than a separate architecture. It said the same policy layer can steer requests according to cost, regional rules, and service health, allowing companies to mix frontier, private, and edge models across different providers and locations.

The feature is available through a command-line interface, a software development kit, and a set of application programming interfaces. Administrators can define policy once in a management plane and have it enforced on every inference request.

The product runs on Envoy AI Gateway, the open-source project that Tetrate co-created and helps maintain. According to Tetrate, Agent Router Enterprise is already used by thousands of developers across hundreds of environments and regions.

David Wang, Head of Product at Tetrate, said the main concern for engineering leaders is that AI agent costs can rise faster than internal controls can contain them.

"Agent token spend is the one line item the engineering organization can't easily cap for CFOs, even though the entire business is looking to them for answers," said David Wang, Head of Product at Tetrate.

He described the update as a way to apply a single budget policy across requests without disrupting developers' existing workflows.

"With this update, admins set a budget once, and it's enforced on every token, falling back to an approved model whenever spend or availability breaks, so cost never gets ahead of policy, agents can never incur unexpected runaway costs, and developers keep working the way they already do," Wang said.

Tetrate also cited customer use in support of the launch. Sunny Benefits said the system gives its platform team one place to manage spending across different models while allowing developers to keep using approved endpoints.

"At Sunny Benefits, managing inference cost across a range of models is something our platform team owns and answers for," said Sashi Desikan, CTO at Sunny Benefits.

"Agent Router Enterprise lets us set a budget once and have it enforced automatically, with fallback to an approved model when we need it. Our developers keep getting the nearest approved model without us reviewing every request," Desikan said.