AI NativeMid-market Operations Lead

AI Run Ledger for Auditing and Billing | Botmanor Use Cases

Tokens, cost, latency, tool calls and retrieved chunks — per run, per agent, per app

AI Run Ledger for Auditing and Billing | Botmanor Use Cases

AI spend goes wrong quietly. A prompt gets longer, a model gets swapped for a better one, a channel integration starts retrying, and nobody notices until the invoice. The only durable fix is that every inference has a record, and Botmanor writes one for every agent run — web chat, channel message, delegated task or workflow-triggered run alike, whether it succeeded, failed, timed out or was cancelled.

Each record carries what an operator actually needs: the agent and app it belongs to, the input and output, the status, the duration in milliseconds, the tokens consumed, the resolved cost, the model that served it, and structured error detail when something broke. Alongside it sits a metrics row that splits input and output tokens and counts the two things that distinguish an agentic run from a completion call — how many tool calls it made and how many retrieved chunks it was grounded on. Aggregations roll the same data up per agent and per app, and a subscription streams execution updates as they happen.

Because the ledger is written by the runtime rather than reconstructed from logs, it answers the questions that follow a bad week. Which agent's cost per run tripled, and after which published version? Which runs answered with zero retrieved chunks, meaning retrieval was silently missing? Which bot is carrying the volume, and which has not run at all this month and should be retired? For an agency billing several client workspaces, the same records are per-client economics rather than an estimate.

Metering sits on top of the same substrate. The units a plan actually prices carry usage directives enforced at the gateway: messages processed, bots created, runs triggered, knowledge bases provisioned. The boundaries are drawn deliberately — internal agents are not metered separately from the bot app that fans out to them, precisely so one logical bot is not double-charged for its own orchestration, and that reasoning is written down next to the directive rather than left for someone to rediscover.

The honest gaps. Customer agent execution runs on the workspace's own provider keys, so what the ledger records is your own inference bill, reported for visibility rather than charged as platform credit — and Botmanor keeps those two things separate on purpose. A dedicated AI-cost view that slices spend by app, agent and model, anomaly alerts when an agent's spend deviates from its own baseline, and the platform credit ceiling for Botmanor-initiated inference are all designed and ticketed, not yet shipped. Today the ledger is complete; the dashboards on top of it are still arriving.

Do it yourself

Run one real agent turn and follow the record it leaves: the execution row with its status, duration, tokens, cost and model, filtered and opened down to the exact input and output, with the metered units counting against your plan.

0 / 6
  1. From the Botmanor sidebar open Agents and pick one that already has a model binding.

    You should see: The agent list loads from the live API; each agent shows its type, status and model binding.

    Open in app

Ready to make this your story?

We use cookies for essential site functions and, with your consent, for analytics to improve Botmanor. We don't use advertising or cross-site tracking cookies. See our Cookie Policy.

Preferences