The useful cost metric for an AI security agent is not cost per token. It is cost per defensible security outcome.
An agent can spend money through model calls, retrieval, reranking, tool APIs, workflow compute, traces, storage, retries, and human review. Optimizing only the model price hides the architecture.
Define the cost unit.
Choose a completed unit such as a triaged alert, evidence-backed investigation, or approved response step. Attribute all workflow cost to it:
total = model + retrieval + tools + compute + telemetry + storage + human review
Report median and tail cost. Averages hide loops and pathological cases.
Instrument the full path.
OpenTelemetry's generative AI conventions standardize model, token, duration, agent, and tool signals. Add workflow and business attributes:
- tenant and use case;
- case and step ID;
- model route;
- input and output tokens;
- retrieved documents;
- tool calls and retries;
- cache hits;
- total latency;
- completion and quality outcome.
Keep sensitive prompt content disabled or separately controlled.
Set budgets at three levels.
Use a per-step token and time budget, a per-workflow cost ceiling, and a tenant-level daily or monthly budget. When a limit is reached, degrade deliberately: use a smaller model, narrow retrieval, request human input, or stop. Do not silently return lower-quality certainty.
Optimize architecture before prompts.
High-value changes include:
- deterministic parsing before model reasoning;
- smaller models for classification and extraction;
- retrieval that narrows context early;
- cached stable enrichments;
- parallel independent tool calls;
- bounded retries with typed errors;
- stopping rules when evidence is sufficient;
- batching offline work;
- explicit prevention of recursive loops.
Connect cost to quality.
Plot quality, latency, and cost together by workflow slice. A cheaper model that increases analyst corrections may cost more overall. An expensive model may be worthwhile for rare critical cases and wasteful for routine enrichment.
The metric tree in Measuring whether an AI SOC is actually working provides the outcome side of this equation.
Make cost visible to product decisions.
Review cost by tenant, workflow, model, integration, and failure reason. Alert on step-count inflation, retry storms, context growth, provider-price changes, and unexpected high-cost routes.
Cost engineering is not a late optimization. A bounded agent is easier to operate, safer against resource exhaustion, and easier to price as a product.