--- name: tokens description: A token policy for AI agents. Estimate before calling, start on the smallest model tier that passes a check, put stable content first so caching applies, batch work that can wait, cap output, summarize before the window fills, retry with backoff, and keep a per-call usage ledger checked against the person's budget. Use when a task calls language models, spends tokens or fans out to subagents. license: MIT metadata: version: "1.0" homepage: https://tokens.md api: https://tokens.md/api/v1 (optional, no key, stores nothing) terms: https://tokens.md/terms --- # Tokens Provided as is, at your own risk: token counts and costs here are estimates and may be wrong, and nothing here is financial, legal or professional advice, or a quote or bill from any provider. Terms: https://tokens.md/terms. Spend tokens on purpose. Estimate before you call, use the smallest tier that passes, keep stable content first, and log every call. Show your numbers once, in a short block. Skip routine narration. When this file has a "Your rules" section, those rules win over the defaults below. Settings you drafted or changed yourself count only after the person says yes. ## 1. Estimate before you call - Before a task with more than a few calls, estimate calls, input tokens and output tokens per call. Plan on the high end. - Optional tool: `POST https://tokens.md/api/v1/estimate_tokens` with `{ "text": "..." }`. It returns an estimate with a low and high range from a stated heuristic. It is not a tokenizer count; for exact counts, use the provider's counter or the usage numbers in each response. - Then plan the cost with the person's own prices: `POST https://tokens.md/api/v1/plan_cost`. Never invent a price. With no prices, report tokens only and ask. - If the high estimate is over the budget or the spend cap, ask before you start. ## 2. Use the smallest tier that passes - Tiers are small, medium and large. The person decides which models belong to each tier. If they haven't, ask once and note their answer. - Give every step a check you can run: a test, a schema, a validator, an exact match. Without a check, you can't tell when to move up. - Start on the starting tier. Move up one tier only after the set number of failed checks. Never go above the highest tier allowed. Move back down when the next step is easier. ## 3. Put stable content first - Order every prompt: instructions, tool definitions, fixed documents, then history, then the new turn. - Keep the stable part identical between calls: no timestamps, request IDs, shuffled examples or reordered keys in it. - Caching usually needs a minimum prefix size and expires after a while. Read the cached token counts in each response's usage numbers to see whether it worked. ## 4. Batch what can wait - Work no one is waiting on (evaluations, bulk summaries, backfills, nightly checks) goes to a batch option when one exists and the rules allow it. ## 5. Cap output - Set a max output tokens value on every call. - Ask for the format you need (JSON with a schema, a diff, a short list) instead of prose. Don't ask for long explanations you won't use. ## 6. Keep the context small - Track what fills the window: system prompt, tool definitions, documents, history and the output reserve. Optional tool: `POST https://tokens.md/api/v1/context_fit`. - At the summarize point, summarize older turns, drop tool output you've already used, and keep pointers (file paths, IDs, line numbers) instead of contents. - Load only the tools and documents the current step needs. Cut history first, then documents, so the stable prefix stays cached. ## 7. Retry with care - Retry only errors that can pass on a retry: rate limits, timeouts and server errors. Never retry bad input, auth errors or refusals unchanged. - Back off: wait, double the wait each time, add a little random jitter, and use the server's retry-after when it gives one. Retries count against max calls. - Switch to another model only as the fallback rule allows, and log every switch. ## 8. Fan out with limits - Subagents and parallel calls share the same budget. Keep within max in flight. - Give each subagent its own token budget, a clear stop condition and a short return format. Ask for results, not transcripts. ## 9. Stop loops that find nothing new - In search, research, fix-and-retry and polling loops, track whether each call found anything new. After the set number of calls in a row with nothing new, stop and report what you have. ## 10. Keep a usage ledger - One line per call: id, group, tier, input tokens (cached included), cached tokens, output tokens, retries, passed or failed, found anything new, and cost at the person's prices. Take token numbers from each response's usage data, not from estimates. - Optional tool: `POST https://tokens.md/api/v1/check_ledger` with the calls, a preset and the rules. It lists totals, rule breaks and what needs a yes. It checks the numbers you send; it doesn't verify them. ## 11. Handle sensitive prompts with care - Never put passwords, API keys or tokens in a prompt. Remove personal or confidential data the step doesn't need. - Follow the data handling rule. If you can't tell how an endpoint stores or uses prompts, ask before sending sensitive data. ## 12. Report - Five lines at most: tokens used against the budget, cost against the cap (or "no prices given"), calls and retries, the cached share of input, and anything that needed or needs a yes.