Rate Limiting and Budget Controls for AI Agents

An agent without rate limits is an agent that can run up an unlimited bill, flood an API endpoint, or execute thousands of database queries in minutes. Rate limiting and budget controls are basic operational safety that every agent deployment needs.

Tool-Call Rate Limits

Set maximum call frequencies per tool, per agent, and per user:

rate_limits:
  - tool: "query_database"
    max_calls: 100
    window: "1h"
  - tool: "call_external_api"
    max_calls: 30
    window: "10m"
  - tool: "*"
    max_calls: 500
    window: "1h"
    per: "user"

When the limit is reached, subsequent calls are denied. The agent receives a denial with a reason ("rate limit exceeded") and must adapt.

Why this matters: A compromised agent executing an injection payload might try to call a data exfiltration tool thousands of times. Rate limits cap the damage even if the injection bypasses other controls.

Budget Caps

Rate limits control frequency. Budget caps control cost. Set maximum spending per session, per day, or per user:

• API call costs (tokens consumed, third-party API pricing)
• Compute time (for expensive operations)
• Data transfer volume (for operations that move data)

budgets:
  - scope: "session"
    max_cost: 5.00
    currency: "USD"
  - scope: "daily"
    max_cost: 50.00
    currency: "USD"
    per: "user"

When the budget is exhausted, the agent cannot make additional costly operations until the budget window resets.

Cumulative Action Limits

Some risks are about total volume, not rate. An agent that deletes one record per minute for 8 hours has deleted 480 records. The per-minute rate is low, but the cumulative impact is high.

Track cumulative actions per session and per day:

• Total records modified
• Total files created
• Total bytes transferred
• Total external API calls made

Set ceilings based on what is reasonable for the use case. A customer service agent should not modify more than 10 records per session.

Sliding Window vs Fixed Window

Fixed window: Reset the counter at fixed intervals (e.g., every hour on the hour). Simple but allows bursts at window boundaries (99 calls at 12:59, 100 calls at 1:00 = 199 calls in 2 minutes).

Sliding window: Track calls within a rolling time window. More accurate but slightly more complex. Prevents boundary bursting.

For agent safety, sliding windows are preferred. The burst behavior of fixed windows can allow short-term damage spikes.

What to Do When Limits Are Hit

Deny the action. The agent cannot proceed with the rate-limited tool. Return a clear error: "Rate limit exceeded for query_database. 100/100 calls used in the current hour."

Notify. If an agent is consistently hitting rate limits, something may be wrong: a bug, a compromise, or a misconfigured workflow. Alert the operations team.

Degrade gracefully. The agent can switch to alternative strategies. If the primary API is rate-limited, the agent might use cached data or ask the user to wait.

Do not queue. Do not accumulate denied requests for later execution. If the action was rate-limited now, it should not silently execute later. The user or agent should explicitly retry.

Rate limits and budgets are boring, unsexy controls. They also prevent some of the most expensive and damaging failure modes. Configure them before your first production deployment.