An agent retry budget limits repeated work across the whole task, not just one library call. Define how many attempts are allowed, which failures are retryable, the maximum elapsed time, and what the agent should tell the user when recovery stops.
Without a shared budget, an SDK, tool wrapper, and reasoning loop can each retry independently. The resulting request volume may be much larger than the agent code suggests.
Count all retry layers
Suppose a model loop allows three tool attempts and the tool’s HTTP client allows three network attempts for each invocation. One logical operation can produce up to nine downstream requests under those assumptions. This is illustrative arithmetic, not a default behavior of any named framework.
Inspect the model provider client, tool SDK, gateway, queue worker, and agent loop. Decide which layer owns recovery for each class of failure. Preserve a task ID and attempt IDs so telemetry can distinguish logical actions from physical requests.
The AWS Builders’ Library discussion of retries explains why retries can amplify load, why timeouts leave side effects ambiguous, and why backoff and jitter matter. An agent loop inherits those distributed-systems concerns even if the retry decision is written in natural language.
Match the policy to the failure
| Failure | Typical response to consider | What would make retry wrong? |
|---|---|---|
| Temporary service unavailability | Bounded retry with delay | Exhausted deadline or unsafe side effect |
| Rate limit | Respect provider guidance within the task budget | Immediate repeated attempts |
| Invalid arguments | Repair the input if the cause is known | Sending identical invalid arguments again |
| Permission denied | Stop and explain or request the proper authorization | Treating authorization as a transient outage |
| Business rejection | Follow the domain result | Retrying to bypass policy |
| Ambiguous mutation timeout | Reconcile the original action | Creating a new action blindly |
These are design choices to validate against each provider’s documented behavior. Error codes alone do not always reveal whether a partial action happened.
Define a task-level contract
A proposed support-agent policy might allow two recovery attempts for a transient read failure, keep total tool time within a fixed deadline, and stop immediately on authorization errors. The exact numbers should come from user expectations and service behavior, not a generic blog recommendation.
Store the remaining budget outside the model’s conversational memory. Deduct actual attempts at the execution boundary. A prompt that says “do not retry too much” can guide behavior but cannot reliably enforce a hard limit.
Distinguish attempts from corrections. If the user supplies a new address, another lookup may be new work. If the agent changes a JSON key spelling to repair a validation error, record that as a repair attempt with its own reason.
Make the stop behavior part of the design
When a budget expires, return an explicit state and a useful next step. “I could not confirm the order status; please try again later” may be appropriate for a read. For a payment, the message should acknowledge that the outcome is being checked rather than claiming failure or encouraging a duplicate payment.
Include the reason for stopping in operational metadata. Otherwise a lower retry count can look like an improvement even when the agent now abandons recoverable tasks.
Test recovery and restraint together
Build cases for a first-attempt failure followed by success, a permanent rejection, repeated rate limiting, and an ambiguous action timeout. Add a healthy run with multiple distinct tool calls so the budget is not simply a cap on useful work.
Measure recovered tasks, terminal failures, unknown outcomes, attempts per task, and end-to-end latency. Report success for the task, not the proportion of individual attempts that returned 200.
In Agnost, compare tool-call patterns with the final user-facing result. Use idempotency for external actions, and duplicate-read detection when the agent repeats already successful lookups. A good retry policy saves recoverable tasks without turning an outage into a loop.