Retry a timed-out agent tool call without refunding twice
Your agent asked the refund tool to refund order 1042. The call timed out, so the harness tried again. Did the customer get one refund or two? By the end of this guide you can:
- See why a lost reply does not show that a tool call failed.
- Add an idempotency key to a tool call, so a retry does not repeat the change.
- Check the result by reading what the shop recorded, not what the agent said.
You need Python 3.12 or newer. No API key, no network and no packages to install.
Before you start
This is episode 4. You should know Python functions, classes and exceptions. If not, open these first. Each link opens a breakdown for that topic with a revision guide and side chats. Sign in to open it.
- Debug an LLM agent: find the first bad step in its trace. Episode 1. We reuse its refund agent and order 1042.
- Turn a failed agent run into an eval that blocks the release. Episode 2. We grade a run on what the shop recorded.
- How an LLM tool-calling loop works. The harness is the code that runs your tools for the model.
- HTTP methods and idempotency. Step 3 relies on a call that can be repeated safely.
The question
In episodes 1 to 3 the refund agent looked up order 1042 and refunded its total, 52.50. Now the refund call itself times out. The shop did the refund, but its reply was lost on the way back. The harness sees only a timeout, so it can give up, try again, or try again with a key. Each choice leaves a different record in the shop.
The shop below is a fake order system, and the agent is plain Python that stands in for a model, so every run gives the same result. It is not a live model, and there is no customer data. We set the lost replies.
A timeout does not tell you what happened. The AWS Builders' Library says a timeout or failure does not necessarily mean the side effects did not happen. So the question is what the shop has recorded, and how to retry without adding to it.
Four words we use
- Lost reply
- The tool may or may not have done its work, and the answer never reached the caller. A timeout is one case. The caller cannot tell which happened.
- Retry
- Calling the same tool again with the same inputs because the first call gave no answer.
- Idempotent
- Doing it twice has the same effect as doing it once. RFC 9110 defines an idempotent method this way: the intended effect of several identical requests is the same as the effect of one.
- Idempotency key
- A unique string the caller makes for one action and sends with every try of that action. The service uses it to see that the tries are one action, so it does not do the work twice. Our shop saves the reply under the key and sends it back on a repeat. How a real service does this depends on the service.
Worked example
Retry a refund after a lost reply, and check what the shop recorded each time.
Lose the reply, do not retry
The shop issues the refund, then loses the reply. The harness makes one try, so it gives up, and the agent says it could not confirm the refund. That answer is honest. The shop recorded [52.5], so the refund did happen. A lost reply does not show that the call failed, and an agent that said the refund failed would be wrong.
Retry without a key
The harness tries again. The first try already refunded, so the second try refunds again. The second reply arrives and the agent says Done, refunded 52.50 for order 1042. Nothing in the log looks wrong. The shop recorded [52.5, 52.5], so the customer was refunded twice. This is why episode 2 grades the record and not the agent's words.
Retry with one key
Make one key for the action before the first try. Here the key is key-1. Send it with every try. The first try refunds, and the shop saves its reply under the key. The retry has the same key, so the shop sends back the saved reply and does nothing else. The agent's log is the same as in step 2, but the shop recorded [52.5].
Make the key inside the retry
A common mistake is to build the key inside the call that gets retried. Every try then has a new key (key-2, then key-3 here), so the shop sees two different actions. The agent's log still looks the same as in step 3, but the shop recorded [52.5, 52.5]. Make the key once, outside the retry loop, and reuse it.
Give up safely, then retry later
Now every reply is lost. The harness stops at its limit of three tries, and the agent says it could not confirm the refund. The shop recorded [52.5]. The first try worked, and the other two were answered from the saved reply. Do not tell the customer the refund failed. Later, a retry with the same key gets the reply, and the record is still [52.5].
Write the retry rule
Retry a read-only call when the failure may be temporary. Retry a call that changes something only when the service supports idempotency, and send the same key, made once for that action, with every try. Set a limit on tries. If a change still has no answer at the limit, report that the outcome is unknown, then check the shop's record or ask a person. Turn these cases into tests like episode 2's, and make each one check the shop's record.
Run it yourself
Put these three files in one folder. Every command here was run on 11 November 2026 with Python 3.12.16 and 3.15.2. The output was the same on both. The code uses only the standard library, so there is nothing to install.
The commands use uv. Without uv, run python compare.py in Python 3.12 or newer.
shop.py is the fake order system. It records refunds, saves the reply for each key it has seen, and loses replies when we ask it to.
class ReplyLost(Exception):
"""The shop may have done the work, but the reply never reached the caller (a timeout)."""
class Shop:
"""A fake order system. It records every refund, so a run can be graded on what really changed.
The first lose_replies calls do their work and then lose the reply."""
def __init__(self, lose_replies=0):
self.refunds, self.saved, self.lose_replies = [], {}, lose_replies
def issue_refund(self, order_id, amount, key=None):
if key in self.saved: # a retry of an action the shop already did: answer again, do nothing
reply = self.saved[key]
else:
self.refunds.append(amount)
reply = f"refunded {amount:.2f} for order {order_id}"
if key is not None:
self.saved[key] = reply
if self.lose_replies:
self.lose_replies -= 1
raise ReplyLost("timeout")
return replyharness.py holds the retry loop. It retries only when the reply was lost, and it stops at a limit.
from shop import ReplyLost
class GaveUp(Exception):
"""Every try ended with a lost reply, so the outcome is unknown."""
def call_tool(call, log, tries=3):
"""Run call(). Retry only when the reply was lost, and stop after tries."""
for attempt in range(1, tries + 1):
try:
reply = call()
except ReplyLost:
log.append(f"attempt {attempt}: reply lost")
continue
log.append(f"attempt {attempt}: {reply}")
return reply
raise GaveUp(f"no reply after {tries} tries")compare.py runs the five steps. Each printed line starts with its step number. PASS means the shop recorded exactly one refund of 52.5.
from itertools import count
from harness import GaveUp, call_tool
from shop import Shop
ids = count(1)
def new_key():
"""A real harness would use uuid.uuid4().hex. A counter keeps the output the same on every run."""
return f"key-{next(ids)}"
def run(step, title, shop, call, tries=3):
log = []
try:
answer = "Done, " + call_tool(call, log, tries) + "."
except GaveUp:
answer = "I could not confirm the refund."
print(f"Step {step} {title}")
for line in log:
print(" " + line)
print(" agent says:", answer)
print(" shop recorded:", shop.refunds, "PASS" if shop.refunds == [52.5] else "FAIL")
shop = Shop(lose_replies=1)
run(1, "the reply is lost and the harness does not retry", shop, lambda: shop.issue_refund(1042, 52.5), tries=1)
shop = Shop(lose_replies=1)
run(2, "retry without a key", shop, lambda: shop.issue_refund(1042, 52.5))
shop, key = Shop(lose_replies=1), new_key()
run(3, "retry with one key made before the first try", shop, lambda: shop.issue_refund(1042, 52.5, key=key))
shop = Shop(lose_replies=1)
run(4, "retry with a new key made on every try", shop, lambda: shop.issue_refund(1042, 52.5, key=new_key()))
shop, key = Shop(lose_replies=3), new_key()
run(5, "every reply is lost, then a later retry", shop, lambda: shop.issue_refund(1042, 52.5, key=key))
print(" later, same key:", shop.issue_refund(1042, 52.5, key=key))
print(" shop recorded:", shop.refunds)uv run --no-project --python 3.12 python compare.pyStep 1 the reply is lost and the harness does not retry
attempt 1: reply lost
agent says: I could not confirm the refund.
shop recorded: [52.5] PASS
Step 2 retry without a key
attempt 1: reply lost
attempt 2: refunded 52.50 for order 1042
agent says: Done, refunded 52.50 for order 1042.
shop recorded: [52.5, 52.5] FAIL
Step 3 retry with one key made before the first try
attempt 1: reply lost
attempt 2: refunded 52.50 for order 1042
agent says: Done, refunded 52.50 for order 1042.
shop recorded: [52.5] PASS
Step 4 retry with a new key made on every try
attempt 1: reply lost
attempt 2: refunded 52.50 for order 1042
agent says: Done, refunded 52.50 for order 1042.
shop recorded: [52.5, 52.5] FAIL
Step 5 every reply is lost, then a later retry
attempt 1: reply lost
attempt 2: reply lost
attempt 3: reply lost
agent says: I could not confirm the refund.
shop recorded: [52.5] PASS
later, same key: refunded 52.50 for order 1042
shop recorded: [52.5]Why a key fixes it
The caller cannot tell a lost request from a lost reply, so retrying a change is a guess. RFC 9110 says a client should not automatically retry a request with a non-idempotent method unless it knows the request is idempotent or can detect that the original was never applied. The RFC is about HTTP requests, and a refund tool that calls an HTTP API is one case of it. The AWS Builders' Library says APIs with side effects are not safe to retry unless they provide idempotency. A key gives the service a way to see that two requests are one action.
Stripe's API works this way. The client makes the key. For API v1, once the endpoint starts running, Stripe saves the status code and body of the first request made with a key, and a retry with the same key returns the saved response. An IETF draft, The Idempotency-Key HTTP Header Field, proposes a standard header for the key. It is a work in progress, not a standard.
What this does not do
- The shop is a fake order system, and the agent is plain Python that stands in for a model. We set the lost replies. This is not a live model, a real payment service or customer data.
- A real service does more with keys. Stripe's API compares the parameters of a retry with the first request and returns an error if they differ. Its docs also say API v1 keys can be removed automatically after at least 24 hours, and a key reused after that starts a new request. Our shop does neither. Check how long your own service keeps keys.
- Our shop handles one call at a time. A real service must also handle two calls with the same key that arrive together.
- The harness makes the key once per action. If the model starts a second refund call on its own, the harness will usually make a new key for it, and the shop treats it as a new action, even if the model meant it as a repeat. A key does not stop that. Guard it with a check of the shop's record, for example that no refund exists for the order. It is not shown here.
- The harness retries only a lost reply. Other errors need other handling. In episode 1 the order lookup was blocked with a 403, and trying again with no change would not fix that.
- There is no wait between tries. A real harness should wait longer after each failure and add some randomness, which AWS calls backoff with jitter.
Explore this topic in the workspace
Topic Breakdown gives you a structured revision guide for this topic and lets you highlight any phrase to open side-by-side chats with worked examples. Sign in to open it.
Open breakdownOr type Retries and idempotency for LLM agent tools into the topic box on the home page. To see a lesson first, open the sample breakdown on Machine Learning. Missed the earlier episodes? Read how to find the first bad step in a trace, how to turn a failed run into a release gate and how to tell a real fix from luck. Running these calls from a queue? See why one slow LLM call stalls your queue. Want to keep raw API keys and destructive routes such as DELETE away from the agent too? Build a local ticket broker in Python.
Sources
All sources were checked on 11 November 2026. Read each page for the full context.
- RFC 9110, section 9.2.2: Idempotent Methods
A method is idempotent if the intended effect of several identical requests is the same as the effect of one. PUT, DELETE and the safe methods are idempotent. A client should not automatically retry a request with a non-idempotent method unless it knows the request is idempotent or can detect that the original was never applied.
- AWS Builders' Library: Timeouts, retries, and backoff with jitter
Published 5 December 2019. A timeout or failure does not necessarily mean the side effects did not happen. APIs with side effects are not safe to retry unless they provide idempotency, and read-only APIs are typically idempotent. Jitter adds randomness to backoff to spread retries over time.
- Stripe docs: Idempotent requests
The client makes the key, and Stripe suggests a random string such as a V4 UUID. For API v1, once the endpoint starts running, Stripe saves the status code and body of the first request made with a key, and a retry with the same key returns the saved response. A request that fails validation, or conflicts with one still running, is not saved and can be retried. Stripe compares the parameters of a retry with the first request and errors if they differ. API v1 keys can be removed after at least 24 hours.
- IETF draft: The Idempotency-Key HTTP Header Field (draft 08)
Dated 22 October 2026. A work in progress, not a standard. Its abstract says the header can make non-idempotent methods such as POST or PATCH fault-tolerant.
- Python docs: uuid
uuid4() creates a random UUID. It is what new_key in compare.py would call in a real harness.