All guides

Retry a timed-out agent tool call without refunding twice

Your agent asked the refund tool to refund order 1042. The call timed out, so the harness tried again. Did the customer get one refund or two? By the end of this guide you can:

  • See why a lost reply does not show that a tool call failed.
  • Add an idempotency key to a tool call, so a retry does not repeat the change.
  • Check the result by reading what the shop recorded, not what the agent said.

You need Python 3.12 or newer. No API key, no network and no packages to install.

Before you start

This is episode 4. You should know Python functions, classes and exceptions. If not, open these first. Each link opens a breakdown for that topic with a revision guide and side chats. Sign in to open it.

  1. Debug an LLM agent: find the first bad step in its trace. Episode 1. We reuse its refund agent and order 1042.
  2. Turn a failed agent run into an eval that blocks the release. Episode 2. We grade a run on what the shop recorded.
  3. How an LLM tool-calling loop works. The harness is the code that runs your tools for the model.
  4. HTTP methods and idempotency. Step 3 relies on a call that can be repeated safely.

The question

In episodes 1 to 3 the refund agent looked up order 1042 and refunded its total, 52.50. Now the refund call itself times out. The shop did the refund, but its reply was lost on the way back. The harness sees only a timeout, so it can give up, try again, or try again with a key. Each choice leaves a different record in the shop.

The shop below is a fake order system, and the agent is plain Python that stands in for a model, so every run gives the same result. It is not a live model, and there is no customer data. We set the lost replies.

A timeout does not tell you what happened. The AWS Builders' Library says a timeout or failure does not necessarily mean the side effects did not happen. So the question is what the shop has recorded, and how to retry without adding to it.

Four words we use

Lost reply
The tool may or may not have done its work, and the answer never reached the caller. A timeout is one case. The caller cannot tell which happened.
Retry
Calling the same tool again with the same inputs because the first call gave no answer.
Idempotent
Doing it twice has the same effect as doing it once. RFC 9110 defines an idempotent method this way: the intended effect of several identical requests is the same as the effect of one.
Idempotency key
A unique string the caller makes for one action and sends with every try of that action. The service uses it to see that the tries are one action, so it does not do the work twice. Our shop saves the reply under the key and sends it back on a repeat. How a real service does this depends on the service.

Worked example

Retry a refund after a lost reply, and check what the shop recorded each time.

  1. Lose the reply, do not retry

    The shop issues the refund, then loses the reply. The harness makes one try, so it gives up, and the agent says it could not confirm the refund. That answer is honest. The shop recorded [52.5], so the refund did happen. A lost reply does not show that the call failed, and an agent that said the refund failed would be wrong.

  2. Retry without a key

    The harness tries again. The first try already refunded, so the second try refunds again. The second reply arrives and the agent says Done, refunded 52.50 for order 1042. Nothing in the log looks wrong. The shop recorded [52.5, 52.5], so the customer was refunded twice. This is why episode 2 grades the record and not the agent's words.

  3. Retry with one key

    Make one key for the action before the first try. Here the key is key-1. Send it with every try. The first try refunds, and the shop saves its reply under the key. The retry has the same key, so the shop sends back the saved reply and does nothing else. The agent's log is the same as in step 2, but the shop recorded [52.5].

  4. Make the key inside the retry

    A common mistake is to build the key inside the call that gets retried. Every try then has a new key (key-2, then key-3 here), so the shop sees two different actions. The agent's log still looks the same as in step 3, but the shop recorded [52.5, 52.5]. Make the key once, outside the retry loop, and reuse it.

  5. Give up safely, then retry later

    Now every reply is lost. The harness stops at its limit of three tries, and the agent says it could not confirm the refund. The shop recorded [52.5]. The first try worked, and the other two were answered from the saved reply. Do not tell the customer the refund failed. Later, a retry with the same key gets the reply, and the record is still [52.5].

  6. Write the retry rule

    Retry a read-only call when the failure may be temporary. Retry a call that changes something only when the service supports idempotency, and send the same key, made once for that action, with every try. Set a limit on tries. If a change still has no answer at the limit, report that the outcome is unknown, then check the shop's record or ask a person. Turn these cases into tests like episode 2's, and make each one check the shop's record.

Run it yourself

Put these three files in one folder. Every command here was run on 11 November 2026 with Python 3.12.16 and 3.15.2. The output was the same on both. The code uses only the standard library, so there is nothing to install.

The commands use uv. Without uv, run python compare.py in Python 3.12 or newer.

shop.py is the fake order system. It records refunds, saves the reply for each key it has seen, and loses replies when we ask it to.

shop.py
class ReplyLost(Exception):
    """The shop may have done the work, but the reply never reached the caller (a timeout)."""


class Shop:
    """A fake order system. It records every refund, so a run can be graded on what really changed.
    The first lose_replies calls do their work and then lose the reply."""

    def __init__(self, lose_replies=0):
        self.refunds, self.saved, self.lose_replies = [], {}, lose_replies

    def issue_refund(self, order_id, amount, key=None):
        if key in self.saved:  # a retry of an action the shop already did: answer again, do nothing
            reply = self.saved[key]
        else:
            self.refunds.append(amount)
            reply = f"refunded {amount:.2f} for order {order_id}"
            if key is not None:
                self.saved[key] = reply
        if self.lose_replies:
            self.lose_replies -= 1
            raise ReplyLost("timeout")
        return reply

harness.py holds the retry loop. It retries only when the reply was lost, and it stops at a limit.

harness.py
from shop import ReplyLost


class GaveUp(Exception):
    """Every try ended with a lost reply, so the outcome is unknown."""


def call_tool(call, log, tries=3):
    """Run call(). Retry only when the reply was lost, and stop after tries."""
    for attempt in range(1, tries + 1):
        try:
            reply = call()
        except ReplyLost:
            log.append(f"attempt {attempt}: reply lost")
            continue
        log.append(f"attempt {attempt}: {reply}")
        return reply
    raise GaveUp(f"no reply after {tries} tries")

compare.py runs the five steps. Each printed line starts with its step number. PASS means the shop recorded exactly one refund of 52.5.

compare.py
from itertools import count

from harness import GaveUp, call_tool
from shop import Shop

ids = count(1)


def new_key():
    """A real harness would use uuid.uuid4().hex. A counter keeps the output the same on every run."""
    return f"key-{next(ids)}"


def run(step, title, shop, call, tries=3):
    log = []
    try:
        answer = "Done, " + call_tool(call, log, tries) + "."
    except GaveUp:
        answer = "I could not confirm the refund."
    print(f"Step {step}  {title}")
    for line in log:
        print("        " + line)
    print("        agent says:", answer)
    print("        shop recorded:", shop.refunds, "PASS" if shop.refunds == [52.5] else "FAIL")


shop = Shop(lose_replies=1)
run(1, "the reply is lost and the harness does not retry", shop, lambda: shop.issue_refund(1042, 52.5), tries=1)

shop = Shop(lose_replies=1)
run(2, "retry without a key", shop, lambda: shop.issue_refund(1042, 52.5))

shop, key = Shop(lose_replies=1), new_key()
run(3, "retry with one key made before the first try", shop, lambda: shop.issue_refund(1042, 52.5, key=key))

shop = Shop(lose_replies=1)
run(4, "retry with a new key made on every try", shop, lambda: shop.issue_refund(1042, 52.5, key=new_key()))

shop, key = Shop(lose_replies=3), new_key()
run(5, "every reply is lost, then a later retry", shop, lambda: shop.issue_refund(1042, 52.5, key=key))
print("        later, same key:", shop.issue_refund(1042, 52.5, key=key))
print("        shop recorded:", shop.refunds)
Terminal
uv run --no-project --python 3.12 python compare.py
Output
Step 1  the reply is lost and the harness does not retry
        attempt 1: reply lost
        agent says: I could not confirm the refund.
        shop recorded: [52.5] PASS
Step 2  retry without a key
        attempt 1: reply lost
        attempt 2: refunded 52.50 for order 1042
        agent says: Done, refunded 52.50 for order 1042.
        shop recorded: [52.5, 52.5] FAIL
Step 3  retry with one key made before the first try
        attempt 1: reply lost
        attempt 2: refunded 52.50 for order 1042
        agent says: Done, refunded 52.50 for order 1042.
        shop recorded: [52.5] PASS
Step 4  retry with a new key made on every try
        attempt 1: reply lost
        attempt 2: refunded 52.50 for order 1042
        agent says: Done, refunded 52.50 for order 1042.
        shop recorded: [52.5, 52.5] FAIL
Step 5  every reply is lost, then a later retry
        attempt 1: reply lost
        attempt 2: reply lost
        attempt 3: reply lost
        agent says: I could not confirm the refund.
        shop recorded: [52.5] PASS
        later, same key: refunded 52.50 for order 1042
        shop recorded: [52.5]

Why a key fixes it

The caller cannot tell a lost request from a lost reply, so retrying a change is a guess. RFC 9110 says a client should not automatically retry a request with a non-idempotent method unless it knows the request is idempotent or can detect that the original was never applied. The RFC is about HTTP requests, and a refund tool that calls an HTTP API is one case of it. The AWS Builders' Library says APIs with side effects are not safe to retry unless they provide idempotency. A key gives the service a way to see that two requests are one action.

Stripe's API works this way. The client makes the key. For API v1, once the endpoint starts running, Stripe saves the status code and body of the first request made with a key, and a retry with the same key returns the saved response. An IETF draft, The Idempotency-Key HTTP Header Field, proposes a standard header for the key. It is a work in progress, not a standard.

What this does not do

  • The shop is a fake order system, and the agent is plain Python that stands in for a model. We set the lost replies. This is not a live model, a real payment service or customer data.
  • A real service does more with keys. Stripe's API compares the parameters of a retry with the first request and returns an error if they differ. Its docs also say API v1 keys can be removed automatically after at least 24 hours, and a key reused after that starts a new request. Our shop does neither. Check how long your own service keeps keys.
  • Our shop handles one call at a time. A real service must also handle two calls with the same key that arrive together.
  • The harness makes the key once per action. If the model starts a second refund call on its own, the harness will usually make a new key for it, and the shop treats it as a new action, even if the model meant it as a repeat. A key does not stop that. Guard it with a check of the shop's record, for example that no refund exists for the order. It is not shown here.
  • The harness retries only a lost reply. Other errors need other handling. In episode 1 the order lookup was blocked with a 403, and trying again with no change would not fix that.
  • There is no wait between tries. A real harness should wait longer after each failure and add some randomness, which AWS calls backoff with jitter.

Explore this topic in the workspace

Topic Breakdown gives you a structured revision guide for this topic and lets you highlight any phrase to open side-by-side chats with worked examples. Sign in to open it.

Open breakdown

Or type Retries and idempotency for LLM agent tools into the topic box on the home page. To see a lesson first, open the sample breakdown on Machine Learning. Missed the earlier episodes? Read how to find the first bad step in a trace, how to turn a failed run into a release gate and how to tell a real fix from luck. Running these calls from a queue? See why one slow LLM call stalls your queue. Want to keep raw API keys and destructive routes such as DELETE away from the agent too? Build a local ticket broker in Python.

Sources

All sources were checked on 11 November 2026. Read each page for the full context.

  • RFC 9110, section 9.2.2: Idempotent Methods

    A method is idempotent if the intended effect of several identical requests is the same as the effect of one. PUT, DELETE and the safe methods are idempotent. A client should not automatically retry a request with a non-idempotent method unless it knows the request is idempotent or can detect that the original was never applied.

  • AWS Builders' Library: Timeouts, retries, and backoff with jitter

    Published 5 December 2019. A timeout or failure does not necessarily mean the side effects did not happen. APIs with side effects are not safe to retry unless they provide idempotency, and read-only APIs are typically idempotent. Jitter adds randomness to backoff to spread retries over time.

  • Stripe docs: Idempotent requests

    The client makes the key, and Stripe suggests a random string such as a V4 UUID. For API v1, once the endpoint starts running, Stripe saves the status code and body of the first request made with a key, and a retry with the same key returns the saved response. A request that fails validation, or conflicts with one still running, is not saved and can be retried. Stripe compares the parameters of a retry with the first request and errors if they differ. API v1 keys can be removed after at least 24 hours.

  • IETF draft: The Idempotency-Key HTTP Header Field (draft 08)

    Dated 22 October 2026. A work in progress, not a standard. Its abstract says the header can make non-idempotent methods such as POST or PATCH fault-tolerant.

  • Python docs: uuid

    uuid4() creates a random UUID. It is what new_key in compare.py would call in a real harness.