An error taxonomy across the wire

I was chasing a retry problem that made no sense. The orchestrator kept hammering a service that was never going to recover, and, on a different day, gave up instantly on one that just needed a second try. Same code, opposite behaviour.

The service was Python. The caller was TypeScript. And every endpoint on the Python side ended the same way.

One shape for every failure#

Here is the pattern I found wrapped around each handler.

try:
    result = do_the_work()
except Exception as e:
    raise HTTPException(status_code=500, detail=str(e))

Tidy, at a glance. Nothing leaks a stack trace, every route is consistent. The problem is what it does to the failures underneath it.

A GPU running out of memory came back as a 500 with a message. A download that got a 403 came back as a 500 with a message. A model that failed to load, and genuinely malformed input from the caller, both came back as, you guessed it, a 500 with a message.

Four completely different situations, arriving at the caller as one indistinguishable failure. The only thing that distinguished them was English prose in a detail string, and no retry policy should be parsing English.

The signal the retry layer needed#

Think about what the orchestrator actually has to decide. When a call fails, it has exactly one question: is this worth trying again?

Out of memory on a busy GPU is a transient. Wait, retry, it will probably clear. A 403 on a download is fatal. The credentials are wrong or the object is gone, and retrying just wastes everyone’s time. Bad input is fatal too, but it is the caller’s fault, not the server’s. A model failing to load might be either, depending on why.

Collapse all of that into a single 500 and you have thrown away the one bit of information the retry layer exists to act on. So it has to guess. Tune it to retry 500s and it batters the fatal cases. Tune it to give up on 500s and it abandons the transient ones. Too aggressive or too timid, and no setting is right, because the setting is being asked to recover a distinction the boundary already deleted. In practice you either pay for retries against failures that were never going to clear, or you drop jobs a single retry would have saved, and either way the customer just sees work failing at random.

A blanket except-to-500 does not hide the error. It hides the difference between the errors, which is the only part that matters.

Give failure a type#

The fix was to stop treating “it failed” as the whole story and give the boundary an explicit taxonomy. Every failure maps to a known category with a stable code, and that code crosses the wire where the caller can read it without guessing.

Concretely, retryable transients (resource exhaustion, an upstream timeout) get one family of codes and a 503-style status. Fatal errors (a 403, a missing object, malformed input) get another, with a 4xx where it is the caller’s fault. The category is a field, not a sentence. The retry policy switches on the category and never touches the human-readable message.

Same handlers, same work, almost the same amount of code. The difference is that a failure now says what kind of failure it is.

The lesson stuck with me because it is so easy to get wrong in the name of being tidy. A service boundary is a translation layer, and a taxonomy of errors is part of what it is supposed to translate. Retryable versus fatal is not a detail the caller can infer. You have to carry it across the wire on purpose.

Collapse it into one code and you are not simplifying. You are deleting the answer to the only question the other side is going to ask.