The bug the happy path can't show you

Some bugs only exist in the shadow of the happy path. Run the normal flow and everything is correct. It is the second job runner, or the failed second write, where the wheels come off. Two of those.

The isolation that only worked on the request#

Multi-tenant isolation is the kind of thing you want to enforce once, centrally, and never think about again. In one system that meant a request-scoped database client that automatically filtered every query by the authenticated user. Pull the user id off the request, extend the client so every read is scoped to them, done. No controller can forget, because the filter is not the controller’s job.

It is a lovely pattern right up until work leaves the request. A background job processor (an OCR run, a nightly digest) has no HTTP request in scope. So the request-scoped client simply is not available there, and the tenant filter that protects every controller protects none of your background work. In practice that means one customer’s documents could surface in another customer’s results the moment an OCR job or a nightly digest ran, which is exactly the kind of leak a multi-tenant app exists to prevent.

The fix was not to bolt request-scope onto the workers. It was to stop deriving identity from the request implicitly and make it an explicit parameter that both paths carry. The job payload includes the user id; the worker constructs a client scoped to it, using the same filtering. The guarantee now travels with the work instead of with the transport.

If your security depends on where the code is called from, it has a blind spot everywhere it is called from somewhere else. Make the guarantee a parameter, not an ambient fact.

The aggregate that saved itself in halves#

The other one was a create method that saved twice. Persist the parent record. Then add the child rows and save again.

On the happy path, flawless. But those two saves are two transactions, and the gap between them is a window. If the second save fails, the parent is already committed, sitting there with no children: an aggregate in a state the domain says can never exist. Nothing errors loudly. You just have invalid data that every later read has to defend against.

The fix was almost nothing: delete the first save. Let a single trailing save persist the whole aggregate in one transaction, so it either all lands or none of it does. The important part was the test. It injected a database context rigged to throw when the child rows are pending, then checked, from a fresh read, that no orphaned parent was left behind. Atomicity you cannot prove is just atomicity you are hoping for.

The common shape#

Both bugs pass every test where nothing goes wrong, because nothing going wrong is exactly the condition under which they are invisible. The tenant filter is fine until the code runs off the request path. The two-step save is fine until the second step fails.

Writing the unhappy path down, on purpose, is the whole job. Ask where this runs that I am not picturing, and what happens if the second thing fails after the first one already committed. The answers are where your real bugs are hiding.