Lindsay Edwards

Startup order is a real thing

Most of the time you can ignore the exact order things happen at startup. The framework wires itself up, your code runs, everyone is happy. Then occasionally the order is the whole bug, and you spend an afternoon learning that lesson properly.

Fail before you can serve a request#

A service had critical secrets it could not run safely without: encryption keys, webhook signing secrets. The obvious place to check they exist is the framework’s config system. The problem is that the config system runs inside the dependency injection container, and the container loads after some modules have already started initialising.

So a “proper”, framework-integrated check runs a few milliseconds too late. In that gap, code that depends on those secrets can already be wiring up, and the worst case is a server that starts and quietly handles requests without the security controls it was supposed to enforce. A validation that runs after the thing it protects is decoration. Put plainly, the app could accept and serve live traffic with its protections not yet switched on, which is a smoke alarm you turn on once the fire is already out.

The fix was almost aggressively simple: a plain function called at the very top of the entrypoint, before the container initialises at all. Missing a required secret in production throws and the process refuses to start. It logs the secret’s name and that it is absent, never its value. Not elegant, exactly, but it runs first, which is the only property that mattered.

Fail-fast only works if the fail runs before the thing it is guarding. Sometimes that means a boring function at the entrypoint instead of the framework’s proper hook.

The logger that was already frozen#

The other one came from the test harness, and the error message was a great little mystery: “The logger is already frozen,” followed by the whole integration suite refusing to start with “entry point exited without ever building a host.”

The cause was a helper trying to seed test data. To get at the services it needed, it called BuildServiceProvider() while the app was still configuring services. That builds a second, throwaway container, and the act of building it finalised the bootstrap logger. When the real host then went to build itself, the logger it expected to configure exactly once was already locked, and the whole startup collapsed.

The fix was to stop building a throwaway provider mid-configuration and do the seeding against the real provider, after the host was actually built. Which is the general lesson: calling BuildServiceProvider() while you are still registering services is a classic foot-gun. It spins up a second container and can freeze singletons, loggers, options, config, that the framework is planning to set up exactly once. Do your setup work against the real, finished container, not a temporary copy of it.

Both of these are the same idea in two different forms: when something runs is part of whether it is correct. A security check has to run before the code it guards. Container setup has to happen once, in order, against the real container. On the happy path you never see it. In the first few milliseconds of a cold start, the order is the bug.

Keep reading