Notes from writing a vulnerability scanner

I spent a while building automated security scanners, and it turns out writing the attacker and the detector at the same time teaches you more about both than doing either alone. A few things that stuck.

An oracle is only as good as the improbability of its marker#

The textbook test for template injection is to send {{7*7}} and look for 49 in the response. It works in a demo and then drowns you in false positives, because 49 shows up in prices, pixel sizes, and ids all day long. Same for 14, 100, 144. A two-digit number is not evidence of anything. Get this wrong in either direction and the tool is useless: bury a security team in false alarms they soon learn to ignore, or miss the real hole that lets an attacker in.

Two fixes, both about making the marker improbable. Use a payload whose result basically cannot occur by chance: multiplying a string, so 7 * "7" yields 7777777, a seven-digit repeated marker you will not find in a price. And never trust a single channel: prove injection exists at all with a polyglot payload first, then confirm with an engine-specific fingerprint or a time-based payload where the response has to be slower by a measured margin. One improbable marker plus one independent corroboration beats a hundred lucky guesses.

Confidence is a number, anchored to how forgeable the evidence is#

Findings should not be true or false. They should carry a confidence score, and the score should reflect how hard the evidence would be to fake. When probing for server-side request forgery, leaked cloud credential keys are basically unforgeable, so that scores near certainty. An internal service banner is strong but weaker. A pure timing anomaly is inherently noisy and gets capped well below certainty, computed from the delta over a measured baseline, so a slow server can never masquerade as a confirmed breach.

Security signals differ enormously in how easily they can be faked. Bake that into the score. Unforgeable evidence earns high confidence; side-channels like timing should be capped so they can never pretend to be proof.

Treat every value the other side controls as hostile, including alg#

The JWT attacks make the point cleanly. The none-algorithm attack strips the signature and sets the algorithm to none, and a good tool tries the case variants too, None, NONE, nOnE, because sloppy allowlists compare against a lowercase string and miss them. The algorithm-confusion attack takes an RS256 token and re-signs it as HS256 using the server’s public key as the HMAC secret, exploiting any verifier that lets the token choose its own algorithm.

Both the vulnerable code and the test tool have to treat the alg header as attacker-controlled. On the defending side the fix is one line of principle: require an explicit expected algorithm at verification time and never let the token pick. On the attacking side, always enumerate the case and encoding permutations, because real bypasses live in the gap between how a value is set and how it is checked.

Your own outbound calls are an attack surface too#

The sharpest lesson was hiding in a feature, not a scan. An outbound webhook delivery signed its payload with HMAC but did no validation of the destination URL. On a service that also has network reach into internal ranges, that is a perfect request-forgery pivot: point the “webhook” at a cloud metadata endpoint or an internal admin panel, and your own trusted worker fetches it for you.

Signing the payload protects the receiver, not you. The same forgery class you scan other people’s apps for applies to every outbound call you make: webhooks, callbacks, image fetchers, link previews. Allowlist destinations, resolve the hostname and pin the IP, and block private and link-local ranges on egress.

And truncate everything the target sends back#

One small habit that saves you repeatedly: every byte that came from the thing you are probing is untrusted and potentially enormous (a file:// read, a metadata dump). Truncate response bodies, evidence snippets, and error strings at the boundary where you store or log them, as a default. Otherwise the first hostile target with a giant response takes out your database or your log pipeline instead of showing up as a finding.

The through-line, and the reason a defender’s eye helps when building anything: assume the input is trying to hurt you, prefer evidence that cannot be faked, and remember that the calls you make are inputs to someone else’s system too.