Normalise at the boundary, not everywhere else

My similarity thresholds would not stay put. A value that cleanly separated matches one week was letting rubbish through the next, and I had not touched the query.

The model was returning raw vectors, and I was doing the length maths at read time in more than one place. Some of those places normalised. Some did not. Cosine similarity across a mix of the two is measuring nothing you can set a threshold on.

One decision, made once#

A vector similarity comparison only behaves if both sides agree on scale. If every vector is unit length, a plain dot product gives you cosine directly, and a threshold of, say, 0.8 means the same thing every time.

The moment some vectors are unit length and some are raw, the dot product is partly measuring direction and partly measuring magnitude. Your threshold is now comparing apples to whatever size apple happened to come out of the model.

So I stopped normalising at read time and moved it to write time. Every vector gets scaled to unit length once, at the boundary, before it is stored. Downstream code can then assume unit length and use a plain dot product without thinking about it.

// normalise once, at write, and guard the zero case
function toUnit(v) {
  const len = Math.sqrt(v.reduce((s, x) => s + x * x, 0));
  if (len === 0) return zeroVector(v.length); // sentinel, see below
  return v.map(x => x / len);
}

The divide-by-zero guard matters more than it looks. A zero vector has no length, so dividing by it gives you not-a-number across the whole array, and one poisoned vector quietly wrecks every comparison it touches.

A sentinel beats a nullable#

Sometimes the model fails and there is no embedding to store. My first thought was a nullable column, but that pushes a null check into every query and every bit of maths downstream, and you only need to forget one.

Instead I return a zero vector as a sentinel meaning “no embedding”. It is a real value, so nothing downstream trips over a null. And because I already guard the zero case during normalising, it never divides. At query time I filter zero vectors out, so they never pretend to be a match.

Decide once whether a value is normalised and enforce it where the value is born. A sentinel you can filter beats a nullable you have to remember.

That cleanly represents absence without a nullable column and without special cases scattered through the code. A missing embedding is just a value that never matches anything, which is exactly what “missing” should mean.

Decide once, at the boundary#

Mixing normalised and raw vectors makes your thresholds meaningless, and the failure is quiet because the numbers still look plausible. In a product that shows up as a search or a recommendation that used to surface the right thing and now returns junk, with no error anywhere to point at. The fix is not more careful maths at every call site. It is deciding once, at the boundary, that every stored vector is unit length, and enforcing it there.

Do the normalising at write time, guard the divide-by-zero, and use a zero vector as a sentinel for absent rather than reaching for nullable. After that, downstream code gets to be simple, because the hard decision was already made before the value arrived.