Green can be bought
I wrote a tool that finds every hard delete in the codebase and fails unless each one is either guarded or explicitly justified. It reported ten sites, zero unreviewed. Green on the first run.
Green was partly bought.
The scan matched .delete({, so every deleteMany({ was outside its field of
view. It matched only calls on the main client, so deletes inside transactions
were invisible too. And the routine that extracts a function’s body had a silent
failure path: if it couldn’t find the closing brace it returned most of the
file, in which every model name appears somewhere, so the guard passed by
accident.
The part that stung: I had described four of those sites as “deliberately kept after review.” I hadn’t reviewed them. I remembered them. Memory was doing the job that call-site discovery exists to end — which was the entire argument for building the tool.
The dangerous moment is the first failure
When a new checker lights up red, the cheapest route to green is an exclusion list. And every exclusion is a judgement made in the same pass that discovered the problem — the worst possible moment to judge, because you want the green.
Four of mine said, in effect, guarded elsewhere. Which is true, and is the one property the tool existed to stop trusting. A guard on a term delete had existed for months; it just counted the wrong relation.
Promoting those four into the checked set failed two of them immediately, and one was money: deleting a grade with no classes destroyed its fee-book lines, because the guard counted classes and nothing else.
What made it honest
Not passing — failing. Deliberately.
Remove an entry, watch it report the site as unreviewed. Rename a guard function, watch it refuse rather than quietly find nothing. Break the terminator and confirm it errors instead of matching the whole file. Four failure modes, each demonstrated by causing it.
A passing test tells you nothing until you have seen it fail. Until then you’ve confirmed that a thing which might be checking nothing reports no problems.
And the number to distrust most is the reassuring one. Zero unreviewed measured
my tool’s field of view, not the system’s health. Widening the view found a
raw DROP DATABASE that no version of the scan had ever seen.