The Part That Wasn’t Written Down
Anyone can build the tool now. The rare thing is the person who still knows what it forgot to ask.
Liz caught the scam before any of us knew there was a scam to catch.
It sat near the top of the list the tool had ranked for us, a nearly perfect candidate with a strong résumé and exactly the right experience. The kind of profile a scoring model rewards without hesitation.
Liz didn’t trust it. It was too clean, too precisely matched, and she had learned that the thing arriving looking flawless is often the thing worth checking first.
She was right. The résumé was bait, engineered to look ideal and get someone to click, and the click was the entire point. The model had read it and rewarded it and never suspected it was being worked, because being worked is not a category a rubric has a field for.
The scam was obvious once someone became suspicious. The more consequential things the tool missed did not look suspicious at all.
Liz founded the agency. She was its first executive director, the person who built the role out of nothing. Two years ago she handed it off, ready to try something new after decades spent building and leading organizations like this one. Now we needed a new director, and the board asked her to join us and lead the search. No one understood the job better than the woman who had invented it.
A hundred résumés came in, more than a volunteer board and a founder with a day job could read with the attention each person deserved. Big companies have software for this, platforms that swallow applicants, sort and score them, and hand back a tidy short list. We are not a big company. We couldn’t afford one, didn’t especially want one, and had no reason to buy an enterprise system for a search we hoped not to repeat anytime soon.
So I built one. I’m the board chair and Liz’s husband, and I’m the one in our house who reaches for the machine when a problem gets big. I sat down with Claude, the job description, and the hundred résumés, and by the end of the afternoon we had a rubric.
What it did, it did honestly well. It read all hundred applications, characterized each candidate against the same criteria, and normalized them so that a hundred different lives could be compared on roughly the same terms. It never got tired or impatient, and it never quietly decided that number seventy-one was someone else’s problem. It gave Liz a running start on a pile that had begun to feel like a second job.
That was the part the tool was good at. The trouble began where everyone assumes the tool is most useful, at the ranking.
A ranked list has a way of looking like an answer. It is really closer to an opening argument, and Liz argued with it.
She worked the list down with another board member, the two of them moving people up and down against what the score had decided. Some of the candidates the model loved, she moved lower. One strong, glossy candidate she set aside because the polish was itself the problem. Too corporate for a small organization that runs on relationships, resourcefulness, and people doing more than their titles suggest. She pulled others up from the middle of the pack because she could see something in them that the rubric had no way to weigh.
They narrowed the field to twelve. The model’s top pick held, and Liz agreed, so it stayed at number one. But several of the finalists came up out of the middle of the ranking, people a scoring system would have quietly let go. The tool had characterized all hundred candidates accurately and consistently. Which of them were right for the role was a different question, and not one it could answer.
Even twelve was not a decision. Getting down to three finalists took the oldest tool there is. We talked to them, real conversations with the actual people behind the résumés, the kind where you hear things no application could ever contain.
That is how we learned that one of our favorites, a candidate near the top of everyone’s list, could not make the long-term commitment the role required. Nothing about that was on the résumé, and it wasn’t tucked inside the file waiting for a sharper model to surface it. It didn’t exist anywhere until we asked and the candidate told us. Once we understood the timeline, the decision made itself. You can’t ask someone to lead a multiyear build when they already know they’ll leave partway through it.
No rubric was ever going to catch that. The information hadn’t been written down yet.
The tool will get better at all of this. It will characterize faster, rank more convincingly, and hand back lists that look more and more like answers. And the better the list looks, the stronger the pull to treat it as the decision instead of the start of one.
That is the real risk, and it runs backward from what you would expect. A tool that fails is easy to distrust, because everyone can see it fail. The dangerous one is the tool that works, the one that hands back a clean ranking and a fair summary and performs so well that the arguing and the phone calls and the long conversations start to look like friction you could cut. But that friction was the work.
The scam was only the cartoon version of the problem. Trust the confident output far enough, and you will click the thing that was built to be clicked.
The failure that worries me is a quiet one. It’s a search that goes smoothly, a tool that performs, a list everyone believes, and no one in the room who remembers what the job actually asks of the person who says yes.
We still had that person. Liz built the role, then set out to find whoever would come next, and she trusted the tool exactly as far as it could be trusted and not an inch further. What I can’t stop thinking about is how many searches don’t have her.
They have the ranked list, the confident summary, and an output everyone has quietly agreed to call the answer. They just don’t have anyone left who knows when to argue with it.
Another brilliantly written post. You had me hooked at the first line.