The ATS lane reads a company's own careers board rather than an aggregator's copy of it, which makes it the highest-quality source in the index. It holds 126,464 rows from 1,014 boards, all fetched inside a day, so the rotation is healthy. The only headroom left was 27,682 candidate company names that had never been probed at all.
At the lane's roughly 125 rows per board, a 10% hit rate over those names projected somewhere in the region of 340,000 jobs. That is worth a run. So I ran one band and stopped.
One band
band 1 500 names 6 boards 20.0% 6 total 30.4mSix new boards, 30 outcomes written, nothing at all written to public.jobs - which is what discover guarantees. It looks like a slow but working run.
It is not. 500 names went in and 30 came back decided. The other 470 were left Incomplete. That is 94% undecided, and it is the same shape that discovery.py already describes in a comment as a fixed bug.
The rewrite fixed the correctness half and not the throughput half
The earlier version parked a throttling host for a flat 75 seconds. A rewrite replaced that with an adaptive per-host pace, on the reasoning that this converts undecided into decided, slowly, which is the whole point.
The correctness half of that worked. The throughput half did not, and here is the measurement: workable returned 429 forty-one times and recruitee ten times inside the first 13 minutes, both climbing the pace ladder to its 30-second ceiling. A name is not decided until every one of six endpoints has answered or one has hit. So those two hosts gate every name queued behind them, exactly as the flat park did.
A pace ceiling is a slower cliff, not the absence of one.
The guard divided by the wrong number
The run carries a lean-band guard: --stop-below 15, stop after three consecutive bands under the floor. It exists precisely so a run that has stopped being worth its call budget kills itself.
It scored this band at 20.0%, comfortably above the floor, so it did not fire. That 20.0% is hit over decided - 6 boards out of 30 answers. Hit over probed is 6 out of 500, which is 1.2%.
A band where almost nothing gets decided therefore scores well on the handful of answers it did manage. The guard built to stop a lean run reads a starved run as a healthy one, and the worse the starvation gets, the better the number looks.
At 30 decided per 30 minutes, the remaining 27,652 names are about 460 hours. Nineteen days of sustained 429s at two third parties.
Why I did not just drop the two hosts
This is the obvious move and I want to record why it is wrong, because it will look right again in three months.
Workable and recruitee cause all of the throttling, and they hold 59 and 1,038 rows against greenhouse's 57,332. The trade looks free. But discover_one returns Incomplete for a skipped host specifically so that a host you did not ask does not get recorded as a company with no board. A miss is durable for retry_after_days, which is 90. Earlier in the month, every workable and recruitee company was written off for 90 days without ever being asked, and that is the bug the Incomplete state was introduced to prevent.
Excluding those hosts re-introduces exactly that, silently, and it would show up in the logs as a speedup.
Stopped after one band
Yield was about 6 boards per half hour, roughly 750 rows. The keyword sweep running concurrently was doing about 50,000 inserts per 31 minutes. So discovery was contributing around 0.15% of the other lane's rate while burning sustained refusals at two third parties, and continuing would have meant days of that.
The two did overlap cleanly, which is worth noting: 6.4 pairs/s before discovery started, 6.3-6.4 during. Discovery writes no job rows, which is the only reason they can run at the same time at all. Resume state is server-side, in ats_candidates.outcome and last_attempt_at, so re-running picks up where this left off with no local file to disagree with.
What the band summary should have printed
The real fix is throughput, and the measured fact that points at it is that four of the six hosts never push back at all. Either raise the global concurrency so names flow through the fast hosts while the pace gate independently holds the slow ones, or decide a name on the four quiet hosts and carry the workable and recruitee probes forward on a separate low-rate queue. Both need measuring and neither should land underneath a running ingest.
But the cheapest change is to the output. 6 boards 20.0% reads as healthy. 30 of 500 decided is the number that would have stopped this in one band instead of needing a database query to notice. The guard should compare hit over probed, and the summary should print the decided count next to the band size.
A metric that improves as the thing it measures gets worse is not a metric. It is a reassurance.