Jobzyl searches 20 job boards and thousands of company careers pages in one query, over an index of more than two million live postings. People assume the hard part is the fetching. It is not. Any afternoon gets you a fan-out across twenty APIs.
The hard part is what you are willing to say about a listing once you have it. An aggregator inherits every gap in every source it reads, and it inherits them silently: the listing that was filled three weeks ago, the salary that is not the employer's number, the figure that is a day rate wearing an annual costume, the same role sitting on four boards under four URLs. Pass all of that through as fact and you have built something worse than the twenty tabs it replaced, because now the twenty tabs have one confident voice.
So the rule the whole backend enforces is narrow: never state something the data does not support. Here is what that cost.
Zero does not mean unpaid
Start with pay, because pay is where a job seeker's trust actually lives.
A provider returning 0 for salary means it had nothing. It does not mean the role is unpaid, so zero is dropped rather than rendered. That one is easy. The interesting case is a figure like GBP 450.
I know GBP 450 is not an annual salary. I do not know what it is. It could be a day rate, an hourly rate on a badly parsed field, or a typo with three zeroes missing. Inferring "daily" would look like a fix and would actually be worse than the bug: it replaces a wrong fact with an invented one, and an invented one is much harder to spot later.
# The gate, in spirit: convert only to check, never to display.
if amount == 0:
return None # the provider had nothing
if not plausible_annual(amount, period):
return None # wrong, and I cannot tell how
return Pay(amount, period) # in the period the employer quotedPay renders in the period the employer quoted, or it does not render. There is no normalising step that quietly turns an unknown into an annual figure.
A model's output is plausible by construction
Then a measurement I did not expect. Adzuna returns a flag saying when a figure is its own market estimate rather than a number the employer wrote. I checked how often that flag was set: on 2026-08-18, 49 of 50 US data-scientist rows that carried any pay at all were predicted.
A plausibility check cannot catch this. The whole point of a good model is that its output looks like the real thing.
Those figures had been rendering as quotes, under a promise of employer-stated pay. They are now out of aggregation entirely and styled distinctly wherever they still appear. The general lesson is uncomfortable: the values most likely to slip past your validation are the ones generated by something that was optimised to pass it.
An estimate that only speaks when the employer did not
Two thirds of listings state no pay at all, and "salary not specified" is a genuinely bad experience. So there is an estimate. It is a lookup over measured cells rather than a model, it never argues with a stated figure, and it has no country-wide fallback, because a national median tells a nurse nothing whatsoever about nursing.
One detail there took a rewrite. Cells have to be deduplicated per job before anything is counted. A single poster ran the same role 220 times at an identical figure, and on raw row counts that one poster set a UK salary cell on their own. Every UK engineer reading that number would have taken it as the market. A median is only as honest as the assumption that its rows are independent, and job postings are emphatically not.
The listing that was already gone
Until this month nothing revalidated a stored posting. I sampled the newest 40 rows per site and 12.1% of what was being served was already dead, over half of it on two of the boards.
So there is a liveness sweep now, and the design constraint is the same one as before: only positive evidence marks a row dead. A 404, an explicit gone-page, or a redirect that drops the posting id. A 403 or a timeout writes nothing at all.
That restraint is not fussiness. 170 of 593 probes came back as refusals, and treating a refusal as an answer would have buried 168,238 live rows in a single pass. Nor can you outrun it: at concurrency 10, 126 of 200 probes came back unknown, so pushing harder converts live rows into unknowns faster rather than resolving more of them.
The same job, four times
Cross-board duplicates are the most visible failure to a user and the least visible to a test. The identity key is accent-folded, punctuation-stripped, with company legal suffixes dropped, and it runs at write time and again on every read path, because write-time dedup can only ever see one batch and a duplicate that arrives tomorrow is still a duplicate.
61,563 redundant rows have collapsed so far. The losing copies are recorded rather than deleted, which turns the whole problem into a feature: a row can say which other boards it also appeared on.
What I still cannot say
The same rule has to point back at my own claims, and this is the part that stung. Not one of the three headline numbers on my own project page had been measured.
- It used to say 2M postings, on no evidence at all. The index did pass 2M on the 2026-08-19 sweep, which is the least satisfying way to be right: a guess that lands is still a guess, and I only know it landed because I finally counted. The number stays on the page because it is now a census with a date on it, not because it was there first.
- It used to say 60+ countries. That figure came from counting distinct search-location strings, so it was counting query regions and multi-counting any country I had searched under several city names. It has been pulled from every surface.
- It used to say first results streamed in about 1.4 seconds. Nothing has timed that path since June, so it is withdrawn rather than repeated.
And two live caveats I would rather write around than hide. Coverage is uneven: five sources account for 77.8% of the corpus, and on the last census six of the twenty disclosed boards had contributed no rows at all. The honest verb is that a search queries up to twenty boards, not that the index draws on twenty evenly. And liveness, for all the care above, had been probed on 0.32% of the corpus. The filter ships. The coverage does not exist yet.
None of this makes the product look bigger. It is the only version of it I would put my name on, and it is the same instinct that made me re-measure my dissertation evaluation after submitting it and publish the correction. A number you cannot reproduce is not a result. It is a claim, and claims are cheap.