In one paragraph
A placement agency was running candidate intake by hand: a prospect tapped a landing CTA, a human recruiter rang back a day later, and spent five to ten minutes on the phone collecting the same structured facts every time — area of work, experience, documents, target country, readiness — before judging fit against a couple of hundred open vacancies. This project replaced that first call with a chat agent that holds the same conversation in three to five minutes, around the clock, and drops a pre-qualified queue into the recruiter’s admin. It shipped in six days and then met real candidates, which is where the interesting work started. Live traffic showed that an agent talking to real people will occasionally say something confident and false — “there are no vacancies in your city” without ever having searched — and that patching each false sentence with a phrase rule is an argument you lose one rephrasing at a time. So the agent was rebuilt around a different contract: code owns the policy, the model does language, and every claim it makes is checked against what the tools actually did before the candidate sees it.
The problem in business terms
The recruiter’s calendar was the original bottleneck, and automating the first call removed it. What replaced it was subtler and more expensive: a system that speaks to customers in natural language can be wrong in ways a form never could. A form that fails is visibly broken. An agent that fails is fluent — it tells a qualified welder in Kraków that there is nothing for him, and he believes it, and he goes to a competitor, and nobody ever files a bug. The business risk is not downtime. It is a confident sentence with nothing behind it, delivered to the one candidate who was worth placing. The maintenance shape tells the same story from the other side: eighty-six of the first hundred and eighty-two commits were bug fixes, and each new rule collided with the last.
- each fix bans one phrasing the model used last week
- the model rephrases around the ban; the bug returns
- roughly 25 regex backstops reading a finished draft
- a false «no vacancies» is indistinguishable from a true one
- nobody can say afterwards which claim was grounded
- the model names its move up front, from a closed set of 11
- code checks that move against what the tools actually did
- an ungrounded claim is re-run, downgraded or executed for real
- a «no vacancies» line is only ever emitted after a real search
- one evidence fold per turn — auditable in the admin trace
What makes this different from “a chatbot wired to a database”
The load-bearing decision is where policy lives. In the first version the model decided what to do and prose rules tried to correct it afterwards, which meant every fix was a guess at a formulation and the guesses never converged. Now the model does the two things models are genuinely good at — understanding what the candidate said, and saying something warm back — and everything else is code. A cheap extractor re-reads each message before any generation, so the whole turn sees a post-comprehension profile. Intake turns are planned by a deterministic ladder; the model’s only job on those turns is one warm sentence with hard honesty bounds. The turns that can touch vacancies return a flat structured reply whose first field is a named intent, and naming the move up front is precisely what makes it checkable: the step trace is folded once into an evidence record, each claiming intent is tested against its grounding condition, and a claim with no evidence is remedied deterministically — the search is re-run for real, or the intent is downgraded, or the withdrawal the model only talked about is actually executed. The final message is then assembled in code, so vacancy facts are never something the model writes.
The second decision is one that gets mistaken for a compromise. The model provider is fixed by EU data residency, because the system holds candidate names, phone numbers and work histories. That constraint is also what forced the architecture to be good: when you cannot buy your way out of a reliability problem with a stronger model, you have to design the honesty in. Within the permitted provider, the mid-tier model was trialled and rejected — pricier and worse at holding conversational state, which is this agent’s entire job.
| criterion | prose rules + regex backstops | structured intent verified against evidence chosen | buy a stronger model |
|---|---|---|---|
| catches a claim never seen before | no | yes | partial |
| fix survives a rephrasing | no | yes | partial |
| auditable after the fact | partial | yes | no |
| clears EU data residency | yes | yes | no |
| cost per conversation | baseline | lower | 3–5× higher |
The provider is not a quality decision, and treating it as one is how teams in regulated flows end up with an architecture they cannot defend. This system holds names, phone numbers and work histories, so the provider is fixed by residency and a data-processing agreement. “Use a stronger model” only becomes an option once a stronger model clears both.
A session in the room
Before the work starts
Two files of context open every session. One describes what the system is and which decisions are settled; the other is the architecture map, generated from the code rather than written alongside it, so it cannot quietly drift from what actually runs.
The three sessions that follow are the three shapes this work actually takes. One is a defect found by driving fake candidates through the real agent, which turned out not to be an agent defect at all. One is a bill that was four times what its own dashboard reported. One is the decision to spend money on an offline extraction pass, and the evidence gathered before spending it.
None of them is a feature. All three are the kind of work that decides whether a system like this survives its second month, which is the month where most of them quietly stop being trusted.
Finding the defect before a candidate does
The agent is tested by another agent. A language model plays a candidate — given a persona’s facts but instructed to reveal them only when asked, one at a time, in the register real people actually use, which is often terse, lowercase and mixed Ukrainian-Russian. That is a far harder test than a scripted conversation, because it probes what the agent does when information arrives in the wrong order.
This round found the expensive kind of failure. A candidate asked for meat-production work and was told there was nothing open, five turns in a row, while four such vacancies sat in the index. Nothing crashed. Nothing was logged. Commercially it is the worst thing the system can do: a candidate who was placeable, quietly not placed.
The root cause was three characters below where anyone was looking. Ukrainian is written with an apostrophe, and the corpus contained six different apostrophe glyphs — a typographic detail that made the same word stem three different ways depending on which key the candidate’s phone produced. Every profession containing one had been unsearchable since before the rebuild, and the extracted synonyms had inherited the damage at extraction time.
What is worth taking from this is not the fix, which is a few lines of normalisation. It is that the harness surfaced it as one named candidate was wrongly told no rather than as a vague sense that search was sometimes flaky. Twenty-eight rounds of that ran before launch and after it, each defect traced to a real transcript and each closed with a regression that fails loudly if it returns.
A bill hiding behind its own dashboard
Unit economics decide whether an intake agent is a product or a science project. At a fraction of a cent a conversation you can put it in front of every inbound lead. Near a dollar, someone starts asking which leads deserve it — and the moment that question gets asked, the automation has stopped being automatic.
The cost dashboard was under-reporting by about four times, because it still priced the conversational model at a rate the provider had since changed. That is the failure mode worth naming: the number was not missing, it was wrong and comfortable, and it had been reassuring everybody for weeks.
Behind it sat the real finding. Roughly a quarter of a million input tokens against four thousand output tokens means the conversation is almost entirely one static prompt, re-sent at full price on every turn. Providers will serve that prefix from cache at a tenth of the rate, but only when the request carries a cache key — and the library in use had no way to pass one, so it goes into the request body on its way out.
Then the same audit found the mistake repeated one layer down, where the per-turn text had been placed ahead of a seven-thousand-character rule block, leaving a cacheable prefix of roughly two hundred characters. Identical text to the model, wrong order, full price on every turn. About seventy per cent of the bill came off, and not one candidate-facing sentence changed.
Deciding to spend, then proving it was worth it
Vacancy rows in the source system are free text written by whoever posted them. Ninety-eight per cent carry stray markup; the average one is more than twice as long as the space a chat card can give it. Somewhere in that prose sits whether there is housing, whether the role is age- or gender-restricted, what the hours actually are — the facts a candidate asks about first, and exactly the facts a keyword search cannot reach.
Nearly a third of the catalogue states an age limit in prose alone. There is no age column to filter on, which means without extraction the agent either shows people jobs they cannot take, or hides jobs they can. That is the argument for spending money on an offline pass, and it is a business argument rather than a technical one.
The spend itself is bounded by design: extraction happens once per vacancy, gated on a content hash, so an unchanged row is never paid for twice, and vacancies that disappear from the source are closed with plain SQL — no model involved, so the index cannot go stale in the dangerous direction even when the extractor is entirely unavailable.
The frozen set is the part that would be easy to skip. Sixteen hand-labelled vacancies, re-extracted on demand, with the fields that gate what a candidate is shown held to full agreement. A prompt tweak that quietly degrades extraction is otherwise invisible until someone is shown a job they cannot legally take. The whole catalogue came to about six dollars, which is the number that made the decision easy.
What’s shipped
- ● 01comprehension passre-reads the message before any generation · fills blanks only
- ● 02state + tool gatederived fresh each turn · an under-age block strips placement
- ● 03plan or generateintake is a deterministic ladder · the rest is a tool loop
- ● 04evidence foldone pass over the step trace · searches, records, withdrawals
- ● 05verify + remedyungrounded intent → re-search, downgrade, or execute for real
- ● 06compose + syncthe only place the final message exists · one question, always
- ● 01marketing landingembedded in the agency's existing monorepo · no new pipeline
- ● 02chat-app agentone vacancy per message, so a reply can name a card
- ● 03recruiter admintranscripts with the tool trace · profile · status workflow
- ● 04offline indexerhash-gated re-extraction · reconciled on every deploy
The conversation is grounded in two things and nothing else: real vacancy rows from the agency’s own system, and a small curated reference of labour-law facts alongside live currency rates. There is no invented market data anywhere in the path — when the agent does not know, the honest answer is code, not a rule the model is asked to remember. Handoff to a human is gated on a complete profile and a saved phone number, and the compliance surface is enforced at the schema: deleting one candidate erases their entire footprint in a single statement.
The cache is prefix-based, which makes the system prompt’s assembly order load-bearing. Move a per-turn block ahead of the static one and the cacheable prefix silently shrinks to whatever precedes it. No test fails. The bill just goes back up, and nothing in the repository tells you why.
What an intake agent actually costs
Two numbers decide whether this is a product or a science project: what a conversation costs to run, and what the same conversation would have cost in recruiter minutes. Plug in your own volume.
runs locally · 0 network calls · deterministic
The verdict from the room
What this system removes is not a task but a category of risk. Any organisation putting a language model in front of its customers eventually meets the same question — how do you know it did not just make that up? — and most answer it with a longer prompt. This one answers it with a record: the agent names its move, code checks the move against what the tools did, and the whole exchange is readable afterwards in the admin trace. That is the part that transfers. The recruitment domain is incidental; the contract between code and model is not.
It automated the first call, which was the ask, and then it did something more useful — it made the automation defensible to a customer who asks why they were told no.
Verifying intent against tool evidence is the move that ages well; my one reservation is that the deterministic remedy path is now load-bearing enough that it deserves the same adversarial testing the model layer gets.
Twenty-eight regression rounds and a frozen extraction set mean the failures we already found stay found — and the indexer degrades in the safe direction when the model is unreachable, which is what I actually care about at 2am.
Six days to first ship. Two months of live traffic. One builder. 1,190 pinned behaviours, seventy per cent of the bill removed, and an agent whose every factual claim has something behind it.
This case describes architecture and patterns. The agency, the source recruitment system, the bot’s public identity and the internal URLs that would name any of them are deliberately abstract. The audience — Ukrainian-speaking blue-collar workers seeking placements across the EU — is part of the product’s public positioning and appears for the same reason it appears on the landing.