There is a failure mode common to AI resume tools that is almost never described honestly in public, because describing it honestly requires admitting that your own tool does it.
The failure works like this. The model is given a CV and a job description. It is asked to score the match. The nearest evidence-shaped text in context is the job posting itself. Without explicit constraints, the model treats the posting's requirements as the candidate's background and scores accordingly. A CV with no mention of Agile, Scrum, Docker or Jira, scored against a posting that requires all four, returns a high match. The candidate did not claim those skills. The model inferred them from what was nearby.
We tested this directly. A deliberately thin CV, two generic roles, no mention of Agile, Scrum, Docker, Jira, or budget management. Scored against a posting that required all of them. Before we had guardrails in place, the analysis returned 90-95% fit with a recommendation to apply, citing sprint rituals and backlog prioritisation the CV never contained.
With the current guardrails: 29-37%, with every missing skill marked missing and a recommendation to reconsider.
The second number is far less flattering. It is also the one that saves you a week.
What causes the inflation
The model is not lying in any meaningful sense. It is doing what language models do: completing the pattern. The job description is the densest source of structured information in context. When the model is asked whether the candidate has experience with Kubernetes, and the CV says nothing either way, the posting's own "required: Kubernetes" becomes the closest relevant text, and the completion drifts toward yes.
This is not a bug in the model. It is a consequence of asking the model to evaluate without giving it a firm constraint on the source of evidence.
The constraint
Job-Agent's Memory Core is that constraint. Everything your profile contains traces to something you stated: a role you held, a responsibility you described, an achievement you quantified. The evaluation prompt reads the Memory Core and the posting together, but the rule is explicit: a claim in the output can only be supported by evidence in the Memory Core. If the evidence is not there, the gap is a gap, not an inference.
This is what "retrieval, not fabrication" means in practice. The model retrieves from what you actually provided. It does not fabricate from what the posting implies you should have.
What this means for your score
If your score is 45%, it is 45%. There are candidates who see that number and contest it, providing evidence they had not included in their profile. That is the right response. The challenge loop exists for exactly that case. But the number does not get rounded up because a higher number feels better.
The structural incentive for a resume tool is to inflate. A high score keeps users engaged, generates more document exports, and produces the feeling of progress. A 45% score that sends you back to your profile to strengthen it is a less pleasant experience and a more useful one.
The test we keep
A fixed fixture in our pipeline tests: a deliberately thin CV against a demanding posting. The expected output is in the range of 29-37%, with every missing skill enumerated. If a pipeline change causes that test to return a higher score without new evidence in the profile, the change is reverted.
We run it because the incentive to drift is real and permanent. The model does not know it is being tested. What it returns on that CV is what it would return for any user with the same background. The test is not a QA exercise. It is a constraint on the product.