An HR manager needs to screen three hundred resumes for one role - fast, fairly, and in a way that survives a bias audit
Why free-form 'rank these resumes' AI screening is a legal and fairness risk, and the rubric-first alternative that keeps a human accountable for every decision.
Last reviewed 1 September 2026
THE PROBLEM
An HR manager at a growing company has 300 applications for a single senior role and needs to get to a shortlist of 10-12 for the hiring manager within a week. Reading 300 resumes properly is a multi-day task, and the obvious shortcut - upload them all and ask an AI tool to 'rank the best candidates' - is fast and is also exactly the kind of automated employment decision tool that has drawn real regulatory attention and real evidence of bias.
Free-form AI resume ranking has a documented failure mode: without an explicit, defined rubric, the model will latch onto proxies correlated with protected characteristics - the prestige of a university name, gaps in employment history, even patterns correlated with gendered names - and present its ranking as if it were purely merit-based, with no visibility into why any candidate was ranked where they were.
Where this task is done (New York City's Local Law 144, for one) explicit legal obligations already exist around automated employment decision tools, including an independent bias audit and candidate notice requirements - a manager who doesn't know whether their screening process counts as an 'automated employment decision tool' under a law like this is carrying real, unmanaged risk.
THE APPROACH
Write the screening rubric before touching any AI tool - the specific, job-related criteria that will determine who advances, defined the same way a structured interview scorecard is defined, before any resume is read. Use AI only to help apply that fixed rubric consistently across 300 resumes faster than a human could alone, scoring against the stated criteria and showing its reasoning for each score - never to freely rank candidates against an undefined, implicit sense of 'best fit.'
A human reviews every score the AI produces, especially every rejection, before any candidate is removed from consideration - the AI's output is a fast first pass a person checks, not a decision a person merely rubber-stamps.
WHY IT WORKS
A rubric defined and fixed before screening starts (years of a specific required skill, a specific certification, demonstrated experience with a named type of project) is a job-related criterion that can be defended and audited. 'Best fit' with no defined criteria is not a criterion at all - it's an invitation for the model to substitute a proxy, and proxies are exactly where bias enters.
Requiring the AI to show its reasoning against each rubric item, per candidate, converts a black-box rank into a checkable record: a human reviewer (and, if it ever comes to it, an auditor) can see specifically why a candidate scored low on a specific criterion, rather than trusting an unexplained overall number.
Human review of every rejection, not just spot-checks, is what keeps a person legally and ethically accountable for the actual decision - the tool assists the screening, it does not make the employment decision.
STEP BY STEP
1.Write the rubric from the job requirements, before screening starts
List the specific, job-related criteria that matter for this role - required years of a named skill, a specific certification, demonstrated experience with a defined kind of project. Each criterion should be something you could defend in a conversation with a rejected candidate.
2.Check whether your jurisdiction's AEDT rules apply
If operating in a jurisdiction with automated-employment-decision-tool requirements (New York City's Local Law 144 is the most established example), confirm whether this workflow counts as one and what bias-audit and notice obligations follow - this is a compliance question worth checking before, not after, deployment.
3.Score against the rubric, not a free-form ranking
Prompt the AI tool to score each resume against each specific rubric criterion individually (e.g. 'does this resume show 3+ years of experience with X: yes/no/unclear, quote the relevant line'), rather than asking for a single overall rank or a 'best fit' judgment.
4.Require quoted evidence for every score
For every criterion score, require the tool to quote the specific line from the resume it based the score on. A score with no quotable evidence should be treated as 'unclear' and sent to human review, not trusted.
5.Human review of every rejection before it's final
A person reviews every resume the process scored as not advancing, specifically checking the quoted evidence against the rubric - this is where an obvious proxy-based misread (an unusual but relevant career path scored low for looking non-standard) gets caught before a candidate is lost.
6.Keep the rubric, scores, and evidence on file
Retain the rubric used, every candidate's per-criterion scores and quoted evidence, for the same retention period your other hiring records require - this is the record that makes the process auditable and defensible.
TOOLS
Warden AI
PaidBias-audit and compliance platform specifically built around automated-employment-decision-tool requirements like NYC Local Law 144.
www.warden-ai.com (opens in a new tab)Claude
FreemiumScoring resumes against an explicit, pre-written rubric with quoted evidence per criterion - not free-form ranking.
claude.ai (opens in a new tab)
LIMITATIONS
A well-written rubric reduces but does not eliminate bias risk - if a rubric criterion is itself a poor proxy for the actual job requirement (e.g. weighting a specific university tier rather than the underlying skill it's meant to signal), the process will faithfully and consistently apply that flawed criterion at scale. Reviewing the rubric itself for job-relatedness is a distinct step from reviewing the AI's scoring.
Regulatory requirements in this space are evolving and vary significantly by jurisdiction - what counts as an 'automated employment decision tool,' what audit and notice obligations apply, and how they're enforced differs by city, state and country. This entry names one well-established example (NYC Local Law 144); it is not legal advice for any specific jurisdiction, and a compliance or legal review before deployment is a genuine requirement, not a formality.
This workflow is for the screening pass that narrows 300 applications to a shortlist. It should not be the sole basis for a final hiring decision - the structured interview and reference-check stages remain a human process this entry doesn't attempt to automate.
Advertisement
EXAMPLE
300 applications for a senior backend engineering role need to become a shortlist of 10-12 within a week.
Rubric written with the hiring manager: 5+ years backend experience (specific stack named), demonstrated experience scaling a system past a defined size, any relevant certification, evidence of technical leadership (mentoring, architecture decisions).
Local Law 144 applicability checked with legal counsel given the company's NY presence - confirmed to apply, existing bias audit vendor already in place from a prior hiring cycle.
All 300 resumes scored against the four criteria individually, with quoted evidence per score.
38 resumes flagged 'unclear' on at least one criterion for lack of clear evidence, plus all 60 scored as not advancing on any criterion - all 98 reviewed by the HR manager directly against the quoted evidence.
Two candidates were moved from 'not advancing' to the shortlist after human review found the AI had scored an unconventional but clearly relevant career path (a candidate who'd led infrastructure at a smaller company under a different job title) as unclear rather than a match.
A defensible, evidence-backed shortlist of 12 candidates produced in three days instead of a week, with a full record of the rubric and per-candidate reasoning - and two strong candidates a pure keyword or 'best fit' ranking would likely have lost.
RELATED
Nakoda editorial · last reviewed
This entry describes a workflow Nakoda recommends - it is not a claim about how any named tool behaves in every case, and it is not paid placement. Spotted something out of date? Tell us.