AI Pre-Employment Assessments: How They Work, Validity and Legal Risk

By Kevin Kim · Updated Oct 11, 2026

An AI pre-employment assessment is a test taken by a job candidate (a game, a job simulation, a recorded video interview or a skills exercise) whose responses are scored by a machine-learning model that predicts how well the person would perform in a role. The output is usually a score, a ranking or a pass/fail band that feeds a hiring decision. The test is only as good as two things: whether the score really predicts job performance (validity), and whether it treats groups of candidates fairly (adverse impact).

This page is for HR directors, talent acquisition leads and hiring managers who are being pitched these tools or already use them. It explains how the scoring works, what the research says about which assessment types predict performance, how adverse impact is measured, and where legal exposure comes from. It's general information, not legal advice. For the wider recruiting picture, start with the AI in recruiting hub.

Key Facts

• Under the US Uniform Guidelines, a selection rate for any group that's less than four-fifths (80%) of the rate for the highest-rate group is generally treated as evidence of adverse impact, though smaller differences can still count (29 CFR 1607.4(D)).
• A 2022 re-estimate of selection-method validity put structured interviews at r = .42, job knowledge tests at .40, work sample tests at .33 and cognitive ability at .31 (HumRRO on Sackett et al., 2022).
• New York City requires a bias audit of an automated employment decision tool within one year of its use, a public summary and candidate notice, and enforcement began on July 5, 2023 (NYC DCWP).
• The EU AI Act lists AI tools for employment and worker management as high-risk, with those rules applying from 2 December 2027, and it prohibits emotion recognition in workplaces, a ban in force since February 2025 (European Commission).

What Counts as an AI Pre-Employment Assessment

The category covers four formats, and they differ in how much the AI is doing.

Simple scoring (counting correct answers against a key) isn't really AI. The AI part arrives when a model learns which response patterns to treat as good, or when it reads open-ended output such as speech and writing.

How the Scoring Works

Most AI assessments follow the same four steps, which map onto the Ingest, Analyze and Predict stages of the ACE Framework.

1. Capture. The platform records raw behavior: clicks, response times, answers, a video, a transcript, code submitted.
2. Feature extraction. The raw data becomes measurable features, such as accuracy under time pressure, how many words a candidate used, or the topics covered in an answer.
3. Model scoring. A trained model maps features to a score. Training needs a "criterion", meaning a measure of job success, such as manager ratings or sales results for people already in the role.
4. Banding and routing. The score is turned into a rank or a band (recommended, review, not recommended), and the candidate is routed to the next stage.

Two details decide whether the result is trustworthy. First, what the model was trained to predict. If the criterion is "people we hired and kept", the model learns your past preferences, flaws included. Second, whether the model's features are job-related. A feature that merely correlates with success in your historical data can quietly stand in for age, gender or disability.

The best-known cautionary case is a recruiting model Amazon built, which scored candidates from one to five stars after learning from about ten years of resumes, mostly from men. It penalized resumes containing the word "women's" and downgraded graduates of two all-women's colleges, and executives lost faith in it and disbanded the team (Reuters report, as carried by Euronews). It was a resume tool, not an assessment, but the mechanism is identical: learn from a skewed past, reproduce it at scale.

A second case shows a vendor correcting course. HireVue said in January 2021 that it had removed facial analysis from its video assessments, explaining that language analysis had become more predictive and visual analysis no longer added significant value (SHRM). The lesson for buyers is to ask what signals a model uses, and to refuse signals you can't tie to the job.

Validity: Does the Score Predict Job Performance?

Validity is the question that matters most and gets asked least. A valid assessment produces scores that correlate with later job performance. The correlation is expressed as r, where 0 means no relationship and 1 means perfect prediction. Real selection methods land well below 1.

The Uniform Guidelines say validity should rest on a review of job information, normally a job analysis, and describe three evidence types: criterion-related (scores correlate with performance), content (the test samples real job content) and construct (it measures a defined trait that relates to the job). The guidelines also note that content validity isn't suitable for abstract traits such as intelligence or personality (29 CFR 1607.14). That matters for game-based tools, which usually claim to measure traits.

On which methods predict best, a widely cited 2022 re-analysis by Sackett and colleagues lowered earlier estimates, which they argued had been inflated by over-correction for restricted range. Their revised figures: structured interviews r = .42, job knowledge tests .40, empirically keyed biodata .38, work sample tests .33 and cognitive ability .31 (HumRRO summary). Three takeaways follow for managers:

• Methods grounded in the actual job tend to do well. Work samples and job knowledge tests directly resemble the work.
• Even the best methods are moderate predictors. No score should be treated as a verdict on a person.
• The vendor's claim needs to match your job. A validation study on call-center agents says little about an engineering manager role. The Guidelines' own transportability standards exist for this reason.

The point of this research isn't that AI scoring is better or worse than traditional scoring. It's that the format and its link to the job matter more than the label "AI". An AI-scored work sample inherits the work sample's strengths; an AI-scored game with no job analysis behind it inherits nothing.

Adverse Impact and the Four-Fifths Rule

Adverse impact means a selection procedure screens out one group at a noticeably higher rate than another, whether or not anyone intended it. The US Uniform Guidelines give a rule of thumb for spotting it. A selection rate for any race, sex or ethnic group that's less than four-fifths (80%) of the rate for the group with the highest rate is generally regarded as evidence of adverse impact. A rate above that threshold generally isn't. The same paragraph adds that smaller differences can still be adverse impact, especially when they're significant in statistical and practical terms, and that differences resting on small numbers may not be reliable (29 CFR 1607.4(D)).

A hypothetical example shows the arithmetic. Say 200 candidates from Group A take an assessment and 100 pass, a 50% rate. Group B has 100 candidates and 30 pass, a 30% rate. Divide 30% by 50% and you get 0.60, below the 0.80 line, so the result flags for review. If instead 42 of the 100 passed, the ratio would be 0.84 and the flag wouldn't trigger on the rule alone.

Three cautions keep the rule useful rather than dangerous:

• It's a screening heuristic, not a safe harbor. Passing the 80% test doesn't mean a tool is lawful, and failing it doesn't mean it's unlawful. It starts a closer look.
• Run it at every stage. Test the overall pass rate, and also each assessment, cut score and band. An assessment that looks fine in aggregate can concentrate the harm at one cut score.
• Adverse impact triggers a validity question. Under the technical standards, procedures with low correlations but large adverse impact get close review, and users should consider whether other procedures with greater or equal validity are available (29 CFR 1607.14). In plain terms: if two tools predict equally well and one screens out more people from a protected group, use the other.

Where the Legal Exposure Sits

Employers carry the liability for tools they buy. The rules below are summarized at a high level and change often, so check current text with counsel in each place you hire.

US federal anti-discrimination law. The Uniform Guidelines above apply to selection procedures generally, including AI-scored ones. The Americans with Disabilities Act adds a separate issue: the Justice Department's guidance says online games and personality assessments must measure only the job-relevant skills they intend to, not impaired sensory, manual or speaking skills, and that an employer must provide an accessible test or other adjustments when a test would eliminate someone who can do the job. It also says the employer stays responsible when the discriminatory technology comes from another company (ADA.gov AI guidance).

New York City. Local Law 144 covers automated employment decision tools. The tool must be subject to a bias audit within one year before use, information about the audit must be public, and candidates must get notice. The city says notice is due 10 business days before the tool is used, and it began enforcement on July 5, 2023 (NYC DCWP).

Illinois. The Artificial Intelligence Video Interview Act, effective January 1, 2020, requires employers using AI to analyze video interviews to give notice, explain how the AI works and what characteristics it evaluates, get the applicant's consent beforehand, limit who sees the video, and destroy copies within 30 days of a request (SHRM summary). A separate amendment to the Illinois Human Rights Act, effective January 1, 2026, bars using AI in hiring and other employment decisions in ways that discriminate against protected classes, bars using zip code as a proxy for them, and requires notice to employees (Crowell & Moring).

European Union. The Commission lists AI tools for employment and worker management as high-risk, with CV-sorting software as its example. High-risk systems face obligations around risk management, dataset quality, logging, documentation and human oversight, and for employment these apply from 2 December 2027 under the AI Omnibus amendments. Emotion recognition in workplaces has been prohibited since February 2025 (European Commission). Any video assessment that infers emotion from face or voice is therefore a problem for EU hiring.

Other US states have their own or pending AI-in-hiring rules, and some have been delayed or narrowed repeatedly. Treat any compliance checklist, including this one, as a starting point.

Real Examples by Function

The same assessment format works differently depending on the job.

• Customer support. A simulation where the candidate answers three realistic tickets, scored by a rubric for accuracy, tone and policy use. Easy to tie to the job, easy to audit.
• Sales. A role-play recorded on video, scored on content (did they ask discovery questions, handle an objection) rather than on voice or facial signals. Content-based scoring is easier to explain and defend.
• Engineering. A timed coding task evaluated against test cases, with a human reviewing the solution's reasoning. Good for skills, silent on collaboration.
• Finance and operations. A spreadsheet exercise with a known answer key and a few judgment calls. Mostly rules, little AI, which is a feature.
• High-volume hourly roles. Short situational judgment questions and a basic skills check, used to route candidates to an interview, never to reject without review. The skills-based hiring approach fits here, because it focuses on what a person can do rather than on credentials.

How to Implement an AI Assessment, Step by Step

1. Start from the job. Write down the 4 to 6 critical tasks and the skills behind them. If you can't describe the job, you can't validate a test for it.
2. Choose the least AI that works. Prefer work samples and knowledge tests tied to those tasks. Add AI scoring only where it clearly saves time, such as reading open-ended answers.
3. Ask the vendor for evidence. Request the validation study, the job types and populations it covered, the features the model uses, and the most recent bias audit with group-level results. If there's no study for a comparable role, plan a local validation.
4. Decide the role of the score. Use it as one input to a structured decision, not as an automatic gate. This is the practice of human-in-the-loop design, and reviewers need real authority to override the score.
5. Set cut scores cautiously. Higher cut scores usually increase adverse impact. Test the effect of each threshold before you launch.
6. Plan accommodations up front. Tell candidates what the assessment is and how it's scored, and give a clear way to request an alternative format.
7. Run the four-fifths check on real outcomes. Compare pass rates by group for the assessment and for the full funnel, on a schedule, with enough volume to mean something.
8. Handle the legal notices. Confirm notice, consent and audit duties for every location you hire in, and keep records of the audit and the notice.
9. Review and retire. Re-check validity and impact at least annually, and after any change to the job or the model. Drop tools that fail.

For vendor options, see the best pre-employment assessment tools roundup. If you're also deciding how AI screens resumes and interviews, see AI resume screening and AI in interviews.

Common Pitfalls

• Treating "AI-powered" as proof of validity. The label says how the score is made, not whether it predicts anything.
• Using trait games with no job link. If you can't explain why a memory or risk task relates to the work, a regulator or plaintiff will ask the same question.
• Letting the score reject people automatically. Automatic rejection removes the human check that most legal frameworks lean on.
• Testing bias once at launch. Candidate pools, cut scores and models change. A one-time audit goes stale.
• Ignoring accessibility. Timed, visual or motor-heavy tasks can screen out people who could do the job.
• Relying on the vendor. Federal disability guidance is explicit that the employer using the tool owns the outcome.
• No explanation for candidates. If nobody can say why someone scored low, you can't defend it. This is why explainable AI matters in selection, and why bias in AI belongs in every vendor conversation.

Where to Go Next

Pick one role, write its critical tasks, and compare any assessment you're considering against them before you look at a demo. Then ask for the validation study and the adverse impact results. The full section map is in the AI in recruiting hub, and the function-wide view is in AI in HR.