AI Hiring Bias: How Employers Can Use Screening Tools Fairly

If your team uses software to make the first pass at applicants, it probably parses resumes, ranks them against the job, scores assessments and decides who a recruiter sees first. That saves a great deal of time. It also means a flaw in the tool, or in the way it was set up, can turn away qualified people before anyone on your team looks at them.

When that happens, the employer answers for it. The US Equal Employment Opportunity Commission's case over automatic age cut-offs was brought against the employer, and New York City puts the duty to audit a screening tool on employers. This guide explains how bias gets into AI screening, what two well-documented cases teach, which rules now apply in the places that regulate it, and seven practical steps to test your tools and keep the decision fair. You can go straight to the seven practical steps or the vendor questions.

DisclosureJake Jorgovan co-founded Alpha Apex Group, which is mentioned in this article.

What AI hiring bias is

AI hiring bias is when an automated tool in your hiring process selects, ranks or rejects people from one group at a different rate from others, for reasons that have nothing to do with the job. The tool might screen resumes, rank candidates, score a video interview or a game-based assessment, or decide who gets an interview invitation.

It is judged by outcomes. Nobody has to program a preference for the tool to have one, and a tool that was fair on the vendor's test data can still be unfair on your applicants, for your jobs.

Two cases worth knowing

Amazon's experimental resume engine

Reuters reported (opens in a new tab) in October 2018 that Amazon had been building software since 2014 to score applicants from one to five stars. By 2015, according to people familiar with the project, the company found it was not rating candidates for software developer and other technical jobs in a gender-neutral way. The models had learned from resumes submitted over a 10-year period, most of them from men. The system penalized resumes containing the word "women's" and downgraded graduates of two all-women's colleges.

Amazon edited the programs to be neutral to those terms, but the people Reuters spoke to said that was no guarantee the system would not find other ways to sort candidates unfairly. The team was disbanded, and recruiters looked at the tool's recommendations but never relied on them alone, they said. Amazon declined to comment. The lesson for an employer is the edit: removing the obvious words gave no assurance that the tool had stopped sorting candidates unfairly. Its outcomes still needed testing.

iTutorGroup's automatic age cut-off

iTutorGroup agreed in 2023 to pay $365,000 to settle a suit (opens in a new tab) brought by the US Equal Employment Opportunity Commission. According to the EEOC's lawsuit, the company programmed its application software to automatically reject female applicants aged 55 or older and male applicants aged 60 or older, turning away more than 200 qualified US applicants because of their age. The alleged discrimination came from a programmed age rule: a person set it, and the software applied it to everyone. The settlement included training, a new anti-discrimination policy and injunctions against requesting applicants' birth dates.

Where the bias comes from

What the tool learns from

A model learns the patterns in whatever it was trained on, including the unfair ones: your past hiring decisions, someone else's, or general text from the web. Researchers at the University of Washington tested how that plays out in simulated resume screening (opens in a new tab) in 2024. They varied names associated with white and Black men and women across more than 550 real resumes and had three open-source language models rank them against over 500 job listings in nine occupations, more than three million comparisons. In the main tests, which used names whose frequencies in a public reference corpus reflected US racial population proportions, the models significantly favored white-associated names in 85% of tests and Black-associated names in 9%. In every test comparing names perceived as Black male with white male names, they favored the white male names. A supplemental test matching names by frequency in that reference corpus reversed the race preference, so the figures depend on how names were chosen. That sensitivity is itself a warning: small features of a resume changed the outcome.

The authors corrected the paper's gender-only results in August 2026 after a replication found a coding error; the race and combined race-and-gender results stood. The study (opens in a new tab) simulated resume screening with open models and did not test any commercial hiring product. Its finding about combinations still matters for any audit: the lead author, Kyra Wilson, said the harm against Black men "wasn't necessarily visible from just looking at race or gender in isolation."

Image

Zoom

Original plot of rounded results reported by the University of Washington in October 2024 for three language models. These are proportions of tests with statistically significant preferences, not hiring rates, and the categories shown do not sum to 100%.washington.edu (opens in a new tab) · arxiv.org (opens in a new tab)

Proxies and filters that people set

A tool does not need a field for race, sex or age to discriminate on them. A zip code, a college, a graduation year, a gap in employment or a word on a resume can stand in for one. Illinois law has barred employers from using zip codes as a proxy for protected classes since January 2026. Many filters are rules a person chose. Harvard Business School's research on "hidden workers" (opens in a new tab) in 2021 found that more than 90 percent of surveyed employers in the US, UK and Germany that used a recruitment management system relied on it to make a first cut or rank middle-skills and high-skills applicants. Those systems leaned on negative filters, such as excluding candidates without college degrees or ranking down people who had been out of work for six months, and 72 percent of surveyed employers recycled existing descriptions for middle-skills jobs. The cost is the employer's too: firms in the survey that hired from these overlooked groups were 36 percent less likely to face talent and skills shortages.

Those filters are where I would start. Talent is not scarce; it is grown, and hiring criteria should leave room for people who can learn the role. A filter that rejects everyone with a gap, or without the exact title you had last time, can exclude people your company could train for the role.

People who stop questioning the ranking

A ranked list looks objective, and a busy recruiter will work down it. If no one checks who the tool rejected, an unfair pattern can run unnoticed. The public is wary of handing over that judgment. In a December 2022 survey of 11,004 US adults, Pew Research Center (opens in a new tab) found Americans opposed AI making final hiring decisions by 71% to 7%, and 66% said they would not want to apply to an employer that used AI to help decide. The same survey found hope as well: 47% thought AI would do better than humans at evaluating all applicants in the same way, against 15% who thought it would do worse.

Image

Zoom

In Pew’s December 2022 survey, respondents were more optimistic about AI treating applicants consistently than recognizing potential. These are public attitudes, not measured hiring-tool accuracy.pewresearch.org (opens in a new tab)

The rules that apply

There is no single law for AI in hiring. Existing anti-discrimination law applies to an automated decision as it does to a human one, and a few jurisdictions have added specific duties. The table summarizes what the regulators' and legislatures' own pages say. It is a planning summary and does not replace legal advice. Colorado rewrote its law and the EU moved its dates in 2026, and the EEOC plans to rescind its part of the federal Uniform Guidelines, so check the current text with counsel for the places you hire. Each law also defines which tools and decisions it covers; New York City's FAQ, for example, explains which tools qualify and the connection to the city an employer or candidate needs.

The EU date in the table is for the high-risk rules. The AI Act has also prohibited emotion recognition in workplaces (opens in a new tab) since February 2025, so check what any video or voice assessment infers before using it with EU candidates.

Two points from these rules are useful even where none applies. First, New York City's regulator says plainly that the vendor is not responsible for the bias audit: employers are, though a vendor may commission one for its tool. Second, the city's audit must report selection or scoring rates and impact ratios for sex, race and ethnicity, and their combinations. That is a useful starting point. It does not cover every protected characteristic: review age-based filters and disability-related screen-outs and accommodations with counsel and a qualified auditor.

How to reduce AI hiring bias in your process

These steps assume you already use, or are about to buy, a tool that screens, ranks or assesses candidates. Most of them are process and governance work that a talent team can own.

1. 1

   List every automated step and what it decides

   Map each place software touches a candidate: sourcing, resume screening, ranking, assessments, video interviews, chat screening and interview scheduling. For each, write down which decisions it makes on its own (a rejection, a score that hides someone from view) and who on your team owns it. You cannot test what you have not found.

2. 2

   Define the job before the tool scores anyone

   I'd write down what success in the role looks like before anyone tunes a screen. The tool should score candidates against the skills and experience the job really needs. A recycled description or the profile of past hires is a poor yardstick. Replace negative filters (no degree, a gap, a missing job title) with the essential skills you can show predict success, and drop any criterion you cannot connect to the work.

3. 3

   Ask the vendor for evidence before you rely on it

   Ask what data the tool was trained on, which inputs it uses, what it is not designed to do, and whether an independent auditor has tested its outcomes by sex, race and ethnicity, and their combinations. A vendor's audit is a starting point; check whose data it used. In New York City, an employer can rely on an audit pooled from several employers' data only if it supplied its own historical data or is using the tool for the first time. The questions below are a checklist. If a recruitment process outsourcing provider runs your screening, ask it the same questions: outsourcing the process does not outsource the outcome.

4. 4

   Test outcomes on your own applicants

   At each automated step, compare the share of applicants from each group who move forward. Divide each group's rate by the highest group's rate. A ratio below 0.8 (four-fifths) has long been the usual flag for adverse impact under the federal Uniform Guidelines, and it remains a simple first test. Smaller differences can still matter when they are statistically and practically significant, and small numbers can mislead. Check combinations as well as single categories, since that is where the Washington researchers found the harm to Black men. Use demographic data candidates gave you, kept apart from the hiring decision: New York City's audits may not use demographics inferred by software. If a result raises concern, pause automated rejection at that step, work through the criterion with the vendor, review the affected applications and re-test before relying on the screen again.

5. 5

   Keep a person where the tool would reject someone

   Do not let software reject a candidate without a person able to see the reason and overrule it. Record which recommendations recruiters review, their reasons and any overrides. If there are no overrides, look at a sample of those reviews to see whether recruiters are exercising their own judgment. Colorado's law, from January 2027, lets Colorado-resident applicants and employees request human review of an adverse decision a tool materially influenced, to the extent commercially reasonable.

6. 6

   Tell candidates and offer another route

   Say when software assesses candidates and what it looks at, and explain how to ask for an accommodation or an alternative. New York City requires that notice 10 business days before use for its residents, and Illinois requires notice to employees when AI is used in these decisions.

7. 7

   Keep records and re-test after every change

   Keep the data behind each automated decision. California's rules require automated-decision data to be kept for at least four years, and New York City requires a fresh audit within a year of use. Re-test whenever the vendor updates its model, you change the job criteria or your applicant pool changes.

Questions to ask an AI hiring vendor

Common questions

What is an example of AI discrimination in hiring?

One documented example is Amazon's experimental resume engine, which Reuters reported in 2018 penalized resumes containing the word "women's" after learning from a decade of mostly male applicants. Amazon declined to comment. An enforcement example is iTutorGroup, which settled an EEOC suit in 2023 alleging its software automatically rejected older applicants.

Is AI hiring bias illegal?

Anti-discrimination law applies to a hiring decision whether a person or a tool makes it. California's civil rights regulators say so explicitly, Illinois has written it into its human rights law, and the EEOC settled its iTutorGroup case under the federal age discrimination law. Some places add specific duties: bias audits and notices in New York City, notice and a ban on zip-code proxies in Illinois, record-keeping in California, and notice and, where commercially reasonable, human review on request in Colorado from January 2027. Check the rules for each place you hire.

Does removing names or gender from resumes fix it?

Not on its own. Amazon edited its system to be neutral to the obvious terms, and the people Reuters spoke to said that did not guarantee it would not sort candidates unfairly in other ways. Other details on a resume can stand in for a protected group, so the dependable check is to measure outcomes by group.

What is a biased feedback loop in AI hiring?

A tool trained on past hires recommends people like them. If those recommendations are hired and their records are used to retrain the tool, the original pattern can be reinforced in later rounds. Re-testing outcomes after each retraining is how you catch it.

Should a person make the final hiring decision?

Yes. Use software to organize and inform the decision, and keep the rejection and the offer with people who can see the reasons and overrule the tool. Most Americans in Pew's December 2022 survey also opposed AI making the final hiring decision.