Eliminating bias from automated recruiting is easier said than done

When Amazon’s machine-learning specialists reportedly began building an AI-assisted recruiting program in 2014, a source told Reuters that proponents hoped the tech would become a “holy grail” for hiring, efficiently sifting through applications to identify top candidates

By 2015, the specialists reportedly determined the AI had a consistent bias in favor of male candidates. Reuters reported that Amazon tried to retrain the model to be more “neutral,” but ultimately could not guarantee the tech would achieve this and dissolved the team developing the programs by early 2017. Amazon did not comment on the tool’s problems to Reuters, but did say it was “never used by Amazon recruiters to evaluate candidates.”

In an attempt to root out such potential biases before companies add AI-powered hiring technology (and any potential legal risks) to their recruiting workflows, some vendors now use audits to vet AI for potential sources of racial or gender discrimination.

New York City will require vendors to obtain annual third-party “bias audits” and share the results with employers who use the tech, or intend to use the tech, to hire NYC residents beginning January 2023. If employers use biased technology in their recruitment process, a company using the platform could, at some point, potentially be on the hook for violating anti-discrimination laws.

EEOC called: It’s urgent. In the fall of 2021, EEOC Chair Charlotte A. Burrows announced a new initiative to ensure that AI-based hiring doesn’t violate federal civil rights laws the agency enforces. Just this month, EEOC Commissioner Keith Sonderling urged an audience of HR professionals at SHRM’s HR Transform conference to take civil-rights considerations in AI seriously, specifically in regards to functions like video-interview scoring and résumé screening.

“Those computers and those programs need to understand some of the very significant civil-rights applications that my agency has to deal with,” Sonderling said.

Tough tasks. The most common method for assessing adverse impact in employment, when “a decision, practice, or policy has a disproportionately negative effect on a protected group,” is the EEOC’s four-fifths rule, which says that employers may be regarded as discriminating against a demographic if their selection rate is less than 80% the rate of the highest selected group.

Mark MacCarthy, a senior fellow at the Institute for Technology Law and Policy at Georgetown Law, told HR Brew that rooting out bias in AI is often a difficult undertaking.

MacCarthy, who publicly testified in support of New York City’s AI audit bill in the fall of 2020, says AI bias may be hard to solve, in part, because engineers tend to be “flying a little blind” as they develop and fine-tune models.

MacCarthy said that machine learning generally works by feeding a number of variables into a model and then turning the computer loose to interpret the data. The results can be “complex” and based on “enormously complicated interactions between the different factors, interactions that may make no intuitive sense at all to researchers.”

When bias does creep in, MacCarthy said it’s no easy task to diagnose which factors or interactions are to blame for a model’s bias.

“So then you go inside the guts of the model, and you start to fiddle with it. You try to say, ‘Let’s change this, let’s change that. Let’s change this other thing.’ And it's all a matter of experimentation and fiddling to try to find what factor you might be able to manipulate or change that would get rid of the bias,” MacCarthy said.

Given the complexity of rooting out biases in algorithms, many risk managers do not feel up to the task of assessing the risks associated with AI. According to a 2019 Accenture report, only 11% of risk managers describe themselves as “fully capable of assessing the risks associated with organization-wide adoption of AI.”

Audacity to audit. Vendors who do audit their AI for potential bias admit it’s difficult. Kevin Parker, the former CEO of HireVue, characterized it as a “pretty significant undertaking.” Still, he and Frida Polli, CEO of pymetrics, another HR tech firm which has published AI audit results, said they felt the process was helpful for building trust in their company and the industry as a whole.

Polli wants more vendors to engage in public audits.

“I know it’s hard to change habits and embark on something like this, but we can be better, and I think [pursuing audits] would be really helpful to the field at large,” Polli said.

MacCarthy agrees. He said if vendors can demonstrate to possible clients their technology has “a certain level of validity…[and] is reasonably free of bias…that would be a great selling point.”

“You’d want to boast about that as loud as you could,” MacCarthy said.

The trouble is, there isn’t exactly a roadmap for how to do that. In part two of our look at bias in AI, we follow three AI tech vendors into the uncharted territories of AI audits.—SV

Do you work in HR or have information about your HR department we should know? Email [email protected] or DM @SusannaVogel1 on Twitter. For completely confidential conversations, ask Susanna for her number on Signal.