What is Bias Audit?

Why It Matters

A bias audit is the only AI accountability artefact with a prescribed arithmetic. It does not ask whether a tool is fair in principle; it calculates who advanced and who did not, broken out by category, and compares each group to the one with the highest selection rate. That produces a number, which is exactly why it is useful and exactly why it is uncomfortable.

New York City requires one before an automated employment decision tool is used, with the summary published and candidates notified ten business days ahead. It remains the only US law that mandates an audit by name, but it has become the baseline evidence elsewhere: jurisdictions that create discrimination or transparency liability do not order an audit, and an audit is the strongest thing to have when the question is asked.

What It Calculates

Selection or scoring rates for each category, using the race, ethnicity and sex categories that employers already report on the EEO-1 Component 1 form, including intersectional combinations.

An impact ratio per category, derived by comparing each rate to the most-selected group, in line with the EEOC Uniform Guidelines on employee selection procedures.

An independent evaluation, which is the part teams underweight: the audit is performed by someone other than the vendor and the deployer, on data the tool actually processed.

Where It Breaks

The first failure is auditing the wrong population. A tool tested on last year’s applicant pool, or on a sample that never went through the ranking step, produces numbers about a system nobody is running.

The second is cadence. An annual audit describes a system as it was, and a criteria change, a model upgrade, or a new requisition type can alter behaviour in between. The ratio is only meaningful if it is computed from live traffic.

The third is the vendor audit. A supplier can measure its own model against a reference dataset, which says little about how the tool behaves on your roles, your criteria, and your candidate mix. The deployer’s own numbers are the ones a regulator or a plaintiff will ask for.

The fourth is a passing ratio that hides a proxy. A tool can clear the reference point while a feature correlated with a protected characteristic does the deciding, which is why the audit belongs alongside disparate impact analysis rather than replacing it.

How Flytebit Handles It

We treat the audit as a report on a running system rather than a project. Selection rates and impact ratios are computed continuously from the decisions the system actually influenced, with the criteria version and the reviewer attached to each one, so the audited population is the live one. Where a ratio crosses the reference point, the criteria and the feature set are examined rather than the number being explained away. The industry application is on our HR & Workforce Technology page, and the control design is our AI governance and risk work.