New York City Takes Aim at Artificial Intelligence in Employment

The rise of artificial intelligence and other computer-based, data-driven decisions may seem attractive to employers looking to parse a deep pool of qualified candidates. But concerns that biases may affect these automated decisions has led New York City to impose requirements on their use. Last week the New York City Council passed a bill requiring employers to take certain steps before implementing an “automated employment decision tool.” The law becomes effective on January 1, 2023.

What Is an Automated Employment Decision Tool?

An automated employment decision tool is some form of “computational process” that “issues a simplified output,” such as a score, classification, or recommendation, which is then used to substantially assist or replace discretionary decision-making processes for hiring or promoting employees within New York City. The law does not apply to other employment decisions, such as retention or termination. Examples of a computational process that qualifies under the law include “machine learning, statistical modeling, data analytics, or artificial intelligence.” The law specifically excludes junk email filters, firewalls, antivirus software, calculators, spreadsheets, databases, and data sets.

What Steps Must be Taken if an Employer Is Using an Automated Employment Decision Tool? 

The new law will require employers using automated employment decision tools to take the following actions:

Step 1: Conduct a Bias Audit
The new law makes it unlawful for an employer to use an automated employment decision tool in hiring or promoting employees without having conducted a “bias audit” for that tool within the past year. The new law defines a bias audit as an “impartial evaluation by an independent auditor” which must test the “automated employment decision tool to assess the tool’s disparate impact on persons” based on race, ethnicity, or sex. It is not yet clear whether AI software vendors qualify as “independent” under the new law, such that they can conduct the bias audit themselves.

Step 2: Publicize the Bias Audit
Before using an automated employment decision tool in hiring or promoting employees, the employer must first make “publicly available on the website of the employer or employment agency” a “summary of the results of the most recent bias audit of such tool.”

Step 3: Give Notice to Candidates and Employees
An employer that uses an automated employment decision tool to screen employees or candidates who live in New York City must notify those employees or candidates (1) that an automated employment decision tool will be used and (2) of the “job qualifications and characteristics” that the tool will consider. Both of these notices must be made at least 10 business days before use of the tool. Notably, the law also requires employers to notify applicants that they may “request an alternative selection process or accommodation.”

Additionally, to the extent not disclosed on the employer’s website, “information about the type of data collected” for the tool, the “source of the data[,] and the employer or employment agency’s data retention policy” must be furnished within 30 days of a written request from an employee or candidate.

What Are the Penalties for Non-Compliance? 

Penalties can be substantial since each day on which an automated employment decision tool is used is deemed a separate violation. The first violation and any additional violations occurring on the same day carry a civil penalty of up to $500 each. Subsequent violations carry civil penalties between $500 and $1,500 per day. Likewise, each failure to provide any of the notices outlined above constitutes a separate violation.

The law may be enforced by the New York City Corporation Counsel (or those designated by the Corporation Counsel) through “any action or proceeding that may be appropriate or necessary for correction of any violation.” The law does not limit the rights of an employee or candidate to bring a private civil action.

Bottom Line 

Effective January 1, 2023, before using any computational process to help make hiring or promotion decisions, New York City employers must ensure they have taken steps to protect themselves from civil penalties. Specifically, before using an automated employment decision tool, the employer must (1) conduct a bias audit for the tool, (2) publicize the results of that audit on its website, and (3) provide certain notices to employees and job applicants. Many open questions remain regarding this new law, including the degree to which employers will be able to rely on independent audits obtained by the AI product vendor, and we expect further clarification to be provided in the one-year period prior to its effective date. In the meantime, employers utilizing AI in their hiring practices should scrutinize the programs they are using to determine what improvements or disclosures will need to be made.

If you need assistance navigating the complex employment laws governing the workplace, please contact Lowenstein Sandler’s Employment Counseling & Litigation practice group.