California regulators are taking steps to limit the use of artificial intelligence (AI) in the hiring process. The proposed regulations, drafted by the California Civil Rights Department, aim to prevent discrimination by warning employers against using AI systems to measure tone of voice, facial expressions, and reaction times. Employers are cautioned that using AI to prioritize or exclude applicants based on protected characteristics such as pregnancy, national origin, religion, or criminal history would be considered discriminatory.
The draft rules cover a wide range of “automated decision systems,” including AI, quizzes, games, resume screening, and advertising placement. They state that analyzing physical characteristics or reaction times using these systems may constitute illegal discrimination. Furthermore, if these systems have an “adverse impact” on candidates based on protected characteristics, they should not be used at all. The regulations also require companies selling predictive services to employers to maintain records for four years to address discrimination claims.
The need for these regulations arises from the potential for AI to obscure responsibility and make it difficult to identify who is accountable for discriminatory decision-making. Ken Wang, a policy associate with the California Employment Lawyers Association, emphasizes the importance of holding employers responsible for the actions of third-party service providers, who are considered agents of the employer under the proposed rules.
The California Civil Rights Department began exploring the impact of algorithms and automated decision systems on job opportunities and workplace discrimination in April 2021. Advocates, researchers, and journalists have presented evidence of AI models automating discrimination, including in hiring processes. The American Civil Liberties Union recently filed a complaint alleging racial and disability discrimination by Aon’s resume screening software, despite the company’s claim of being “bias-free.” Additionally, evaluations of OpenAI’s GPT-3.5 technology revealed racial, gender, and religious biases in automatically sifting through job applicants’ resumes.
The proposed regulations in California align with other initiatives aimed at protecting workers from harmful AI practices. New York City lawmakers passed a law in 2021 to combat algorithmic discrimination in hiring, although recent assessments suggest its effectiveness has been limited. The Equal Employment Opportunity Commission and the U.S. Justice Department have clarified that employers must comply with the Americans with Disabilities Act when using automation in the hiring process. The California Privacy Protection Agency is also considering draft rules to define the information employers can collect on contractors, job applicants, and workers, allowing individuals to opt-out or request human review.
Assembly Bill 2930, pending legislation, would empower the California Civil Rights Department to demand impact assessments from businesses and state agencies using AI to prevent automated discrimination. Labor representatives have also engaged in discussions with California officials to ensure workers have a say in assessing the effectiveness and potential harms of AI.