California bar exam scandal reveals AI-created questions
- The State Bar of California revealed that AI assistance was used to develop questions for the February 2025 bar exam amid technical problems.
- Criticism grew over the use of AI, with law educators expressing concerns about the implications on competence assessment in licensing.
- The situation has prompted discussions on the validity of the exam and possible adjustments to the test scores for takers.
In California, the bar exam in February 2025 has faced significant criticism after the State Bar revealed that a portion of the multiple-choice questions was developed with the assistance of artificial intelligence. The announcement followed numerous complaints about technical problems during the exam, including applicants being unable to complete the exam due to online platform failures. The State Bar disclosed that of the 171 scored questions, 23 were developed by ACS Ventures using AI, while the remaining questions were created by Kaplan Exam Services and drawn from a first-year law exam. Despite the criticisms, including concerns about the quality and appropriateness of the AI-generated questions, the State Bar defended its process, highlighting that all questions were reviewed by content validation panels and subject matter experts prior to the exam. The use of AI in this capacity has raised alarm among legal educators, who are questioning the competence standards for a licensing examination designed to evaluate applicants’ legal knowledge and skill. Leaders in the legal education field, such as Mary Basick from the University of California, Irvine, and Katie Moran from the University of San Francisco, expressed their disbelief and skepticism about the decision to incorporate AI in crafting exam questions, arguing that it undermines the seriousness of a bar exam that determines an individual's ability to practice law. Additionally, ongoing issues with testing platforms have caused serious disruptions, leading to discussions of potential adjustment of the test scores given to those who took the exam, as well as calls for accountability regarding the effectiveness and reliability of the bar exam format being implemented by the State Bar.