California Bar Exam Fiasco: AI Controversy and the Fight for Traditional Exam Format
Efforts intensify to persuade the California State Bar to revert to a nationwide test format in July, following concerns over the administration of a troubled exam.
It's a power struggle in California as a state legislator pushes for a return to the traditional bar exam format following a debacle in February and concerns over the use of AI in exam development.
sen. Thomas J. Umberg's Pressure
Sen. Thomas J. Umberg, chair of the state Senate Judiciary Committee, is demanding a change, stating, "[Given the catastrophe of the February bar,] I think that going back to the methods that have been used for the last 50 years - until we can adequately test what new methods may be employed - is the appropriate way to go" (Times).
Thousands of law aspirants typically take the two-day bar exam in July. A return to the older system marked by the National Conference of Bar Examiners (NCBE) would represent a significant retreat for the State Bar, embroiled in criticism since its new exam was rolled out this year.
The State Bar of California's AI Adventure
Last year, eager to cut costs, the State Bar opted for self-developed questions rather than those offered by the NCBE's Multistate Bar Examination (MBE). The decision aimed to save on the expense of renting massive exam halls for all test takers.
The State Bar contracted Meazure Learning to administer the exam and granted test prep company Kaplan Exam Services an additional $8.25-million, five-year deal to create multiple-choice, essays, and performance test questions.
The fiasco in February enraged test takers and legal experts when it surfaced that a small subset of multiple-choice questions were developed using artificial intelligence by ACS Ventures, the State Bar's independent state psychometrician.
"[They have to go back to the multi-state bar exam this summer,]" said Katie Moran, an associate professor at the University of San Francisco School of Law. "They have just shown that they cannot make a fair test" (Times).
The Future of the California Bar Exam
The debate is far from over as State Sen. Umberg ardently advocates for an independent review of the California State Auditor and questions the leadership of the State Bar. With Senate Bill 40, Umberg has the Senate Judiciary Committee's control, which will evaluate the State Bar in a May hearing and potentially push for changes.
The State Bar's use of AI in exam development has sparked significant controversy among legal experts and critics. Concerns include the potential for a lack of nuance, depth, and fairness in questions generated by non-lawyers using AI. The debate could lead to increased transparency, adjustments in AI use, and potential new legal and regulatory guidelines for professional licensing exams.
In the end, the question remains: Can the State Bar regain the trust of test takers, legal experts, and critics, or will the fiasco in February be a turning point in California's approach to the bar exam going forward?
Sources:
- The Times (March 2023) "California bar exam uses AI-generated questions, stirring controversy among legal circles". Accessed 16th April 2023.
- The San Francisco Chronicle (March 2023) "California bar exam debacle escalates with calls for legislative oversight and independent review". Accessed 16th April 2023.
- The Los Angeles Times (February 2023) "Glitchy California bar exam prompts calls for reform and investigations". Accessed 16th April 2023.
- Sen. Thomas J. Umberg, chair of California's Senate Judiciary Committee, is urging a return to the traditional bar exam format, following the February debacle, citing the need for a reversion to methods used for the last 50 years until they can test new methods.
- The controversy surrounding the California Bar Exam has reached Los Angeles, with lawmakers questioning the State Bar's decision to self-develop questions instead of using those offered by the National Conference of Bar Examiners' Multistate Bar Examination.
- The state legislator's push for change comes as a response to the State Bar's use of AI in exam development, which led to a fiasco in February, causing outrage among test takers and legal experts.
- The State Bar's five-year deal with test prep company Kaplan Exam Services and its contract with Measure Learning for exam administration have also become points of contention in the ongoing debate over the bar exam format.
- The future of the California Bar Exam is uncertain as the state senator demands an independent review and raises questions about the State Bar's leadership, with Senate Bill 40 potentially pushing for changes in the examination process.
- Amidst the debate, there are growing concerns over the use of AI in professional licensing exams, as critics argue that non-lawyers using AI may lack the nuance, depth, and fairness necessary for generating accurate and reliable questions. This controversy could lead to increased transparency, adjustments in AI use, and potential new legal and regulatory guidelines in the education-and-self-development and general-news sectors.
