Examining the Bar Exam: An Empirical Analysis of Racial Bias in the Uniform Bar Examination

Scott Devito, Kelsey Hample & Erin Lain*

Abstract

The legal profession is among the least diverse in the United States. Given continuing issues of systemic racism, the central position that the justice system occupies in society, and the vital role that lawyers play in that system, it is incumbent upon legal professionals to identify and remedy the causes of this lack of diversity. This Article seeks to understand how the bar examination—the final hurdle to entering the profession—contributes to this dearth of diversity. Using publicly available data, we analyze whether the ethnic makeup of a law school’s entering class correlates to the school’s first-time bar passage rates on the Uniform Bar Examination (UBE). We find that higher proportions of Black and Hispanic students in a law school’s entering class are associated with lower first-time bar passage rates for that school in its reported UBE jurisdictions three years later. This effect persists after controlling for other potentially causal factors like undergraduate grade-point average (UGPA), law school admission test (LSAT) score, geographic region, or law school tier. Moreover, the results are statistically robust at a p-value of 0.01 (indicating just a 1% chance that the results are due to random variation in the data). Because these are school-level results, they may not fully account for relevant factors identifiable only in student-level data. As a result, we argue that follow-up study using data relating to individual students is necessary to fully understand why the UBE produces racially and ethnically disparate results.

Table of Contents

I am an invisible man. No, I am not a spook like those who haunted Edgar Allan Poe; nor am I one of your Hollywood-movie ectoplasms. I am a man of substance, of flesh and bone, fiber and liquids—and I might even be said to possess a mind. I am invisible, understand, simply because people refuse to see me. Like the bodiless heads you see sometimes in circus sideshows, it is as though I have been surrounded by mirrors of hard, distorting glass. When they approach me they see only my surroundings, themselves, or figments of their imagination—indeed, everything and anything except me.

Ralph Ellison, Invisible Man¹Ralph Ellison, Invisible Man 1 (1947).

Introduction: The Unexamined Exam

This Article is written with two purposes. First, we seek to sound the alarm that the bar exam is racially and ethnically biased.²All reliable evidence demonstrates that White examinees outperform examinees from communities of color with similar academic indicators. See discussion infra Part III (discussing empirical studies of the interaction between race/ethnicity and the bar examination). Additionally, this Article capitalizes all terms that refer to socially-constructed race and ethnic categories. For many, such an alarm will seem absurd. They will argue that the bar examination is difficult, but it is also fair; passing is a function of ability, work ethic, writing skill, and knowledge of the law—not race or ethnicity. Unfortunately, such a belief is unsupported by the evidence.³See id.

Second, we write this Article as a call to action. Philosophers have, for millennia, warned us of the dangers of accepting beliefs without adequately testing them.⁴Over 2,000 years ago, Socrates argued that the good life was one in which we investigate and challenge our beliefs to determine whether what we think we know, we know. See Plato, Apology, in 1 Plato in Twelve Volumes 38a (Harold North Fowler trans., Harvard University Press 1966) (contending that death is a better outcome than living an unexamined life). In the twentieth century, Karl Popper proposed the theory of falsificationism which held that theories that could not, in practice, be shown to be false were pseudo-science, not scientific theories. Karl R. Popper, The Logic of Scientific Discovery 40–41 (1959) (proposing falsifiability as a criterion of the demarcation of science from non-science). Yet that is precisely what the legal community has done with regard to the “objectivity” of the bar examination. The profession simply proceeds as if race and ethnicity are irrelevant to the probability that a bar-taker will pass the bar examination,⁵For example, despite testimony and evidence adduced to the Council of the ABA Section of Legal Education and Admissions to the Bar that the change it was considering for ABA-accredited law school bar passage standards was harmful to minorities, the Section simply, and without publicly releasing any documentary support, changed the standard anyway. See, e.g., Society of American Law Teachers, Letter to the American Bar Association House of Delegates (Jan. 21, 2019), https://www.lwionline.org/sites/default/files/SALT%20Jan%202019.pdf [https://perma.cc/U9BF-TYL2] (“Adopting the proposed standard will have substantial negative impact on HBCU and other law schools with significant enrollment of people of color, including the law schools in Puerto Rico.”). or whitewashes the evidence of disparate outcomes and deems those differences inconsequential.⁶See discussion infra Part III (discussing empirical studies of the bar exam). Given empirical evidence to the contrary, we contend that the legal community must demand in-depth analysis of the bar examination’s questions, administration, and grading to determine why race and ethnicity appear to impact bar passage rates.

This Article engages in statistical analysis of first-time bar passage rates, at the school-jurisdiction level,⁷A “school-jurisdiction pass rate” is the pass rate of all students who graduated from a specific law school (e.g., the University of Connecticut School of Law) and passed the bar examination in a given jurisdiction (e.g., New York). for schools in Uniform Bar Examination (UBE) jurisdictions.⁸See discussion infra Part V. The analysis reveals a highly significant, negative correlation between a school’s proportion of Black or Hispanic students and the first-time pass rate for that school-jurisdiction.⁹See discussion infra Section V.B (describing the results of our analysis of the UBE). In essence, as a school’s proportion of Black or Hispanic students increases, the school’s first-time bar passage rates decline three years later (when the enrollees are expected to graduate). This result is statistically significant to a p-value of 0.01.¹⁰See id. (noting that the correlations found in our study are statistically significant at the ninety-nine percent confidence interval). All measures of statistical significance discussed in this Article relate to the p-value of a statistical hypothesis. We will consider a result to be statistically significant if its corresponding p-value is less than or equal to 0.01. This means that there is no more than a 1 in 100 chance that our result is due to random variation. David Hensher, John M. Rose & William H. Greene, Applied Choice Analysis: A Primer 46–47 (2005) (explaining p-values and statistical significance). Normally, in social science, a p-value of 0.05 (the result has a 1 in 20 chance of being due to random variation) is used as a measure of statistical significance. See, e.g., id.; Scott E. Maxwell & Harold D. Delany, Designing Experiments and Analyzing Data: A Model Comparison Perspective 47 (Wadsworth Publishing Company, 2d ed. 2004). That our results are statistically significant at a more stringent p-value of 0.01 (the result has a 1 in 100 chance of being due to random variation) demonstrates the robustness of those results. Such a result should greatly concern the legal profession as it provides clear evidence of disproportionate bar examination outcomes based on race and ethnicity and suggests that such disproportionality may result from the exam itself. It should also prompt the legal community to further study the bar exam, using student-specific data, to better understand why bar passage rates decline as a school’s proportion of Black or Hispanic students increases.

This Article proceeds in multiple stages. Part I provides a short history of the bar exam and its racist roots. Part II then discusses the current, long-standing racial and ethnic imbalance in the legal field and two possible explanations for the profession’s ignorance of this imbalance. We then turn, in Part III, to review and analyze previous empirical studies of race/ethnicity and the bar examination. These studies can be divided into three categories: (1) those that cannot reach a conclusion due to insufficient information, (2) those that whitewash their empirical findings of disparate outcomes to ultimately support the conclusion that the bar is neither racially nor ethnically biased, and (3) surveys conducted by government actors showing racially and ethnically disparate results. Part IV then discusses some factors that may cause these disparate outcomes. This Article’s original study and results are presented in Part V. In this study, we empirically examine the relationship between a school’s bar passage rate and the percentage of its class that is comprised of different racial and ethnic groups (American Indian/Alaska Native, Asian, Black, Hispanic, Native Hawaiian/Pacific Islander, and two or more races), the school’s median LSAT, its geographic location, and the school tier. Our study finds a highly significant correlation between the percentage of a law school’s student body that is comprised of Black or Hispanic students and the school’s bar passage rate, under the UBE, at the time those students are expected to graduate. Finally, Part VI argues that student-level study is required to fully understand the results of this research and to make policy decisions aimed at improving bar outcomes for students from communities of color.

I. The Bar Exam: A History of Racial Gatekeeping

Admission to the bar has not always been as uniform or academic as the procedures utilized today. For much of the nineteenth century, the bar consisted of an oral examination administered by an official acting on behalf of a particular jurisdiction constrained by few formal guidelines.¹¹Margo Melli, Passing the Bar: A Brief History of Bar Exam Standards, 21 Univ. Wis. L. Sch. Gargoyle 3, 3 (1990) (discussing the early history of admissions to the bar in the United States). Toward the latter half of the nineteenth century, these relaxed standards were challenged. Beginning in 1880, states created centralized bar examiners that gradually introduced state-wide written examinations for bar admission.¹²See id. at 3–4 (discussing admission to the bar during the nineteenth Century).

The process of formalization and standardization continued into the first half of the twentieth century. By 1931, all states—except for Indiana—had formalized and centralized boards of bar examiners.¹³Id. at 4. That same year, the American Association of Law Schools assembled a committee to explore creating a national organization for bar examiners—and the National Conference of Bar Examiners (NCBE) was subsequently formed.¹⁴Id. (discussing the development of the NCBE). Initially, the NCBE sought to help state bars improve their approach to the bar exam.¹⁵Id. Before the NCBE’s formation, states focused exam questions on black-letter law, including asking applicants to, for example, “[l]ist the kinds” of evidence or to “[d]efine the term substantial compliance.”¹⁶Id. The NCBE worked with states to transition from this form of question to one based on a hypothetical fact pattern—a type of questioning quite familiar to current lawyers and bar examinees.¹⁷Id.

Scholars have argued that this change in the bar examination served to restrict immigrants and non-White applicants from becoming lawyers. For example, employment discrimination scholar Subotnik draws the connection between testing and anti-immigrant status, explaining that the profession expressed concern over the quality of immigrants and applicants of color.¹⁸E.g., Dan Subotnik, Does Testing = Race Discrimination?: Ricci, the Bar Exam, the LSAT, and the Challenge to Learning, 8 U. Mass. L. Rev. 332, 365 (2013). At the end of the nineteenth century and into the beginning of the twentieth, critical race theorist Roithmayr argues, leaders in the legal profession were troubled by the possibility of immigrants and non-Whites entering the field.¹⁹See Daria Roithmayr, Deconstructing the Distinction Between Bias and Merit, 10 La Raza L.J. 363, 392–93 (1998) (discussing the history of law school admission standards). At the same time, the American Bar Association (ABA) became instrumental in the push for limiting which applicants to the bar would actually be accepted, with anti-immigrant and racist sentiments shaping the measures proposed and supported by the ABA.²⁰See id. at 393–94 (discussing reports of Alfred Z. Reed). Law and society scholar Friedman, in describing the origin of the ABA and its motivations for developing formal bar admission requirements, emphasizes the pervasiveness of exclusionary beliefs among ABA members and leaders, pointing to the role that such beliefs played in discussions surrounding the prospect of a formal bar.²¹Lawrence M. Friedman, A History of American Law 648–54 (2d ed. 1985).

Throughout this period, the ABA was an all-White organization that actively excluded persons who were not “White” from membership.²²See Roithmayr, supra note 19, at 398 (discussing the ABA’s accidental admission, and subsequent revocation, of membership to three Black attorneys). For example, in 1912, the ABA mistakenly admitted three Black lawyers to the association and justified revoking their admission by explaining that they wanted to keep “pure the Anglo-Saxon race.”²³Id. The ABA was not alone in its efforts to marginalize lawyers from communities of color and maintain an all-White profession. For example, in 1925, Texas passed a law limiting law school admission to only White students.²⁴See Sweatt v. Painter, 339 U.S. 629, 631 n.1 (1950) (“It appears that the University has been restricted to white students, in accordance with the State law.”) (citing Tex. Const. art. VII, §§ 7, 14; 1925 Tex. Rev. Civ. Stat. 2643b (repealed 1971); 1925 Tex. Rev. Civ. Stat. 2719 (repealed 1969); 1925 Tex. Rev. Civ. Stat. 2900 (Supp.)). The first Black student to be admitted to the University of Texas Law School, Heman Marion Sweatt, was permitted enrollment only after the U.S. Supreme Court found that the school’s refusal to admit Mr. Sweatt violated his “constitutional right: legal education equivalent to that offered by the State [of Texas] to students of other races.”²⁵Id. at 635–36 (finding that the University of Texas Law School’s denial of admission to Mr. Sweatt violated his right to equal protection under the Fourteenth Amendment). Additionally, “[a]s late as 1938, the University of Missouri Law School continued to formally exclude Black applicants on the grounds that it was contrary to the constitution, laws and public policy of the State to admit a negro as a student in the University of Missouri.”²⁶Roithmayr, supra note 19, at 399 (quoting Missouri ex rel. Gaines v. Canada, 305 U.S. 337, 343 (1938)) (internal quotation marks omitted). Thus, throughout the first half of the twentieth century, there was widespread sentiment that the profession should be limited to Whites. Raising admission standards, including through use of the bar exam, was a mechanism for achieving this goal.

The modern bar exam came about in the 1970s.²⁷Melli, supra note 11, at 4 (discussing the history of the bar exam). The NCBE, through a grant from the ABA, developed the six-hour, multiple-choice exam that would be known as the Multistate Bar Examination (MBE).²⁸Id. This exam could be administered by all examining jurisdictions, could be machine-graded, and provided a uniform test, while leaving control of passing scores up to individual jurisdictions.²⁹Id. The idea of a uniform bar exam had been discussed since the 1940s, and a multistate bar exam was ultimately created because most professions already had nationalized standards that applicants were required to meet.³⁰Id. Like other bar exam unification initiatives, the MBE was an attempt to raise standards for entering the practice of law.³¹Id.

States adopted the MBE into their bar exam practices because it relieved some of the burden of creating and grading their own exams.³²See id. Further pressure to adopt the bar exam arose as, starting in the 1970s, the number of bar examinees increased significantly.³³See id. In 1963, 20,776 students entered law school.³⁴See Enrollment and Degrees Awarded 1963–2012 Academic Years, Am. Bar Ass’n, https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/statistics/enrollment_degrees_awarded.authcheckdam.pdf [perma.cc/BV8Y-LUVH]. Just ten years later, in 1973, that number reached over 37,000.³⁵See id. Law school enrollment continued to increase each decade, peaking at enrollment of 52,400 students starting law school in 2010.³⁶See id. This increase has played an essential role in state adoption of the MBE as a means of testing applicants for admission to the bar.³⁷Melli, supra note 11, at 4 (discussing the history of the bar exam).

The modern history of the bar examination has been a steady march toward a national test; the MBE was introduced in 1972, followed by the Multistate Professional Responsibility Examination (MPRE) in 1980, the Multistate Essay Examination (MEE) in 1988, and the Multistate Performance Test (MPT) in 1997.³⁸Am. Bar Ass’n, Resolution 109 and Report to The House of Delegates 2 (2016) https://www.ncbex.org/pdfviewer/?file=%2Fdmsdocument%2F193 [perma.cc/5RAC-F5N6] (providing a short history of the modern bar exam). Then the NCBE first offered the UBE, which “is composed of the MEE, the MPT, and the MBE,” in February 2011.³⁹See id. at 3 (describing the UBE). One of the stated benefits of the UBE is that its adoption will “help ensure the consistency and quality of the bar exam.”⁴⁰Id. at 6 (discussing benefits of UBE adoption). The UBE is currently offered in thirty-nine jurisdictions.⁴¹See Jurisdictions That Have Adopted the UBE, Nat’l Conf. Bar Exam’rs, https://www.ncbex.org/exams/ube/ [perma.cc/GD4S-FPB3].

While this shift to exam uniformity (and portability of score in the case of the UBE⁴²See id. (“[The UBE] results in a portable score that can be used to apply for admission in other UBE jurisdictions.”).) is laudable, uniformity alone does not guarantee neutrality as to race or ethnicity.⁴³For example, algorithmic systems designed to assess risk “bring uniformity, transparency, and accountability to the task,” yet nonetheless are subject to bias. Sandra G. Mayson, Bias in, Bias out, 128 Yale L.J. 2218, 2248, 2280 (2019) (“An algorithm can be designed to achieve any one of the [discussed] metrics of output equality, but not all of them together.”). Similarly, as we saw in the case of the different sentencing guidelines for crack cocaine and powder cocaine, even though Congress sought to create a uniform system, and therefore limit bias, the very law itself created racially disparate and unfair outcomes. See Dorsey v. United States, 567 U.S. 260, 264 (2012) (noting that the objectives of the Federal Sentencing Guidelines include “uniformity and proportionality in sentencing”); cf. id. at 268 (“[T]he Commission and others in the law enforcement community strongly criticized Congress’ decision to set the crack-to-powder mandatory minimum ration at 100-to-1 . . . . because the public had come to understand sentences embodying the 100-to-1 ration as reflecting unjustified race-based differences.”). In the 1970s (after the MBE’s initial introduction), a string of lawsuits across the country alleged discrimination in the bar exam.⁴⁴See, e.g., Tyler v. Vickery, 517 F.2d 1089 (5th Cir. 1975); Parrish v. Bd. of Comm’rs of Ala. State Bar, 505 F.2d 12 (5th Cir. 1974), opinion withdrawn, 509 F.2d 540 (5th Cir. 1975), and on reh’g sub nom. Parrish v. Bd. of Comm’rs of Alabama State Bar, 524 F.2d 98 (5th Cir. 1975); Richardson v. McFadden, 540 F.2d 744 (4th Cir. 1976), on reh’g, 563 F.2d 1130 (4th Cir. 1977); Pettit v. Gingerich, 427 F. Supp. 282 (D. Md. 1977), aff’d 582 F.2d 869 (4th Cir. 1978); Delgado v. McTighe, 442 F. Supp. 725 (E.D. Pa. 1977). Most notably, in Tyler v. Vickery—a class action suit filed on behalf of Black bar examinees who failed the Georgia exam—the plaintiffs alleged outright discrimination, disparate impact, and lack of due process in Georgia’s practices.⁴⁵The appellants argued
1) that the examiners have used the bar examination to purposefully discriminate against black applicants on the basis of race; 2) that the bar examination inherently violates the fourteenth amendment’s equal protection clause because of the highly disparate passing rates of black and white applicants; and 3) that the examination violates due process because there is no procedure for review of a failing grade.
Tyler, 517 F.2d at 1093. The court rejected all three of these claims. See id. at 1093–1105 (discussing appellants’ arguments). The court rejected all three claims, finding that the bar examinees failed to establish intentional discrimination by the Georgia Bar Examiners.⁴⁶Id. at 1093 ( “Appellants’ . . . contention is that the bar examiners utilize the [bar] examination as a device to purposefully discriminate against prospective black attorneys on the basis of race.”). Two years later, Black and Puerto Rican examinees who failed the bar in Pennsylvania also sued based on the Fourteenth Amendment’s Due Process Clause and Equal Protection Clause.⁴⁷Delgado, 442 F. Supp. at 726 (outlining the cause of action). They alleged that changes in the passing score requirements were arbitrary and intentionally discriminated against Black and Puerto Rican examinees.⁴⁸Id. These examinees achieved scores on the bar exam that would have passed in years prior, but due to Pennsylvania’s increased score requirements for passing, the petitioners failed.⁴⁹Id. Additionally, as noted in Section II.B infra, in the 1980s and 90s, state bars and judiciaries were sufficiently concerned about racial and ethnic bias in the judicial system (including in the bar exam) that they formed committees to study the issue; those committees concluded that the bar produced disparate outcomes based on the examinees’ race and ethnicity.⁵⁰See infra text accompanying notes 76–90 (discussing formation of committees to study disparate outcomes on the bar exam).

II. Invisible People

In recent years, the oppression faced by Black, Indigenous, and people of color (BIPOC) in the United States has been pushed to the forefront of American life.⁵¹See, e.g., Larry Buchanan, Quoctrung Bui & Jugal K. Patel, Black Lives Matter May Be the Largest Movement in U.S. History, N.Y. Times (July 3, 2020), https://www.nytimes.com/interactive/2020/07/03/us/george-floyd-protests-crowd-size.html [https://perma.cc/44E9-YDPW] (discussing Black Lives Matter protests in the United States); Code Switch, A Decade of Watching Black People Die, NPR (May 31, 2020), https://www.npr.org/2020/05/29/865261916/a-decade-of-watching-black-people-die [https://perma.cc/6HEM-UDDW] (discussing the current Black Lives Matter protest in the context of Eric Garner’s July 2014 death and listing some of the Black people killed by the police since Eric Garner’s death). As this Article was written, people across the world protested the deaths of George Floyd, Breonna Taylor, Philando Castile, Eric Garner, Michael Brown, Tamir Rice, and many other victims⁵²See, e.g., Code Switch, supra note 51 (discussing the then current Black Lives Matter protests). of racist police brutality⁵³Police brutality is often targeted against a person’s race. See, e.g., Alexa P. Freeman, Unscheduled Departures: The Circumvention of Just Sentencing for Police Brutality, 47 Hastings L.J. 677, 694–98 (1996) (discussing that racist beliefs underlay many acts of police violence against persons of color). At the same time, it is important to note that police brutality can, and is, targeted against other aspects of a person’s identity including “sexual orientation, race, gender or gender identity, age or economic status.” Amnesty Int’l, USA: Stonewalled: Police Abuse and Misconduct Against Lesbian, Gay, Bisexual and Transgender People in the U.S. 164 (2005), https://www.amnesty.org/en/wp-content/uploads/2021/08/amr511222005en.pdf [https://perma.cc/4EDQ-MN8M]. and racism endemic to the U.S. criminal justice system.⁵⁴Rasheena Latham, Who Really Murdered Trayvon? A Critical Analysis of the Relationship Between Institutional Racism in the Criminal Justice System and Trayvon Martin’s Death, 9 S.J. Pol’y & Just. 80, 81–82 (2014). Institutional racism is particularly pernicious as “[i]t is discrimination permeated in our society from healthcare, education, law enforcement and virtually every institution or organization in America.” Id. at 82–83. “Institutional racism occurs where an institution adopts a policy, practice, or procedure that, although it appears neutral, has a disproportionately negative impact on members of a racial or ethnic minority group.” Vernellia R. Randall, The Misuses of the LSAT: Discriminating Against Blacks and Other Minorities in Law School Admissions, 80 St. John’s L. Rev. 107, 107 (2006). In a sense, what was always visible but unseen, and often ignored by White Americans, has now, to some extent, become visible to them. Yet this transition from invisible to visible appears to have stalled in the legal field, which remains one of the least diverse professions in the United States.⁵⁵Deborah L. Rhode, Law Is the Least Diverse Profession in the Nation. And Lawyers Aren’t Doing Enough to Change That, Wash. Post (May 27, 2015), https://www.washingtonpost.com/posteverything/wp/2015/05/27/law-is-the-least-diverse-profession-in-the-nation-and-lawyers-arent-doing-enough-to-change-that/ [https://perma.cc/3PJM-8WXZ] (discussing the lack of diversity in the legal profession); see also A.B.A. National Lawyer Population Survey: 10-Year Trend in Lawyer Demographics, Am. Bar Ass’n (2018), https://www.americanbar.org/content/dam/aba/administrative/market_research/National_Lawyer_Population_Demographics_2008-2018.pdf [https://perma.cc/R6AS-WJZ9] (listing the percentage of active attorneys by race and ethnicity). Moreover, as Table 1 demonstrates, the percentage of active lawyers who are Asian, Black, or Hispanic continues to trail core U.S. demographic categories: in the percentage of persons admitted to law school, in the total U.S. population, and in the percent of the U.S. population aged 18–29 with a bachelor’s degree. This places people of color at risk, as they are forced to rely on attorneys who are not from their communities and are thereby prone to implicit bias against people of color.⁵⁶There is clear evidence that lawyers, like all Americans, demonstrate implicit bias. See, e.g., Justin D. Levinson, Mark W. Bennett & Koichi Hioki, Judging Implicit Bias: A National Empirical Study of Judicial Stereotypes, 69 Fla. L. Rev. 63, 104–05 (2017) (discussing results of study of implicit bias among judges relating to Asian Americans and Jewish Americans); Brian Libgober, Getting a Lawyer While Black: A Field Experiment, 24 Lewis & Clark L. Rev. 53, 54–55 (2020) (discussing his studies showing lower callback rates when the prospective client has a Black-sounding name); Praatika Prasad, Implicit Racial Biases in Prosecutorial Summations: Proposing an Integrated Response, 86 Fordham L. Rev. 3091, 3104–09 (2018) (noting how racial themes can arise in the context of prosecutorial summations); L. Song Richardson & Phillip Atiba Goff, Implicit Racial Bias in Public Defender Triage, 122 Yale L.J. 2626, 2631–40 (2013) (discussing how implicit racial bias may affect public defender decisions).

Table 1. Lawyer Demographics

Some may argue that the true cause of the racial and ethnic imbalance in the legal profession is that there are not enough qualified BIPOC candidates.⁶²See, e.g., George B. Shepherd, No African Lawyers Allowed: The Inefficient Racism of the ABA’s Accredited Schools, 53 J. Legal Educ. 103, 104–05 (discussing how the American Bar Association’s accreditation system’s focus on “qualifications” excludes Black people from law school); Sandra S. Yamate, Quest for the ‘Qualified’ Minority, Or. State Bar Bull. 9, 9 (2002) (discussing the problems with law firms seeking to recruit “qualified” candidates from law schools). But as Eugene K. Pettis notes:

That argument is baseless. Somehow recruiters find a way to enroll a disproportionately higher percentage of African-American football and basketball players to Division 1 schools across the country in comparison to their overall numbers in higher education. A “shallow pool of college attendees” never gets in the way of that recruitment effort.⁶³Eugene K. Pettis, RX Warning: Quitting Diversity Efforts Too Soon May Result in Harmful Relapse, 9 Fla. B.J. 18, 21 (2018) (discussing diversity in the judiciary).

Similarly, if the legal profession wanted more BIPOC attorneys, it could find a way to fill more slots in law schools with students from communities of color and ensure that those law graduates enter the profession at a higher rate. Those raising this “shallow pool problem” to explain the lack of attorneys and law students from communities of color are falling prey to implicit bias when they assume that the current system is fair (does not unjustly burden or benefit any racial or ethnic group) and accurate (uses the correct predictors of future success in law school and in the practice of law)—it is neither.⁶⁴See, e.g., Lu Hong & Scott E. Page, Groups of Diverse Problem Solvers Can Outperform Groups of High-Ability Problem Solvers, 101 PNAS 16835, 16835 (2004) (providing the results of a decision-making model demonstrating that increasing diversity among decision makers can result in better outcomes than lower diversity focused solely on “high-performing” decision makers); Ayesha Whyte, Recognizing Implicit Bias to Promote Diversity and Support a Culture of Inclusion and Innovation, Forbes (Jan. 29, 2021), https://www.forbes.com/sites/forbeshumanresourcescouncil/2021/01/29/recognizing-implicit-bias-to-promote-diversity-and-support-a-culture-of-inclusion-and-innovation/?sh=2d7a93ce1cdb [https://perma.cc/N8ZT-LLVV]. Hiring practices and their “hidden biases” result in “hir[ing] from a shallow talent pool.” Whyte, supra. The legal profession’s racial and ethnic imbalance is all the more problematic considering its longstanding history. As Table 2 shows, for the last ten years, the proportion of lawyers who are American Indian/Alaska Native and Black have seen small declines; Native Hawaiian/Pacific Islander lawyers have seen no measurable change; Asian and Hispanic lawyers have seen small increases; and lawyers who self-identify as Two or More Races have seen the largest, but still small, increase.

Table 2. Ten-Year Trends in Lawyer Demographics⁶⁵ABA National Lawyer Population Survey: 10-Year Trend in Lawyer Demographics, supra note 58 (providing lawyer population demographics from 2011 to 2021).

A. Maybe No One Will Notice the Problem

Well-known racial and ethnic imbalances in the legal profession beg a preliminary question: Why do the legal profession⁶⁶While debate over potential changes to the bar exam can be hotly contended at the time they are proposed, once the change is made, the legal profession is effectively walled out from knowing whether the change to the bar exam caused any (negative or positive) changes. For example, when the Florida Supreme Court raised the score required to pass the bar exam in 2003, three years later, the then-chair of the Florida Board of Bar Examiners refused to provide information as to the impact of that change on the pass rates of BIPOC test-takers stating: “This is a question that the board is studying and will forward its findings to the court.” Jan Pudlow, Has Raising the Pass/Fail Lines on the Bar Exam Had a Disparate Impact on Minority Applicants?, Fla. Bar News (Dec. 1, 2006), https://www.floridabar.org/the-florida-bar-news/examining-the-exam/ [https://perma.cc/W3UX-WB8Y]. This lack of information may lead members of the profession to assume that “no news is good news”—when that may not be the case. and the public⁶⁷The public seems to ignore the fact of racial/ethnic disparity in bar results even when that data is made known. For example, the ABA has just begun releasing bar data by race. See Stephanie Francis Ward, New ABA Data Parses out Bar Exam Pass Rates by Race and Ethnicity (June 22, 2021), https://www.abajournal.com/news/article/new-aba-data-parses-out-bar-exam-pass-rates-by-ethnicity [https://perma.cc/JEC8-5DNQ]. When we look at that data, we see that BIPOC exam takers underperform against White test takers. See Summary Bar Pass Data: Race, Ethnicity, and Gender 2020 and 2021 Bar Passage Questionnaire, Am. Bar Ass’n, https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/statistics/20210621-bpq-national-summary-data-race-ethnicity-gender.pdf. Public response to this data has been effectively non-existent. The Authors have searched major news media outlets and could not find discussion of these results. Similarly, while a search using Google produces hits on the article, only one moderately well-known news source mentions the article. See Sam Skolnik, Bar Exam Race Gap Shown in New Passage Rate Data for Law Grads, Bloomberg L. (June 22, 2021), https://news.bloomberglaw.com/daily-labor-report/bar-exam-race-gap-shown-in-new-passage-rate-data-for-law-grads [https://perma.cc/4PDM-AJPL]. All other search results are for niche law professor blogs like Taxprof.blog and law news aggregators/websites. largely ignore the issue of the bar exam as a factor in creating this imbalance?⁶⁸Despite the fact that the issue of racial and ethnic disparity in bar results is not in the public eye, there are a number of people working on the problem. See, e.g., Claudia Angelos, Sara J. Berman, Mary Lu Bilek, Carol L. Chomsky, Andra A. Curcio, Marsha Griggs, Joan W. Howarth, Eileen Kaufman, Deborah Jones Merritt, Patricia E. Salkin & Judith Welch Wegner, The Bar Exam and the Covid-19 Pandemic: The Need For Immediate Action (Ohio State Univ. Moritz Coll. of L., Legal Studies Working Paper No. 537, 2020), https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3559060_code182808.pdf?abstractid=3559060&mirid=1 (discussing the need for and proposing options for licensing attorneys in light of the COVID-19 crisis); Deborah Jones Merritt & Logal Cornett, Inst. for the Advancement of the Am. Legal Sys., Building a Better Bar: The Twelve Building Blocks of Minimum Competence (2020), https://iaals.du.edu/sites/default/files/documents/publications/building_a_better_bar.pdf [https://perma.cc/6WQF-2SVE] (discussing the creation of competency-based measures for admission to the bar). We believe that two key factors produce this result. First, there is effectively no publicly available data regarding bar passage rates by race or ethnicity.⁶⁹See infra Appendix (providing the results of our review of publicly available jurisdiction-specific data on race and bar passage rates). As noted infra, in Section II.B, the second factor is that, due to historical efforts to address diversity in law, many in the legal profession believe the issue has already been remediated to the degree possible. California is the only U.S. jurisdiction that provides pass rates by race and ethnicity in its reported statistics each year.⁷⁰See State Bar of Cal., General Statistics Report: July 2019 California Bar Examination (2019), https://www.calbar.ca.gov/Portals/0/documents/July2019-CBX-Statistics.pdf [https://perma.cc/Z83X-T54C](providing bar passage results by race/ethnic group). Comparing July 2018 to July 2019, we see a major shift in the racial and ethnic categories with fewer examinees now categorized as “White,” “Black,” “Hispanic,” or “Asian,” while the vast majority are recorded as “Other.” Id. at 2. The vast majority of test takers did not provide their racial or ethnic background, and so these test takers were categorized as “Other.” Compare id., with State Bar of Cal., General Statistics Report: July 2018 California Bar Examination (2018), https://www.calbar.ca.gov/Portals/0/documents/admissions/JULY2018_CBX_Statistics.pdf [https://perma.cc/G6DA-HWSH]. It is unclear why this change occurred. We also undertook a review of State Bar reporting sites to confirm that California was the only state currently reporting results by race and ethnicity. See infra Appendix (finding only one state, California, reporting aggregate bar passage results by race and ethnicity); see also Pudlow, supra note 66 (noting that the California Bar publicly provides data on bar passage by race and ethnicity). Information from the July 2004 administration of the Texas bar exam is also available, as a result of the legislature directing the Texas Board of Law Examiners to produce such data.⁷¹Tex. Gov’t Code Ann. § 82.029 (West 2021) (“The Board of Law Examiners shall compile a report indicating the number of applicants who fail the July 2004 bar examination. The data shall be aggregated by gender, ethnicity, and race. The report shall also include an analysis of the identifiable causes of failure and recommendations, if any, to address the causes of failure. The board shall deliver the report to the legislature not later than December 31, 2004.”). Given this state of affairs, people of good intentions simply lack concrete proof that a problem exists.

Nonetheless, the data from California and Texas are clear: BIPOC examinees pass the bar exam at much lower rates than White examinees. For example, Table 3 shows the July first-time bar passage rates in California from 2010 to 2019, highlighting a striking difference in pass rates—with Asian, Black, and Hispanic examinees passing at lower rates than White examinees.

Figure 1. July First-Time California Pass Rate—Graduates of ABA-Accredited Law Schools⁷²See Exam Statistics, State Bar of Cal., http://www.calbar.ca.gov/Admissions/Law-School-Regulation/Exam-Statistics [https://perma.cc/LS4G-ZH82] (providing bar examination outcomes by race for each bar examination from June 2007 to February 2021).

This data shows that for the last decade, on average, compared to White examinees, Asian examinees’ pass rates are 8.4 percentage points lower, Black examinees’ pass rates are 22.9 percentage points lower, and Hispanic examinees’ pass rates are 15.5 percentage points lower, on the annual July administration of the California bar exam. In addition, the minimum difference between White and Asian examinees is 3.1 percentage points, for Black examinees is 16.7 percentage points, and for Hispanic examinees is 10.2 percentage points. Furthermore, the maximum difference between Asian examinees and White examinees is 16 percentage points, for Black examinees is 27 percentage points, and for Hispanic examinees is 27.6 percentage points. Finally, at no point is the pass rate of Black, Hispanic, or Asian examinees higher than that of White examinees.

We find similar results in the Texas data. Pursuant to legislation, the Texas Board of Law Examiners collected pass rates from the July 2004 Texas Bar Exam for Asian, Black, Hispanic, and White Texas bar examinees.⁷³Tex. Gov’t Code Ann § 82.029. As Table 3 shows, the first-time pass rates for Black, Hispanic, and Asian examinees were below those of White examinees. Asian examinees’ pass rates were 9 percentage points lower, Black examinees’ pass rates were 33 percentage points lower, and Hispanic examinees’ pass rates were 16 percentage points lower. Over the next two years (from 2004 to 2006), pass rates increased as those who failed the exam retook it. This closed the gap, but White examinees’ two-year pass rate was still higher than Asian examinees by 2 percentage points, Black examinees by 17 percentage points lower, and Hispanic examinees by 5 percentage points.

Table 3. July Texas Pass Rate—Graduates of ABA-Accredited Law Schools, 2004-2006⁷⁴Stephen P. Klein & Roger Bolus, Initial and Eventual Passing Rates of July 2004 First Timers 5 tbl.5 (2006), https://ble.texas.gov/klein-report-0606 [https://perma.cc/W5PJ-T4YT] [hereinafter TX Passing Rates Report] (reporting on results of study of Texas bar takers); see also Stephen P. Klein & Roger Bolus, Analysis of July 2004 Texas Bar Exam Results by Gender and Racial/Ethnic Group (2004), https://ble.texas.gov/statistics?keyword=klein#Question2 [https://perma.cc/VCX5-NFRP] [hereinafter Gender & Racial/Ethnic Group Analysis].

While limited, the data support only one conclusion: BIPOC examinees underperform on the bar examination compared to their White peers. Moreover, the fact that only two states provide such data, and only one of those states does so regularly, further supports our contention that there is a vital need for data and research in this area.

B. Didn’t We Already Fix This?

Along with the lack of publicly available data, this Article urges a second explanation for the profession’s failure to take clear and decisive action: the profession previously acted to address the problem and thereby concluded that the issue is fixed or being addressed by someone else.⁷⁵In addition, as noted above, because the data have been withheld from public view, the profession has limited visibility into the problem and therefore has difficulty seeing that the problem has not been fixed. See supra Section III.A (discussing the lack of publicly available data).

In 1988, the Conference of Chief Justices adopted a resolution “encouraging all chief justices to establish task forces devoted to the study of gender bias and minority concerns as they relate to the judicial system.”⁷⁶Myra C. Selby, Examining Race and Gender Bias in the Courts: A Legacy of Indifference or Opportunity?, 32 Ind. L. Rev. 1167, 1169 (1999). At the time, there were already four such task forces examining issues of racial and ethnic bias (in New Jersey, Michigan, New York, and Washington) and subsequent to the conference resolution, over twenty other states created task forces to examine issues of racial and ethnic bias in the judicial system.⁷⁷See id. at 1169–70 (discussing the formation of task forces examining racial and gender bias in the judicial system).

For example, in 1989, Chief Justice Raymond Ehrlich of the Florida Supreme Court ordered that the Racial and Ethnic Bias Study Commission be created “to address the question of whether racial or ethnic considerations adversely affect the dispensation of justice to minority Floridians.”⁷⁸Fla. Sup. Ct. Racial & Ethnic Bias Comm’n, “Where the Injured Fly for Justice”: Reforming Practices Which Impede the Dispensation of Justice to Minorities in Florida, First Report Executive Summary 4 (1990), http://www.floridasupremecourt.org/pub_info/documents/racial.pdf [https://perma.cc/N4G9-RURL]. The Commission found a “stark disparity” in bar passage rates of Black examinees as compared to White examinees.⁷⁹See id. at 20–21. For the February 1991 bar administration, only 39% of Black examinees passed compared to 74% of White examinees.⁸⁰Id. That July, only 46% of Black examinees passed compared to 76% of White examinees.⁸¹Id. at 21. Based on the results of its study, the Commission recommended that the Florida Board of Bar Examiners take eight separate actions, including monitoring performance by race, reviewing questions on which Black and White test takers perform differently, reviewing questions for cultural bias, and including “minorities among those individuals who develop . . . questions for use in the Florida Bar Exam,” to remedy the problem.⁸²Id. at 21–22.

Similarly, in 1991, the New York Judicial Commission on Minorities published the results of their study on the interaction between race and bar passage.⁸³See Report of the New York State Judicial Commission on Minorities, 19 Fordham Urb. L.J. 181, 262 (1992) (discussing the role of the bar exam in admission to the bar). The Commission found, for the July administration of the state’s bar examination between 1985 and 1988, that examinees from communities of color had lower first-time bar passage rates as compared to White first-time takers.⁸⁴See id. at 263 (describing bar passage rates by race and ethnicity). The Commission found that, on average, Asian examinees passed at a 62.9% rate, Black examinees at 31.0%, Hispanic examinees at 40.9%, and Native American examinees at 33.3%, while White examinees passed at a rate of 73.1%.⁸⁵See id. The New York Commission found, as a matter of fact, that: (1) examinees from communities of color have “exceedingly low” pass rates, (2) the legal community as a whole “has a stake in increasing minority pass rates,” (3) the bar examination “has not been evaluated for cultural/economic bias and job-relatedness,” and (4) “[m]inorities are not adequately represented among contract graders and staff of the New York State Board of Law Examiners.”⁸⁶Id. at 269. In light of these findings, the Commission made a number of recommendations, including monitoring performance by race and reviewing questions for cultural bias.⁸⁷See id.

The Minnesota Supreme Court Task Force on Racial Bias in the Judicial System also found racial and ethnic bias throughout the Minnesota judicial system, including in the bar exam.⁸⁸Richelle M. Wahi, Minnesota Judicial Branch Action Following the 1993 Minnesota Supreme Court Task Force on Racial Bias in the Judicial System and Recommendations for Minnesota Judicial Branch Action in FY20-21 at 4, 29 (2019), https://www.mncourts.gov/mncourtsgov/media/scao_library/CEJ/Racial%20Fairness%20Committee/2019-Progress-Report-ON-1993-RACE-BIAS-TASK-FORCE-AND-RECOMMENDATIONS-UPDATED-4-29-19-WITH-APPENDICES.pdf [https://perma.cc/A4RT-HK35]. A number of factors were isolated as potential causes of the racial and ethnic bias in bar exam outcomes, including:

English as a second language; unequal quality of education received prior to law school; financial status (i.e. needing to work during law school and during preparation for the bar); availability and/or efficacy of minority‐focused tutoring programs; possible bias in some elements of law school curricula; possible bias in private bar preparation program curricula; the impact of poverty; the particular law school attended, LSAT scores, law school rank, etc.⁸⁹See id. at 29.

In response to these concerns, the Minnesota State Bar Association and Board of Law Examiners implemented several interventions, including “ensur[ing] that all law examination questions are reviewed for bias and that at least 25% of graders are people of color.”⁹⁰See id.

As we can see, many state bars and judiciaries took seriously the problem of bias and proposed clear, common-sense solutions. Particularly in light of the lack of publicly-available data to the contrary, the average lawyer or judge aware of this history could conclude that the problem has been solved (or is being addressed), and that any difference in bar outcomes along racial and ethnic lines is either minimal or due to differences in entering credentials—not bias in the exam.⁹¹In addition, some may argue that any negative correlation between bar passage rates and the percentage of BIPOC graduates (i.e., as the percentage of BIPOC graduates increases, the school’s pass rate decreases) is due to some percentage of those students being admitted to schools whose median credentials are well above that of those students. This mismatch theory contends “that because professors pitch their lectures and assignments to the level of the median student, students with academic credentials well below their school’s median find it hard to understand lectures and assignments and otherwise keep up.” Richard Lempert, Mismatch and Science Desistance: Failed Arguments Against Affirmative Action, 64 UCLA L. Rev. Discourse 136, 138 (2016). This argument fails because the empirical evidence “find[s] that minorities benefit from attending schools where they are, according to the theory, overmatched.” Id. at 141 (discussing empirical studies on the impact of affirmative action on beneficiaries of affirmative action).

III. Empirical Studies of Race and the Bar Examination: Data Desert, Whitewashing, and Heads Buried in the Sand

Research into the relationship between race/ethnicity and bar passage rates is difficult to undertake. Individual examinee bar passage data is confidential⁹²See, e.g., Keith Kamisugi, Privacy, Not Political Correctness: Ideology, Not Science, Behind Richard Sander Request for Confidential Bar Exam Data, Equal Just. Soc’y (Sept. 15, 2008), https://equaljusticesociety.org/2008/09/15/privacy-not-political-correctness-ideology-not-science-behind-richard-sander-request-for-confidential-bar-exam-data/ [https://perma.cc/436F-YPFR] (discussing how a lawsuit seeking California Bar examination records from the State Bar violates the privacy of the applicants to the bar and that “[t]he State Bar holds the confidential information of Bar applicants in trust”); NCBE Privacy Policy, Nat’l Conf. Bar Exam’rs (May 17, 2021), https://www.ncbex.org/privacy-policy/ [https://perma.cc/TLW4-99H6] (providing the NCBE’s privacy policy and noting that the NCBE “is committed to protecting your privacy”); Pa. Bd. of L. Exam’rs, Pennsylvania Bar Admission Rules 33–34 r. 402 (2019), https://www.pabarexam.org/pdf/rules.pdf [https://perma.cc/2456-2T8B] (requiring that the actions and records of the Pennsylvania Board of Law Examiners be confidential and “not . . . disclosed or open to inspection by the public”). and, as a result, primarily in the hands of the State Bar, State Bar examiners, and the individuals themselves. At the same time, data about examinees that could be used to understand bar passage results (like law school GPA and LSAT score) are confidential under federal law⁹³See, e.g., Family Educational Rights and Privacy Act (FERPA), 34 C.F.R. § 99.2 (2008) (“The purpose of this part is to set out requirements for the protection of privacy of parents and students under . . . the General Education Provisions Act.”); FERPA, 34 C.F.R.§ 30(a) (2004)(“[P]arent[s] or eligible student[s] shall provide a signed and dated written consent before an educational agency or institution discloses personally identifiable information from the student’s education records.”); FERPA, 34 C.F.R.§ 99.3 (2011) (“Education records . . . [a]re (1) [d]irectly related to a student; and (2) [m]aintained by an education agency or institution or by a party acting for the agency or institution.”); cf. FERPA, 34 C.F.R.§ 99.31 (2011) (permitting the disclosure of directory information); FERPA, 34 C.F.R. § 99.3 (2011) (“Directory information [to] include[] . . . the student’s name; address; telephone listing; electronic mail address; photograph; date and place of birth; major field of study; grade level; enrollment status . . . ; dates of attendance; participation in officially recognized activities and sports; weight and height of members of athletic teams; degrees, honors, and awards received; and the most recent educational agency or institution attended.”). and remain in the hands of the law schools that gathered the information and the individual examinees from whom the data were collected. These organizations are also reluctant or unable to share even anonymized data.⁹⁴The Authors reached out to law schools in an attempt to gather this data. This produced three results: (1) the dean of the law school ignored the request for a call to discuss the data; (2) the dean of the law school expressed interest but ultimately did not share the data; and (3) in the minority of cases, the dean of the law school agreed to share the data. One of our goals in publishing this study is to help law school deans see the need for more in-depth study and therefore provide access to the data necessary for such analysis. In addition, seeking this information directly from individual bar examinees would be cost prohibitive.

Despite these difficulties, several empirical studies have been undertaken. These studies can be divided into three categories based on their conclusions: (1) one study that cannot draw a conclusion due to lack of data,⁹⁵See George Neff Stevens, Bar Examinations and Minority Group Applicants, 56 ABA J. 969, 969–70 (1970) (discussing failure to record information as to race/ethnicity and bar passage in 1969 survey of law school deans). (2) studies that find disparate outcomes along racial and ethnic lines but then attempt to whitewash (obscure or explain) the results,⁹⁶See Stephen P. Klein & Anthony McDermott, An Examination of Possible Item, Test, and Grader Bias in the California Bar Examination, 4 Black L.J. 553, 557 (1975) (claiming that even though their study found evidence of disparate outcomes, those outcomes were not the result of bias in the exam); Linda F. Wightman, LSAC National Longitudinal Bar Passage Study 52 (1998), http://files.eric.ed.gov/fulltext/ED469370.pdf [https://perma.cc/HWS7-SLSK] (arguing that while race and ethnicity are a statistically relevant factor for bar passage, the effect of their addition to a model including LSAT and LGPA is minimal); see also Tex. Gov’t Code Ann. § 82.0291 (West 2004) (expired 2005) (directing the Texas Board of Law Examiners to report on bar passage rates for the July 2004 bar exam by gender, ethnicity, and race). and (3) studies conducted by state-level actors finding racial and ethnic disparities in bar passage outcomes.⁹⁷See Kristin Booth Glen, Thinking out of the Bar Exam Box: A Proposal to “MacCrate” Entry to the Profession, 23 Pace L. Rev. 343 (2003) (discussing 1992 study commissioned by the New York Court of Appeals); William C. Kidder, The Bar Examination and the Dream Deferred: A Critical Analysis of the MBE, Social Closure, and Racial and Ethnic Stratification, 29 L. & Soc. Inquiry 547, 570 (2004) (discussing 2001 study conducted by the Florida Board of Bar Examiners as ordered by the Florida Supreme Court); Nat’l Conf. of Bar Exam’rs, Executive Summary: Impact of Adoption of the Uniform Bar Examination in New York (2019), https://www.nybarexam.org/UBEReport/NY%20UBE%20Adoption%20Part%201%20Executive%20Summary.pdf [https://perma.cc/AFE8-UQAP] (reporting on request from the New York State Board of Law Examiners to the NCBE to examine the impact of adoption of the UBE in New York). As we discuss below, all of these studies (other than that which failed for lack of data) support the conclusion that race and ethnicity are factors in bar passage.

A. No Data, No Result

The earliest large-scale, empirical study of race/ethnicity-related differences in bar passage rates that we could find was conducted in 1969, by George Neff Stevens—professor at the Texas Tech University School of Law and former Dean of the University of Washington School of Law—when “the deans of 133 law schools approved by the [AALS and ABA]” received a questionnaire on bar passage rates.⁹⁸Stevens, supra note 95, at 969, 971 (outlining the study and providing Professor Stevens’ biographical sketch). This study requested information regarding the total number of graduates, the number who passed, the number who failed, and the number for whom the pass rate was unknown.⁹⁹Id. at 969. This information was also “elicited with respect to [Black], [Native American], Mexican-American, [Hispanic] and [Asian]-American students.”¹⁰⁰Id. In this Article, when describing the results of a survey or study, we have replaced outdated terms for racial and ethnic groups that are no longer used and that may be offensive to readers today. You will find the replaced terms in brackets instead of the original terms used. We chose to change the terms because of the harm that historical language can cause. Please reach out to the authors if you would like the original categorizations. The ninety-eight questionnaires completed and returned demonstrated that “very few [Black], [Native American], Mexican-American[], [Hispanic] and [Asian]-American[ students] [] graduated from approved law schools during the period of 1964–1968.”¹⁰¹Id. Unfortunately, many deans simply had no data as to pass rates.¹⁰²Id. But, given the paucity of graduates from communities of color, some deans were able to provide a “recollection” report and

along these lines, a substantial number of deans stated that all their minority group graduates had passed a bar examination somewhere [or] . . . had done better than their white counterparts on the bar examinations . . . or at least as well . . . or that they compared favorably, or showed about the same ratio of pass-fail in each quartile. Several Deans stressed the point that, because they had so few minority group graduates, any attempt at comparison would be inappropriate.¹⁰³Id. at 969–70.

The study concluded that the lack of data made it “virtually impossible” to determine if the bar exam produced racially or ethnically disparate results and recommended the creation of a national data bank to track such information.¹⁰⁴Id. at 970. Thus, the first empirical effort to understand whether the bar exam was racially or ethnically biased failed for lack of data.

B. Whitewashing Racially Disparate Outcomes

In 1975, Professors Stephen Klein and Anthony McDermott published the results of a study “to determine if there was cultural bias in the California Bar Examination.”¹⁰⁵Klein & McDermott, supra note 96, at 553. Using a “predictor score” computed based on undergraduate grade point average (UGPA), law school admission test (LSAT) score, and law school grade point average (LGPA),¹⁰⁶See id. at 555 (discussing the regression equation for the study). this study found that Anglo (White) examinees received higher scores on the bar exam than Black or “Chicano”¹⁰⁷
. Since the early twentieth century, Mexican Americans have used the word “Chicano” to describe people of Mexican origin living in the United States (feminine: Chicana, gender-neutral: Chicanx). Roque Planas, Chicano: What Does the Word Mean and Where Does It Come From?, HuffPost: Latino Voices (Oct. 21, 2012) https://www.huffpost.com/entry/chicano_n_1990226 [https://perma.cc/JUP6-ZWAR]. Klein and McDermott use the masculine “Chicano” and “Latino,” but since their study in 1975, gender-neutral terms like “Latinx” and “Latine” have increased in popularity. Compare Klein & McDermott, supra note 96, with Terry Blas, “Latinx” Is Growing in Popularity. I Made a Comic to Help You Understand Why., Vox (Oct. 23, 2019) https://www.vox.com/the-highlight/2019/10/15/20914347/latin-latina-latino-latinx-means [https://perma.cc/4HRC-N6KJ. examinees with the same predictor score.¹⁰⁸See Klein & McDermott, supra note 96, at 555. Another problem we find in interpreting studies over time is the changing language used to name various ethnic groups. Thus far (with only two studies discussed) we have already seen examinees who likely would be called “Hispanic” divided into three categories “Mexican-American,” “Spanish-American,” and “Chicano.” Moreover, while our first study included Asian examinees, they are not included in this study. Having established that there were racially/ethnically disparate outcomes on the bar exam, Klein and McDermott conducted further analysis that, they claim, shows that the exam itself is not actually biased.¹⁰⁹See id. at 555­–56 (describing two methods for determining if Black or Chicano students would have passed the bar had the exam not been biased against them). They use two methods to estimate how many Black and Chicano students would have passed the bar if there was no bias as to scores.¹¹⁰See id. They conclude from these analyses that there was “no significant difference between the percentage of minority group members who actually passed versus those who would have been expected to pass had no bias existed.”¹¹¹Id. at 556 (emphasis in original). In essence, they claim that while there is bias in exam scores, it is inconsequential because no significant difference in pass rates could be expected even if the test was not biased. They conclude that:

the major implication of this [sic] findings for the present study is that it further reduces the likelihood that the bar examination itself is biased. In other words, what differences in performance that are observed between Anglo and minority candidates appear to be primarily due to differences in ability rather than some inherent bias in the test as a whole.¹¹²Id. at 557.

So, paradoxically, even though Klein and McDermott found clear evidence of racial and ethnic bias in bar exam scores, they conclude that the test itself is not racially or ethnically biased.¹¹³See id. (“On the basis of the foregoing discussion and findings, it seems reasonable to conclude that there is no statistical evidence that the predictors of Bar performance are biased against minority group members.”).

There are, at least, three problems with this conclusion. First, Klein and McDermott provided unequivocal evidence that the California bar produced racially and ethnically disparate outcomes: they found that Anglo examinees outperformed Black and Chicano examinees who had the same predictor scores.¹¹⁴See id. Moreover, they note that students with the same LSAT scores who differ in race or ethnicity can expect different scores on the bar exam.¹¹⁵Id. at 556 (discussing their first method for calculating whether the difference in score would produce a difference in bar passage rates). These results are the very essence of bias.

Second, the reported difference in bar scores between Anglo and Black or Chicano examinees is considerable. For example, Klein and McDermott report that “a Black candidate with an LSAT score of 541 would be expected to score 1547 on the Bar, whereas an Anglo with a 541 LSAT would be expected to score 1600.”¹¹⁶Id. at 556. The LSAT score scale has changed over time. It was scored on a 200–800 scale until 1982, changed to a scale of 10–48 from 1982 to 1989, and in 1990 was changed to a scale of 120–180. See Leslie G. Espinoza, The LSAT: Narratives and Bias, 1 Am. U. J. Gender & L. 121, 159 n.248 (1993). That 53-point difference is 3.3% of the Anglo examinee’s score. That difference does not seem to be “insignificant,” as characterized. But we have no way of understanding this difference because, rather than fully explaining the disparity, Klein and McDermott merely tell us that “the magnitude of the bias is not the same throughout the distribution of predictor scores, for example the bias appears greatest for Black examinees with low LSAT scores while the bias against Chicano examinees is greatest for those with high LSAT scores.”¹¹⁷Klein & McDermott, supra note 96, at 555. Thus, we do not know if the example score difference is low, high, or near the average.

Third, Klein and McDermott’s predictor includes the examinees’ LSAT score. They note that “[t]he assumption underlying the analysis of test bias is that [LSAT is itself] unbiased with respect to assessing a candidate’s ability.”¹¹⁸Id. at 556. They then contend “that the LSAT may be biased in favor of minority groups in the sense that compared to Anglos, it overestimates minority group performance levels.”¹¹⁹Id. at 557. The conclusion that the LSAT is not biased against examinees from communities of color is not justified by the more recent record. As seen in Table 4, the average LSAT score is different depending on the examinees’ race or ethnicity:

Table 4. LSAT Scores and Examinee Demographics 2007-2014¹²⁰See Susan P. Dalessandro, Lisa C. Anthony & Lynda M. Reese, L. Sch. Admission Council, LSAT Technical Report 14-02, LSAT Performance with Regional, Gender, and Racial/Ethnic Breakdowns: 2007-2008 Through 2013-2014 Testing Years 22–23 (2014) (providing average score by race and ethnicity).

These substantial differences demonstrate that the LSAT does, in fact, produce disparate results based on race and ethnicity.

In addition, there is evidence that the difference in outcomes does not dissipate when controlling for factors such as college attended, UGPA, or major. For example, William Kidder “matched African American, Chicano/Latino, Native American, and Asian Pacific American applicants with White applicants who possessed equivalent [UGPAs] from the same colleges during the same time period” (1996 to 1998).¹²¹See William C. Kidder, Does the LSAT Mirror or Magnify Racial and Ethnic Differences in Education Attainment?: A Study of Equally Achieving “Elite” College Students, 89 Calif. L. Rev. 1055, 1058 (2001) (discussing the study population). Kidder then looked to see if there were racial and ethnic differences in their LSAT scores. He found “that among law school applicants with essentially the same performance in college, students of color encounter a substantial performance difference on the LSAT compared to their White classmates. These gaps are most severe for African American and Chicano/Latino applicants.”¹²²Id. These outcomes did not change even when Kidder matched by major within the same school.¹²³See id.

The LSAT has also been determined to be a “speeded” exam that tests examinees’ reasoning ability and test-taking speed.¹²⁴See William D. Henderson, The LSAT, Law School Exams, and Meritocracy: The Surprising and Undertheorized Role of Test-Taking Speed, 82 Tex. L. Rev. 975, 1032 (discussing that the LSAT appears to measure two variables). Studies show that minority students are disparately impacted by time limits in the context of exams where test-taking speed is relevant.¹²⁵See Franklin R. Evans & Richard R. Reilly, A Study of Speededness as a Source of Test Bias, 9 J. Educ. Measurement 124, 127 (1972) (finding that Black examinees, at historically Black colleges/universities, gained thirty-three points on an unspeeded reading comprehension section compared to Black examinees who took a speeded exam at the same location, while White examinees gained only twenty-two points compared to speeded test takers at the same location); see also id. at 196 (noting that Black female examinees improved more than White female examinees when ten minutes were added to a reading comprehension section, while Black men also improved but not as much relative to White male examinees). Thus, the speeded nature of the exam could explain the differential outcomes based on an examinees’ race/ethnicity. That difference would not be due to the examinees reasoning ability, but rather the speed with which they completed the test.

In this context, Klein and McDermott’s reliance on the race- and ethnic-neutrality of the LSAT to support their conclusion that the bar is not racially/ethnically biased—combined with their finding a large difference in bar score based on race and ethnicity—makes their conclusion that the bar exam is not biased highly unreliable.

In 1991, Dr. Linda Wightman, Law School Admission Council (LSAC) Vice President for Test Development and Research, undertook for LSAC a five-year national study of bar passage rates.¹²⁶Wightman, supra note 96, at vi, 12. This study of 23,103 subjects is the largest and most comprehensive empirical study on the issue of bias in the bar exam that we have identified to date.¹²⁷See id. at 6. The study found statistically significant differences in first-time passage rates between White examinees and examinees from communities of color.¹²⁸Id. at 32.

Table 5. LSAC National Study of Bar Passage Rates¹²⁹Id. at 27 (providing listing passage rates by race and ethnicity).

Dr. Wightman also divided the data into six clusters based on “law school group,” where each school was placed into a group with schools “most like themselves.”¹³⁰Id. at 28. Cluster analysis was used to place the schools into clusters based on seven factors: size, cost, selectivity, faculty/student ratio, percent minority, median LSAT, and median UGPA. See id. at 8–9. “The cluster analysis identified six naturally occurring clusters or groups of law schools.” Id. at 9. Even when examinees were clustered in this manner, there remained a statistically significant negative correlation between race/ethnicity and bar passage rate.¹³¹Id. at 29. The results of this analysis are provided in Table 6.

Table 6. Bar Passage Rates Among Law School Clusters¹³²Id. at 28 tbl.7 (“Number and percentage of study participants who passed the bar on the first attempt, by ethnic group and law school cluster”).

The difference becomes starker when comparing the results between each racial/ethnic group and White examinees. In every single case, examinees from communities of color underperform White examines.

Table 7. Results of Racial/Ethnic Groups as Compared to White Examinees (values in percentage points)¹³³See id. at 28 tbl.7.

Dr. Wightman also found that adding ethnicity to a model of bar passage rates based on just LGPA and LSAT “showed a modest but statistically significant improvement over the LGPA and LSAT score model.”¹³⁴Id. at 52 (discussing models of first-time bar passage based on study data). The implication of the study’s conclusion, that “[t]he data . . . demonstrate that LGPA and LSAT score explain more of the variation in bar passage outcomes than do any of the other variables [including race/ethnicity] examined,”¹³⁵See id. at 48. is that race and ethnicity are not important factors for bar passage. Rather, the study posits, the difference is explained by the lower entering credentials and weaker law school performance of students from communities of color (compared to White students).¹³⁶See id. at 80.

There are two problems with this result. First, it is worrisome that the primary investigator, Dr. Wightman, is an employee of LSAC, which develops and administers the LSAT.¹³⁷See Wightman, supra note 96, at vi, viii–ix; About the Law School Admission Council (LSAC), L. Sch. Admission Council, https://www.lsac.org/about [perma.cc/MXR3-SJB6] (discussing services that LSAC provides including administering the LSAT). As such, her analysis may have been influenced by her position in the organization.¹³⁸While we seek to cast no aspersions on the character of Dr. Wightman, the dangers associated with research conducted by those with a pecuniary interest in the outcome are well-known and long-established. See, e.g., Mark Barnes & Patrick S. Florencio, Financial Conflicts of Interest in Human Subjects Research: The Problem of Institutional Conflicts, 30 J.L. Med. & Ethics 390, 391–92 (2002) (discussing how financial incentives can affect professional judgment); Bryan K. Church & Xi (Jason) Kuang, Conflicts of Interest, Disclosure, and (Costly) Sanctions: Experimental Evidence, 38 J. Legal Stud. 505, 505–06 (2009) (discussing the problem of financial conflicts of interest); Pilar N. Ossorio, Pills, Bills and Shills: Physician-Researcher’s Conflicts of Interest, 8 Widener L. Symp. J. 75, 88 (2001) (“One reason that conflicts of interest create the probability that physician-researchers’ obligations will go unfulfilled is because conflicts may undermine judgment.”).

Second, as noted above, LSAT scores themselves correlate with race and ethnicity. In essence, because LSAT scores are correlated with both some aspect of academic ability and race/ethnicity, a model based on LSAT and LGPA is really a model based on some measure of academic ability, race, ethnicity, and LGPA (success in law school). Adding a second variable that accounts for race or ethnicity should be expected to improve predictability only marginally, as that new variable accounts only for aspects of race and ethnicity not already captured by the LSAT. The fact that the addition of race and ethnicity improves the model only underscores the vital importance of race and ethnicity to bar passage rates. Furthermore, Dr. Wightman presents regression results showing negative and statistically significant relationships between Black, Hispanic, and Asian American students’ race/ethnicity and the probability of passing the bar, after controlling for LSAT and LGPA scores.¹³⁹See Wightman, supra note 96, at 52 n.85 (“The data in the table above show that, for study participants who had the same LGPA and LSAT score, being Hispanic or Asian American instead of white reduced the odds ratio to approximately two thirds, while being black reduced it to approximately three quarters.”). And, as we will see, when we remove LSAT measures from our own analysis, leaving only race/ethnicity in the model, there is merely a small change in the predictivity of the model.¹⁴⁰See infra Section V.B (discussing the LSAT Model and Lagged LSAT Model compared, respectively, to the Race & Ethnicity Model and the Lagged Race & Ethnicity Model).

C. Heads Buried in the Sand

In 1992, the New York Court of Appeals commissioned a study of the New York bar exam.¹⁴¹See Booth Glen, supra note 97, at 346–349, 503 (discussing the study conducted by the Commission on Legal Education and Admission to the Bar, Association of the Bar of the City of New York and the Commission’s Report on Admission to the Bar in New York in the twenty-first century). The study found that on the July 1992 bar exam, pass rates differed significantly based on the examinees’ race and ethnicity.¹⁴²See id. at 508–10.

Table 8. July 1992 New York Bar Exam Outcomes¹⁴³Id. at 509 (listing pass rates by race and ethnicity).

Nearly a decade later, “[i]n August 2001, the Florida Supreme Court ordered the Board of Bar Examiners to release racial data for first-time test takers on the February 2000 and July 2000 Florida bar exam.”¹⁴⁴Kidder, supra note 97, at 570. That study noted that “with a cutoff score of 131, 79.7% of . . . whites passed, compared to 65.6% of . . . people of color.”¹⁴⁵Id. The study further estimated what effect an increased cutoff score would have on pass rates, concluding that under a higher cutoff score, “68.5% of whites would pass, compared with 53.2% [of] people of color.”¹⁴⁶Id. Interestingly, “the Florida Board of Bar Examiners refused to release racial/ethnic data on the Florida bar exam in 2000.”¹⁴⁷Id. at 569 (discussing disparate impact of higher bar standards).

More recently, the Texas Board of Law Examiners was directed to “report to the legislature” on the passage rates for the July 2004 bar examination with “[t]he data to be aggregated by gender, ethnicity, and race.”¹⁴⁸Tex. Gov’t Code Ann. § 82.0291 (West 2021). This study found that the percentage of examinees passing was lower among communities of color as compared to White examinees.¹⁴⁹TX Passing Rates Report, supra note 74, at 5; see also Gender & Racial/Ethnic Group Analysis, supra note 74.

Table 9. Texas July 2004 Bar Passage Rates¹⁵⁰TX Passing Rates Report, supra note 74, at 5.

The Texas Board of Bar Examiners attributed these disparities to differences in entering credentials for law students.¹⁵¹See Gender & Racial/Ethnic Group Analysis, supra note 74. The Board supported its conclusion in two ways. First, it noted:

that the 8-point difference in mean LGPA between Whites and Blacks was equivalent to 0.78 standard deviation units. This was nearly identical to the difference (in standard deviation units) between these groups’ mean total scale scores. The size of the difference between Whites and Hispanics on LGPA also was very similar to the difference (in standard deviation units) between these groups in total scale scores. Asians were the only group that did not do quite as well on the bar exam as would be predicted on the basis of their LGPAs.¹⁵²Id.

Second, the Board created two models of bar passage rates accounting for the “applicant’s admissions credentials and law school grades.”¹⁵³Id. The first model included UGPA, LSAT, and LGPA, while the second model included those factors plus the applicant’s gender and racial/ethnic group.¹⁵⁴Id. The addition of gender and race/ethnicity improved the amount of variation explained by the first model by 0.6 percentage points (from 37.2% to 37.8%).¹⁵⁵Id. (describing regression models used).

There are several issues with the Board’s explanation of the results. First, it is problematic to connect pass rate to LGPA because it does not account for the fact that different law schools may have different grade curves.¹⁵⁶See, e.g., Nancy H. Kaufman, A Survey of Law School Grading Practices, 77 J. Legal Educ. 415, 420–21 (1994) (discussing the types of grade curves utilized at law schools). A 3.0 LGPA at School A may be an average LGPA (because the school’s grade curve mean is set to 3.0) while that same 3.0 at School B may be above average (because School B’s mean is set to 2.7). Thus, simply using LGPA is problematic because we do not know what that LGPA means in the context of that school.

Furthermore, in using its two models to whitewash the results, Texas misunderstands what an LSAT score represents—both an examinee’s academic ability and their race/ethnicity.¹⁵⁷See, e.g., Kidder, supra note 121, at 1074 & tbl.1 (2001) (finding that students of different races/ethnicities with equal academic accomplishments at the college-level have different LSAT scores). In essence, the model already includes a variable that captures an examinee’s race and ethnicity: the LSAT. Thus, the small improvement in the model by the addition of race or ethnicity is not an indication of the relative low importance of those factors, but rather an indication that race and ethnicity are related to both LSAT score and pass rate.

In 2019, the New York State Board of Law Examiners requested that researchers at the NCBE conduct a study to “investigate the impact of adoption of the Uniform Bar Examination in New York.”¹⁵⁸Nat’l Conf. of Bar Exam’rs, supra note 97, at 1; see also Press Release, N.Y. Ct. of Appeals, Impact of the Uniform Bar Examination in New York (Aug. 20, 2019), https://www.nybarexam.org/Docs/UBE_Report.pdf [perma.cc/CTU5-UJAQ] (announcing the release of the report). The study found that scores for all racial/ethnic groups tended to improve within two years of New York’s adoption of the UBE.¹⁵⁹See Nat’l Conf. of Bar Exam’rs, supra note 97, at 1, 4 (discussing how pass rates for racial groups changed after adoption of the UBE); see also Nat’l. Conf. of Bar Exam’rs, Rsch. Dep’t, Impact of Adoption of the Uniform Bar Examination in New York, 1, 83, https://www.nybarexam.org/UBEReport/NY%20UBE%20Adoption%20Part%202%20Study.pdf [https://perma.cc/9NZM-6Y9U] (discussing pass rates for racial and ethnic groups) Nonetheless, the “White group tended to have the highest average scores on the bar exam, followed by the Asian/Pacific Islander group or the Hispanic/Latino group . . . , and then the Black/African American group.”¹⁶⁰Nat’l Conf. of Bar Exam’rs, supra note 97, at 4. These score differences persisted even when “predictive” background characteristics, such as UGPA or LSAT score, improved or remained the same for examinees who identified as Asian/Pacific Islander, Black, and Hispanic, while those same characteristics remained constant or decreased for White examinees.¹⁶¹See id. at 3 (discussing changes in background characteristics). Thus, the gap in bar scores between White and non-White students grew even as the gaps in background characteristics shrank.

Finally, in 2020, California released a simulation report analyzing what would happen to pass rates for racial/ethnic groups (among other groups) were the score necessary to pass the bar (the “cut score”) decreased.¹⁶²See State Bar of Cal., Simulation of the Impact of Different Bar Exam Cut Scores on Bar Passage, by Gender, Race/Ethnicity, and Law School Type, at 2 (2020), https://www.calbar.ca.gov/Portals/0/documents/reports/CA-State-Bar-Exam-Cut-Score-Simulations-Analysis.pdf [https://perma.cc/5GVB-SQW5] (explaining the background as to the purpose of the study). Currently a score of 1440 is required to pass.¹⁶³Id. at 4. The report analyzed more than 85,000 examinees that collectively took the bar exam more than 140,000 times between 2009 and 2019.¹⁶⁴Id. at 2 (discussing the simulation data and methodology). At the current cut score, the study shows that 52% of White examinees achieve the requisite 1440 on the exam, while 24% of Black examinees, 36% of Latino examinees, and 40% of Asian examinees meet that same threshold.¹⁶⁵Id. at 8 tbl.4 (listing percentage pass rate at the current and proposed cut-scores). The study, reproduced as Table 10, concluded that pass rates for examinees of color would substantially increase were the score required to pass the bar exam reduced.¹⁶⁶Id.

Table 10. California 2020 Report Assessing Impact of Cut Score¹⁶⁷Id.

D. Avoiding the Obvious Conclusion

Our analysis of the empirical studies of the bar examination shows that of the studies undertaken, few demonstrate that an examinee’s probability of passing the bar examination is related to their race and ethnicity. Rachel Gregory has argued that, considering the history of minority exclusion from the bar, it is not a coincidence that modern academic selectivity in the bar exam excludes people of color.¹⁶⁸See Rachel L. Gregory, Florida’s Bar Exam: Ensuring Racial Disparity, Not Competence, 18 Geo. J. Legal Ethics 771, 773 (2005). Furthermore, Cecil Hunt has argued that the relative lack of investigation into racial disparities in the bar has been intentional, both because the remedy would be complex and difficult to implement, and because institutions do not want to appear to be engaging in discriminatory behavior.¹⁶⁹See Cecil J. Hunt II, Guests in Another’s House: An Analysis of Racially Disparate Bar Performance, 23 Fla. St. U. L. Rev. 721, 723–25 (1996).

IV. Potential Causes of Differential Bar Passage Rates

Several scholars have investigated the racial and ethnic disparities in bar passage rates and have speculated that cultural bias within the exam is a factor in such differences. This bias might manifest in the form of language barriers and interpretations, promotion of dominant values, equal experience assumptions, and subjective or flawed item selection.¹⁷⁰Christina Shu Jien Chong, Battling Biases: How Can Diverse Students Overcome Test Bias on the Multistate Bar Examination, 18 U. Md. L.J. Race, Religion, Gender & Class 31, 44 (2018). For example, a test taker may encounter a description of what seems to be a universal norm, but the concept is grounded in dominate White culture, such as who counts as a relative, or certain holiday traditions. Understanding that cultural norm is critical to answering the bar question correctly. Other scholars have noted that environmental factors may impact BIPOC students studying for the bar exam, including issues of isolation, self-efficacy, and access to resources.¹⁷¹Erin Lain, I Think I Can: How Self-Efficacy and Self-Regulation Impacts Black and Latinx Bar Examinees, 10 Ind. J.L. & Soc. Equal. 113 (2022), https://www.repository.law.indiana.edu/cgi/viewcontent.cgi?article=1134&context=ijlse [https://perma.cc/N3EE-QDTN]. This qualitative study examined attorneys of color who passed the bar exam on the first attempt versus on the second attempt. Id. For those who failed their first attempt, themes of isolation during studying and experiencing outside distractions played a significant role in their perceptions of why they did not pass. Id.

The environment experienced by a BIPOC test taker can substantially impact their ability to prepare for the exam.¹⁷²Hunt, supra note 169, at 770–86 (1996) (discussing a variety of factors that lead to environmental barriers experienced by students of color in law school). For example, throughout the test taker’s law school experience, they may have been bombarded by microaggressions about their ability to succeed in law school and beyond, and this environmental factor may have significantly impaired their self-efficacy.¹⁷³Meera E. Deo, Walter R. Allen, A.T. Panter, Charles Daye & Linda Wightman, Struggles & Support: Diversity in U.S. Law Schools, 23 Nat’l Black L.J. 71, 73–74 (2010) (discussing the environmental challenges that students of color face). Subjective grading may be another significant factor.¹⁷⁴See Milo Colton, What Is Wrong with the Texas Bar Exam? A Minority Report, 28 T. Marshall L. Rev. 53, 60 (2002) (discussing the results of the Florida Supreme Court’s Racial and Ethnic Bias Commission study of bar passage in Florida). Bar exam graders inevitably bring their own lens to the grading process.¹⁷⁵John M. Malouff & Einar B. Thorsteinsson, Bias in Grading: A Meta-Analysis of Experimental Research Findings, 60 Australian J. Educ. 245–256 (2016) (providing a meta-analysis looking at over 1,900 graders and the influence of bias in essay grading). The way in which essays are graded, grammar and syntax are evaluated, and words are analyzed is inevitably infused with the grader’s cultural positioning.¹⁷⁶Id.

In 1996, Hunt wrote a detailed analysis of possible factors contributing to disparities in bar exam pass rates.¹⁷⁷Hunt, supra note 169. While this article is nearly thirty years old, the factors may still be contributors to disparate pass rates.¹⁷⁸Id. at 769. Hunt notes that many speculate that differences in pass rates are due to a lack of academic preparation and ability, specifically as a result of poor schooling in the K-12 and undergraduate pipeline.¹⁷⁹Id. at 770. Yet this does not explain why those with similar predictive indicators—such as UGPAs, LGPAs, and LSAT scores—may still fail the bar exam.¹⁸⁰See Katherine A. Austin, Catherine Martin Christopher & Darby Dickerson, Will I Pass the Bar Exam?: Predicting Student Success Using LSAT Scores and Law School Performance, 45 Hofstra L. Rev. 753, 765-79 (2017) (finding that undergraduate GPA is not predictive of bar exam success, LSAT scores explained thirteen percent of bar exam performance, and first year and final law school GPA predict fifty-two percent of an individual’s bar exam performance). Hunt suggests that environmental factors within law school contribute to the disparities.¹⁸¹Id. at 770–86. Specifically, low expectations for students of color, a hostile environment where students of color are stigmatized and isolated, and a lack of academic support may all contribute to lower pass rates.¹⁸²Id. Although academic support programs are more prevalent in law schools today,¹⁸³Standard 309 of the ABA Standards and Rules of Procedure for Approval of Law School requires all law schools to provide academic support in order to give students a “reasonable opportunity” to join the profession. Am. Bar Ass’n, ABA Standards and Rules of Procedure for Approval of Law Schools 2020–2021, at 22 (Standard 309) (2020), https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/standards/2020-2021/2020-21-aba-standards-and-rules-for-approval-of-law-schools.pdf. extended programs that assist with bar exam preparation are not yet standard.¹⁸⁴See, e.g., id. at 22 (Standard 309) (listing standards requiring law schools to provide academic support in order to obtain a law degree, but silent on law schools providing support for passing the bar exam); Stephanie Francis Ward, More Law Schools Are Covering Bar Review Costs: Is it Improving Pass Rates?, Am. Bar Ass’n J. (Oct. 20, 2016, 8:30 AM), https://www.abajournal.com/news/article/more_law_schools_covering_bar_review_costs_is_it_improving_pass_rates [https://perma.cc/DQJ6-CE4D]; see also Taking the Bar Exam, Harvard L. Sch., https://hls.harvard.edu/dept/dos/taking-the-bar-exam/ [https://perma.cc/2TBR-8AKL]. These factors, coupled with the way in which the bar exam is written and graded as well as larger systemic issues of oppression,¹⁸⁵Chong, supra note 170, at 44–54. may be the primary causes of the bar passage gap.

V. Our Statistical Analysis

This study relies on publicly available data reported by ABA-accredited law schools as part of the ABA’s Standard 509 Required Disclosures between 2012 and 2019.¹⁸⁶This data is publicly available at two different website addresses. Section of Legal Education – ABA Required Disclosures, 509 Required Disclosures, Am. Bar Ass’n, http://www.abarequireddisclosures.org/Disclosure509.aspx (last visited Nov. 20, 2021); Statistics, Am. Bar Ass’n, https://www.americanbar.org/groups/legal_education/resources/statistics/ (last visited Nov. 20, 2021). Data as to bar passage by jurisdiction, entering class credentials, race, ethnicity, geographic location, and law school rank were combined to form the research dataset.¹⁸⁷To merge and append all the data together, inconsistencies in school names and the type of data reported had to be corrected. Incorrect data, such as percentages larger than 100, were corrected or removed from the dataset as appropriate. Using this dataset, we examined the relationships between these factors and first-time bar passage rates. To ensure both a large enough dataset and uniformity as to the meaning of the bar pass rate, we limited the data to schools from UBE jurisdictions during periods where that jurisdiction tested using the UBE.

The statistical analysis demonstrates that even after controlling for school characteristics—such as tier, entering class credentials, and median LSAT scores—higher proportions of students who identify as Black or Hispanic are significantly associated with lower bar pass rates.¹⁸⁸See discussion infra Section V.B. This decrease in pass rates cannot fully be explained by LSAT or UGPA quartiles for the entering class.¹⁸⁹Moreover, as noted infra, LSAT score is itself a variable that accounts for examinee race and ethnicity. See infra Section V.B (discussing the LSAT Model and Lagged LSAT Model compared, respectively, to the Race & Ethnicity Model and the Lagged Race & Ethnicity Model).

A. The Model

We utilize a fractional logistic regression analysis¹⁹⁰Fractional logistic models provide a good fit to data, like the bar passage rates, where the dependent variable must fall within the unit interval (between 0 and 1). See Jeffrey M. Wooldridge, Econometric Analysis of Cross Section and Panel Data 661–62 (2002) (discussing fractional logit regression). to understand the relationship between a school’s first-time bar pass rates in a particular jurisdiction and the proportion of students who identified as American Indian/Alaskan Native, Asian, Black, Hispanic, Native Hawaiian/Pacific Islander, Two or More Races, and White.¹⁹¹The analysis was run in STATA using a generalized linear model (“glm”) with a logit link to incorporate a curve. Our pass rate data has a binomial distribution because it is the number of successes divided by the number of trials.

Regression analysis identifies a curve of best fit that describes the relationship between the independent and dependent variables.¹⁹²See, e.g., Jeffrey S. Kinsler & Jeffrey Omar Usman, Law Schools, Bar Passage, and Under and Over-Performing Expectation, 36 Quinnipiac L. Rev. 183, 198 (2018). This curve does not intersect with each data point,¹⁹³See, e.g., Daniel J. McGarvey & Brett Marshall, Making Sense of Scientists and “Sound Science”: Truth and Consequences for Endangered Species in the Klamath Basin and Beyond, 32 Ecology L.Q. 73, 90 n.81 (2005) (discussing how data points will lie above and below the regression line). meaning that the relationship described by our curve of best fit—in this case between student characteristics and first-time bar pass rates—is not perfect.¹⁹⁴See id. When the curve of best fit does not touch a specific data point, we measure the error between the pass rate predicted by our curve (or model) and the actual pass rate for that school, jurisdiction, and year.¹⁹⁵See Damodar Gujarati, Econometrics By Example 13–14 (2011) (discussing R² as a measure of goodness of fit). Measuring those errors results in an “R-squared” value and indicates how well the line of best fit performs in describing the relationship.¹⁹⁶See id. The following regression models permit such errors to be correlated to each other if they stem from the same school. For example, if the model predicts a pass rate for School X in 2016 that is much lower than its actual pass rate from 2016, it assumes that its 2017 prediction is likely to be too low as well. Standard errors are clustered at the level of the school. Our full empirical model is defined in Equation 1 below.

Equation 1. Our Empirical Model

Pass Rate_ijt = α
+ Percent Minorities_it * β
+ Class Credentials_it * Δ
+ School Characteristics_it * ϕ
+ υ_t
+ ε_itj

Where i indicates a specific school, j indicates a specific UBE jurisdiction, and t indicates the exam year.

is the intercept and is the predicted pass rate for any school-jurisdiction in 2012 if all the other variables here had a value of 0.¹⁹⁷While that interpretation is not relevant here with variables that take on non-zero values, including it in the regression is the least restrictive option; removing the intercept would force our regression line through the origin, while retaining the intercept allows the regression line to lie as close to the data as possible.

Pass Rate is a school’s first-time bar pass rate in a UBE jurisdiction in a specified year where the school reported at least 70% of its graduates that year.¹⁹⁸Prior to 2019, law schools were required to “report [first-time] bar passage results from as many jurisdictions as necessary to account for at least 70 percent of its graduates each year, starting with the jurisdiction in which the highest number of graduates took the bar exam and proceeding in descending order of frequency.” Am. Bar Ass’n, ABA Standards and Rules of Procedure for Approval of Law Schools 2018–2019, at 24 (Standard 316) (2018), https://www.americanbar.org/content/dam/aba/publications/misc/legal_education/Standards/2018-2019ABAStandardsforApprovalofLawSchools/2018-2019-aba-standards-rules-approval-law-schools-final.pdf; cf. Am. Bar Ass’n, ABA Standards and Rules of Procedure for Approval of Law Schools 2019–2020, at 24 (Standard 316) (2019), https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/standards/2019-2020/2019-2020-aba-standards-and-rules-of-procedure.pdf (“[Revised Standard 316 is] [a]t least 75 percent of a law school’s graduates in a calendar year who sat for a bar examination must have passed a bar examination administered within two years of their date of graduation.”).
Percent Minority is a set of variables representing the percent of a specified school’s entering class who have self-identified as members of a specific race or ethnicity in that year. We also include a control, the percent of the entering class identified as Non-Resident Aliens. Though this is not a race or ethnicity, it is important to control for this group of students so that regression results can be compared to White students exclusively.

is a set of regression coefficients that tells us what happens to the pass rate when increasing the proportion of enrolled students who identify with various races or ethnicities. also includes a regression coefficient for the proportion of Non-Resident Aliens.

Class Credentials includes the entering class’s median LSAT for each school-year observation.

is a set of regression coefficients that tells us how pass rate changes when any median LSAT increases by one point.

School Characteristics includes the school’s region and tier.
is a set of regression coefficients that tells us how pass rate changes with changes to region and increases in tier.

is a time-fixed effect that controls for any significant changes in pass rate in a given year that affected all law schools and jurisdictions. For example, this would account for any years that had unusually high or low pass rates across the sample. These results are suppressed from the table and available from authors upon request.

is the error term described above. This is the difference between the jurisdiction-level pass rate for a specific school in a specific year and the model’s prediction of that value.

Our analysis is based on several models, of increasing sophistication, examining the relationship between race/ethnicity and first-time pass rate.¹⁹⁹For the remainder of this section, “first-time pass rate” should be understood to refer to school-jurisdiction level pass rates in a given year. “School-jurisdiction level” means that the first-time bar passage rate is the rate reported by an ABA accredited law school in a UBE jurisdiction. For example, the reported first-time pass rate for University of Connecticut School of Law alumni taking the bar in New York in 2017 is a school-jurisdiction level pass rate for 2017. In our first model (the “Base Model”), first-time bar passage rates²⁰⁰Section of Legal Education – ABA Required Disclosures, 509 Required Disclosures, supra note 186 (providing data for first-time bar passage rates for accredited law schools from 2014 to 2016); Statistics, supra note 186 (providing data from 2017 to 2019). Data was restricted to jurisdictions that used the UBE. If a jurisdiction started (or stopped) using the UBE, data was included only for those years that the jurisdiction used the UBE. The same logic applies to schools that opened or closed and gained or lost ABA accreditation. Law schools reported these first-time pass rates to the ABA for each of the jurisdictions where the largest number of their graduates took the exam, up until each school had accounted for at least 70% of its graduates from that year. See supra note 198. for a school-jurisdiction in a given year are regressed on school characteristics (geographic region and tier) from that same year.²⁰¹See infra Table 11. The Base Model does not account for the variables we seek to understand—race, ethnic identity, and entering credentials—but it does provide a baseline to which we can compare other models that include the desired variables.

Our second model (the “LSAT Model”) adds a variable for median LSAT to the Base Model to better understand the effect of entering credentials on bar exam pass rates. Our third model (the “Race & Ethnicity Model”) instead adds to the Base Model the proportion of students who identify with various racial, ethnic, and non-resident alien categories to better understand the relationship between the proportion of minority students and the pass rate. Finally, our fourth model (the “Full Model”) includes all variables studied thus far (race, ethnicity, non-resident alien, median LSAT, geographic region, and law school tier).

These four models (Base Model, LSAT Model, Race & Ethnicity Model, and Full Model) compare first-time pass rates from each reported jurisdiction in a specified year to school characteristics (race, ethnicity, LSAT, geographic region, and tier) from that same year. Because school-level characteristics like entering credentials and racial/ethnic makeup may change over time, we performed regressions on a second set of models (Lagged Base Model, Lagged LSAT Model, Lagged Race & Ethnicity Model, and Lagged Full Model) that compare a school’s characteristics in a given year to the pass rate three years later.²⁰²See infra Table 12. For example, the enrollment characteristics in year 2013 were compared to the first-time bar passage rate in 2016.

Because the ABA-required disclosures report data in the aggregate (at the level of a school in a given year),²⁰³See, e.g., Bar Passage Outcomes, Am. Bar Ass’n, http://www.abarequireddisclosures.org/BarPassageOutcomes.aspx (last visited Nov. 20, 2021) (providing drop down menus that allow access to collected data on a given school and year). their data are insufficient to follow individual students over time. As a result, our model assumes that most exam-takers in a given year (e.g., 2016) graduated from the same law school at which they enrolled three years earlier (e.g., in 2013). Though this method is not perfect, it may offer an improvement over the models in Table 11, infra, which ignore possible changes in student demographics over time. The results of the lagged analysis are in Table 12, infra.

B. Results: Race Is a Statistically Significant Factor in UBE Bar Passage Rates

Table 11 presents the exponentiated coefficients²⁰⁴See Wooldridge, supra note 190, at 662 (discussing the fractional model as a logistic function). To interpret results about pass rates, then, we must undo that log by exponentiating. from our regression analysis from each of the models. Certain coefficients have asterisks next to them. A single asterisk (*) means that the model is 90% confident that the relationship between that independent variable and pass rate exists and is therefore different from 0; two asterisks (**) indicate that the model is 95% confident that the relationship exists; and three asterisks (***) indicate that the model is 99% confident. We draw conclusions only from coefficients marked with asterisks because of these very high levels of confidence.

The Base Model looks at the relationship between a school-jurisdiction pass rate in a given year and the school’s geographic location (Midwest, Northeast, or West) and Tier (1, 2, or 3) in the same year. Regressing on the Base Model finds that being in Tier 1, 2, or 3 is positively correlated with bar passage rates at a 99% confidence level.²⁰⁵See infra Table 11. The Base Model accounts for 34.6% of the variation in school-jurisdiction bar passage rates.²⁰⁶Id. (listing the McFadden’s Pseudo R² value as 0.3462).

The LSAT Model adds the median LSAT score²⁰⁷We only use median LSAT score because including more than one LSAT quartile results in significant multicollinearity with Variance Inflation Factor (VIF) values in the range of 16,000 to 48,000. Though there is no theoretically derived threshold value for VIF, a common approach is to remove variables that have a VIF larger than 10. The very high VIF values here indicate that entering LSAT quartiles are highly correlated with each other. Similarly, entering LSAT quartiles are highly correlated with entering GPA quartiles, which is why entering GPA quartiles are excluded in models here. See Jeffrey M. Wooldridge, Introductory Econometrics: A Modern Approach 98 (5th ed. 2012) (discussing the variance inflation factor). Models using first- or third-quartile LSAT score, using all three quartiles, and models including entering GPA quartiles all lead to similar results. These other specifications are available from the authors upon request. for the entering class to the Base Model. The coefficient from the median LSAT indicates that a one-point increase in an entering class’s median LSAT, keeping all other school characteristics unchanged, is associated with a 9.27 percentage point increase in the pass rates.²⁰⁸See infra Table 11. This result is significant at the 99% confidence level.²⁰⁹Social scientists ordinarily use a p-value of 0.05 (i.e., a result has a one in twenty chance of being due to random variation and is used as a measure of statistical significance). See, e.g., Maxwell & Delany, supra note 10, at 47. That our results are statistically significant at a more stringent p-value demonstrates the robustness of those results. Additionally, adding a measure of median entering credentials increases the explanatory power of the model. Explanatory power is indicated by a Coefficient of Determination statistic, which measures how much of the variation in the dependent variable is explained by the regression analysis. Because the dependent variable in this case—pass rate—requires fractional logistic regression analysis, the Coefficient of Determination used here is McFadden’s Pseudo R-square. A value near 1 means that the model explains nearly 100% of the changes in pass rate across school-jurisdictions-years, and a value near 0 means that the model cannot explain any of the variation in pass rates. The bottom row of Table 11 shows that adding median LSAT to the Base Model increases the Pseudo R-square from 34.6% to 35.5%, an increase of 2.54% in explaining a school-jurisdiction’s pass rate in a year.²¹⁰See infra Table 11.

The Race & Ethnicity Model adds the percent of the entering class that identified with each racial or ethnic category to the Base Model instead of median LSAT. This model examines the relationship between race/ethnicity and first-time bar passage results by jurisdiction, using the portion of White students as the omitted reference category.²¹¹In regression analysis, with categories like race, one category must be excluded. It is common practice to exclude the largest category. An independent variable cannot be included in regression if it can be perfectly predicted from others, which is the case when all percentages add to 100%, because the estimation strategy cannot separate the effect of one from the others. See Wooldridge, supra note 207, at 84–86 (discussing the requirement that multilinear regression analysis lack perfect collinearity). The analysis reveals a negative relationship between first-time pass rate and the proportion of students who identify as Black, Hispanic, Two or More Races, or Unknown Race.²¹²See infra Table 11. For example, as the proportion of students who identify as Black increases by 1 percentage point, all school characteristics held equal, the pass rate for that school-jurisdiction-year is predicted to be only 98.41% of what it would have been without that increase in Black students, a 1.59 percentage point reduction. This relationship is statistically significant at the 99% level of confidence. The explanatory power of the Race & Ethnicity Model, as measured by McFadden’s Pseudo R², increased by 2.17% compared to the Base Model.²¹³See id. This is similar in size to the improvement that resulted from adding median LSAT to the Base Model (2.54%).²¹⁴See infra Table 11.

Finally, the Full Model adds both entering credentials, as measured by median LSAT, and the proportion of students identifying with each racial or ethnic group. For the first time, this model reveals the relationship between pass rate and the proportion of minority students after controlling for median LSAT. This is especially important if the LSAT is correlated with racial or ethnic identity, as previous research indicates.²¹⁵See, e.g., Kidder, supra note 121, 1074 & tbl.1 (2001) (finding that students of different races/ethnicities with equal academic accomplishments at the college-level have different LSAT scores). If they are correlated, both must be included in the regression to avoid omitted variable bias, and thus to estimate accurate coefficients. Consider the context here. LSAT scores tend to be lower for Black students.²¹⁶See, e.g., Summary Bar Pass Data: Race, Ethnicity, and Gender: 2020 and 2021 Bar Passage Questionnaire, supra note 67 (providing national summary statistics on bar passage rates by race and ethnicity). If we try to explain pass rates using the proportion of Black students and do not include LSAT, the coefficient for the proportion of Black students would capture both possible effects: a negative impact on pass rates due to lower LSAT scores and any racial or ethnic bias that exists in the bar exam after controlling for the LSAT. In this case, the coefficient would be overstated because both possible effects work to lower pass rates. Similarly, a model that includes the LSAT but excludes the proportion of Black students will have a biased coefficient for the LSAT.

The Full Model controls for both effects. A one-point increase in median LSAT score is associated with a 9.5 percentage point increase in bar passage rates.²¹⁷See infra Table 11 (listing the coefficient for LSAT 50th Percentile under the full model as 1.0951). A one percentage point increase in the proportion of students who identify as Black is associated with a 1.06 percentage point decrease in bar passage rates after controlling for median LSAT score.²¹⁸See infra Table 11 (listing the coefficient for percentage Black under the full model as 0. 9894—where 1 – 0.9894 = 0.0106 or 1.06%). The Full Model also reveals negative relationships for the proportion of students who identify as Two or More Races and Unknown Race.²¹⁹See infra Table 11 (listing the coefficients for percentage Two or More Races, percentage Non-Resident Alien, and percentage Unknown Race as, respectively, 0.9858, 0.9680, and 0.9890—where coefficients below 1.0 correspond to decreases in pass rates as the related variable increases). The Full Model provides a 4.16% increase in explanatory power compared to the Base Model.²²⁰See infra Table 11 (listing the Full Model’s McFadden’s Pseudo R² as 0.3606).

The Full Model indicates a negative relationship between a school-jurisdiction’s pass rate in a year and the proportion of several minority student groups, even after controlling for entering credentials. Comparing the Pseudo R² values from the Race & Ethnicity Model and the Full Model shows that adding the LSAT into a model of school characteristics and racial and ethnic proportions increased the explanatory power by 1.95%.²²¹See infra Table 11 (listing the Race & Ethnicity Model’s McFadden’s Psuedo R² as 0.3537 and the Full Model’s McFadden’s Psuedo R² as 0.3606, and thus seeing an increase of 1.95% (0.3606/0.3537) by adding LSAT into the Full Model). Comparing the LSAT Model to the Full Model shows that adding racial and ethnic proportions to a model including school characteristics and the LSAT increases the explanatory power by 1.58%.²²²See infra Table 11 (listing the LSAT Model’s McFadden’s Psuedo R² as 0.3550 and the Full Model’s McFadden’s Psuedo R² as 0.3606, and thus seeing an increase of 1.58% (0.3606/0.3550) by adding LSAT into the Full Model). Thus, this sample indicates that it is nearly equally important to include both a measure of entering credentials and measures of minority student proportions.

Table 11. School-Jurisdiction Pass Rate, Not Lagged

The models in Table 12, infra, are more realistic predictors of a school’s bar passage rates because they compare a school’s first-time pass rate in a jurisdiction to the entering class most likely to have taken that bar examination.²²³See infra Table 12. The initial models described in Table 11 compare first-time bar passage outcomes in a given year with school characteristics from the same year, while the models described in Table 12 compare first-time bar passage outcomes in a given year to the school characteristics from the year of admission. See supra Table 11­­­; infra Table 12. In these models, the Pseudo R-squared has increased to reflect that they explain nearly half the variation in pass rates. As noted by the increases in the Pseudo R-squared values, this realism is evident in models that capture more of the variation in first-time pass rates.

Table 12. School-Jurisdiction Pass Rate, Enrollment Data Lagged

Table 12 presents results from the lagged analyses that correspond to the models described in Table 11. The important difference is that the models in Table 12 include independent variables (e.g., race and ethnicity) with values collected three years before the relevant pass rate. In the first model (the “Lagged Base Model”), first-time bar passage rates²²⁴First-time bar passage rates for accredited law schools were collected from the ABA’s Standard 509 Required Disclosures from 2014 to 2016 and from the ABA’s Statistics from 2017 to 2019. See Section of Legal Education – ABA Required Disclosures, Am. Bar Ass’n, https://www.abarequireddisclosures.org/Disclosure509.aspx (last visited Feb. 21, 2022) (to access 509 required disclosures click on “509 Required Disclosures” and then input a year and school to receive school/year specific data or year and section to receive year-level data). Data was restricted to jurisdictions that used the UBE. If a jurisdiction started (or stopped) using the UBE, data was included only for those years that the jurisdiction used the UBE. The same logic applies to schools that opened or closed and gained or lost ABA accreditation. Law schools reported these first-time pass rates to the ABA for each of the jurisdictions where the largest number of their graduates took the exam, up until each school had accounted for at least 70% of its graduates from that year. for a school-jurisdiction in a given year are regressed on school characteristics (geographic region and tier) from three years before. The Lagged Base Model does not account for the variables we seek to understand—race, ethnic identity, and entering credentials—but it does provide a baseline to which we can compare other models that include the desired variables.

The Lagged Base Model, which includes only school characteristics and the exam year, explains 48.85% of the variation in pass rates throughout the sample, as noted by the Pseudo R-squared value.²²⁵See supra Table 12 (listing the Lagged Full Model’s McFadden’s Pseudo R² as 0.4885). This explanatory power is improved when median entering LSAT from the entering class three years before is added (in the Lagged LSAT Model).²²⁶See id. (listing the Lagged Base Model’s McFadden’s Psuedo R² as 0.4885 and the Lagged LSAT Model’s McFadden’s Psuedo R² as 0.5000, and thus seeing an increase of 1.95% (0.3606/0.3537) by adding LSAT into the Full Model). Controlling for median LSAT increases the model’s explanatory power by 2.35% (from 48.85% to 50% of the variability in pass rates explained).²²⁷See id. Again, a one-point increase in median LSAT score is associated with an increase in pass rates at the 99% level of confidence.²²⁸See id. (listing the Lagged LSAT Model’s coefficient for LSAT 50th Percentile as 1.1134, where coefficients above 1.0 entail that increases in the related variable result in an increase in the bar passage rate). While the Lagged LSAT Model adds median LSAT score for the class that entered three years before the bar exam to the Lagged Base Model, the Lagged Race & Ethnicity Model instead adds the proportions of students who identify with each racial and ethnic category as well as the proportion of non-resident alien students, also using the entering class three years before the bar exam. Adding the racial and ethnic category proportions increase the explanatory power by 0.78% (from 48.85% to 49.23% of the variation explained). Both the Lagged LSAT Model and the Lagged Race & Ethnicity Model suffer from omitted variable bias because they do not control for the relationship between median LSAT and the proportion of minority students.²²⁹See Wooldridge, supra note 190, at 61–63 (discussing omitted variable bias). Moreover, in contrast to the models in Table 11, comparing these lagged models indicates that controlling for entering credentials is more important to increasing the Pseudo R-squared than controlling for racial and ethnic identities.

The Lagged Full Model includes both LSAT and racial and ethnic proportions. This Model is, unsurprisingly, an improvement from the Lagged Base Model with a 2.99% increase in explanatory power (from 48.85% to 50.31% of the variability in pass rates explained). Importantly, the Lagged Full Model removes the bias generated by omitting either LSAT or the racial and ethnic categories. After controlling for entering credentials, the Lagged Full Model indicates that increasing the proportion of Black students by one percentage point, holding median LSAT and other school characteristics constant, is associated with a 0.71 percentage point decrease in the pass rate.²³⁰See supra Table 12 (listing the coefficient for percentage Black in the Lagged Full Model as 0.9929). This is significant at the 99% level of confidence. Similarly, the Lagged Full Model shows negative relationships between pass rates and the proportions of students who identified as Asian and Race Unknown in schools with similar characteristics and the same median LSAT scores.²³¹See supra Table 12 (listing the coefficient for percentage Asian, percentage Non-Resident Alien, and percentage Race Unknown as, respectively, 0.9788, 0.9680, and 0.9890).

Unlike previous empirical work examining pass rates, our analysis reproduces statistical results for the reader and provides comparisons between models that do and do not control for race/ethnicity in addition to background characteristics such as LSAT. Additionally, our work highlights the statistical significance of the relationship between pass rates and race/ethnicity after controlling for LSAT. This stands in contrast to previous studies that find statistical significance but then ignore these findings when arguing that the effect is not large in size. One of the advantages of careful statistical analysis is the ability to uncover even small effects with a stated level of confidence.

While the focus on statistical significance and reproduction of results is a clear improvement on existing empirical work, it is insufficient to determine whether the bar exam contributes to the lower pass rates associated with larger proportions of students from communities of color. The ideal analysis must be performed at the level of a test taker. Instead of gathering data about an incoming class, a single examinee must be considered. That level of analysis will be able to determine the probability that a particular test taker will pass the bar, given their own LSAT, race and ethnicity, and other characteristics. Such analysis, then, will reveal whether race and ethnicity have a statistically significant effect on the probability of passing the bar.

V. Discussion: The Need for Data

The legal profession is one of the least diverse professions in the United States.²³²See Deborah L. Rhode, Law Is the Least Diverse Profession in the Nation. And Lawyers Aren’t Doing Enough to Change That, Wash. Post (May 27, 2015), https://www.washingtonpost.com/posteverything/wp/2015/05/27/law-is-the-least-diverse-profession-in-the-nation-and-lawyers-arent-doing-enough-to-change-that/ [https://perma.cc/B92F-KS68] (discussing the lack of diversity in the legal profession). As we demonstrate above, potential bias in the bar exam may be a factor contributing to this lack of diversity. While students of color face obstacles throughout their tenure in primary and secondary education,²³³See Jessika H. Bottiani, Catherine P. Bradshaw & Tamar Mendelson, A Multilevel Examination of Racial Disparities in High School Discipline: Black and White Adolescents’ Perceived Equity, School Belonging, and Adjustment Problems, 109 J. Educ. Psych. 532–45 (2017) (discussing disparate disciplinary action towards Black students in high school); David M. Merolla, Completing the Educational Career: High School Graduation, Four-Year College Enrollment, and Bachelor’s Degree Completion Among Black, Hispanic, and White Students, 4 Socio. Race & Ethnicity 281–97 (2017) (exploring differing educational trajectories by race and ethnicity in the high school and college setting); Tachelle Banks & Jennifer Dohy, Mitigating Barriers to Persistence: A Review of Efforts to Improve Retention and Graduation Rates for Students of Color in Higher Education, 9 Higher Educ. Stud. 118, 119–21 (2019) (exploring the various factors contributing to disparate graduation rates for students of color in college). our analysis shows that those who have made it through law school may be weeded out from the profession at the last possible moment. At the same time, young people from communities of color watch as their friends, siblings, parents, and mentors accumulate colossal debt to attend law school²³⁴See, e.g., Valerie Fontenot, Disparities in Student Loans: How Did We Get Here and What Can
We Do?, Am. Bar Ass’n (July 16, 2019), https://www.americanbar.org/groups/litigation/committees/diversity-inclusion/articles/2019/summer2019-disparities-in-student-loans/ [https://perma.cc/2FKJ-42GH] (discussing disparities in student loan debt by race). and then struggle to pass the bar exam. In that context, a legal career becomes too risky a venture to undertake, and students who would otherwise seek such a career focus their educational and professional efforts elsewhere.²³⁵See Legal Skills Prof, Is Law School a Riskier Investment for Minority Students?, Legal Skills Prof Blog (July 5, 2012), https://lawprofessors.typepad.com/legal_skills/2012/07/is-law-school-a-riskier-investment-for-minority-students.html [https://perma.cc/UY9V-AA2Q] (discussing Professor Deborah Jones Merritt’s analysis indicating that BIPOC law graduates experience worse outcomes than White law graduates in terms of carrying more debt and passing the bar at lower rates).

Additionally, bar examiners’ characterizations of the bar exam as a test of minimum competence²³⁶See, e.g., Deborah Jones Merritt, Validity, Competence, and the Bar Exam, AALS News (Spring 2017), https://www.aals.org/about/publications/newsletters/aals-news-spring-2017/faculty-perspectives/ [https://perma.cc/MJM7-GQSK] (“Bar examiners tell us that the exam assesses ‘minimum competence to practice law’ . . . .”). perpetuates racial and ethnic stereotypes given the lower pass rates of BIPOC examinees that stem from potential exam biases. Calling the bar exam a test of “minimum” competence, when there are evidenced disparities and so many BIPOC test takers fail, exacerbates an already fraught situation. While the exam is supposedly a minimally invasive final hurdle to becoming a lawyer, it instead, as noted above, disenfranchises racial and ethnic groups. By reinforcing the legal profession’s lack of racial and ethnic represen­tation through a bar exam that exposes greater percentages of BIPOC examinees than White examinees to the stigma and costs of failing, the bar exam feeds the racial and ethnic disparities prevalent in our society today.

So, what should we do? First, as we have noted, our study can only demonstrate that the UBE may be racially and ethnically biased. While we have accounted for relevant factors that could mask the true relationship between race/ethnicity and bar passage, our analysis is still at too high a level of data (the level of the school, not the student) to

precisely determine whether, and to what extent, race and ethnicity matter for bar passage rates. Determining the relationship between race/ethnicity and bar passage with certainty would require examining data at the individual student level. Our next step as a profession and as researchers must be to analyze individual student bar passage rates (both first-time and ultimate). We must take into account the factors we have seen, and expect to see, influencing bar passage rates—where those factors are properly normalized such that we compare apples to apples.²³⁷For example, we expected UGPA to be a relevant predictor of bar passage. But examinees come from a wide range of schools with differing grade curves and exclusivity. Moreover, one’s major at a university also likely matters—a UGPA of 3.8 with a Chemical Engineering major likely provides different information than a UGPA of 3.8 with an English major. Doing so will enable us to determine whether the difference we see at the school level is due to differences “unrelated” to race and ethnicity or whether the UBE is genuinely biased against students from communities of color.

Once the true relationship between bar passage rates and race and ethnicity is established, we can begin to develop policies that will support and foster the success of BIPOC students. For example, we can better understand the impact that test configuration, grading, programming, demographics, and outside law school support may have in positively impacting racial and ethnic disparities. In so doing, we will move closer to our goal of having a legal profession that truly represents the people in our country.

To this end, we propose a wide-scale study that examines bar passage rates of students who take the bar examination in UBE jurisdictions. We have already received Institutional Review Board approval for this study. In the future study, we will ask schools to provide five years of data on student bar passage information, student entering credentials, and markers of student success in law school (e.g., LGPA and rank). The information provided by each school will be blinded so that the data cannot identify the students. In addition, the study will be blinded as to the schools that participate; results will be shared in a way that does not indicate which schools took part in the study.

We have begun reaching out to law schools to participate in this study and are also working to find institutional sponsors (like the ABA, LSAC, National Association for Law Placement, Society for American Law Teachers). We are encouraged by the fact that the law school deans we have spoken with have universally agreed that this study is important and needed. Simultaneously, we are discouraged that, despite the recognition of the need for and importance of the study, very few deans have been willing, thus far, to provide the data. We hope that, through continued discussion and negotiation with law school deans, we will be able to acquire sufficient data to continue our work, although this is not guaranteed.

Conducting empirical research on this issue is imperative to narrowing the bar pass gap amongst racial and ethnic groups. And in support of that research and the public interest, we call on states to begin regularly releasing bar passage data by race and ethnicity. This type of scientific exploration into the bar passage disparities is essential to improving access and representation in the legal profession. Without this type of in-depth analysis, the legal profession is consciously ignoring a significant ethical problem that continues to perpetuate inequalities—a problem that the legal system is in place to protect against. The current study, and proposed future study, will aid in our efforts of making the legal profession accessible to all.

Appendix. Exam Statistics Reporting by Jurisdiction