Wednesday, November 13, 2013

How Bar Passage Rates Should Be Reported

I have decided, for no particular reason, to post some modest suggestions for improving the world. This is the first.

Law schools compete for students, even more fiercely than usual in the current environment of declining enrollments. Most prospective students plan to be lawyers, being a lawyer requires, in most states, that you pass the bar exam, so an important metric of a law school's quality, arguably the most important, is the fraction of its graduates who pass the bar.

Whether a student passes the bar depends in part on the education he got from his law school, in part on how smart and hard working he is—characteristics for which LSAT score and undergraduate grade point average provide at least an approximate measure. Thus a law school can increase its bar passage rate, quality of education held constant, by admitting students with higher LSAT's and GPA's. Since increasing that rate will result in more students applying to the school, there is an obvious incentive to do so. 

One way of getting better qualified students to go to your school is by offering them scholarships. The result is a system where the ablest students, the ones most likely to end up as high paid partners in elite law firms, get an education subsidized by the tuition payments of the least able students, the ones most likely to end up with a large debt and no job. It is a bizarre result by almost any standard, and particularly anomalous given that the culture of most law schools is predominantly left of center and egalitarian.

It is also the result of a logical mistake, a fallacy of composition. Getting a student who is almost certain to pass the bar to come to my law school raises the school's bar passage rate, but not by raising the odds that other students will pass the bar. The information relevant to a prospective student is not what the average bar passage rate at a school is but what the bar passage rate is for students like him.

I  see no reason why that information cannot be generated and made available. Let each law school run a regression fitting bar passage rate for, say, the past three graduating classes to LSAT, showing how likely a student with any given LSAT is to pass the bar. Do the same thing with GPA. Publish the results. It might turn out that the student who is readily accepted at SCU but can barely get into Stanford will have a better chance of passing the bar if he goes to the former school, even though the latter has a higher average bar passage rate. The question is not which school is better but which school is better for which student.

If that information is made available and students choose to base their decisions on it, the perverse incentive currently faced by law schools largely disappears. They can still increase applications by doing a better job of educating their students to pass the bar. But stacking the deck by competing to get better qualified students to come will no longer work.

There is a second advantage to this change from the standpoint of those law professors, probably a sizable majority, who are in favor of affirmative action—meaning, in practice, lower admission standards for black students. There are two arguments against the practice that ought to concern them. One, from the standpoint of the school's selfish interest, is that lower standards will mean a lower bar passage rate, making it harder to get students to apply. Another, from the standpoint of the people the policy is supposed to help, is that admitting underqualified students may mean setting them up for failure. The same student who would get a good education at SCU, where he is about as well qualified as his fellow students, might have a much harder time at Stanford, where he is near the bottom of the distribution, and similarly all along the line. It is at least arguable, may well be true, that the real effect of affirmative action by law schools is to reduce, not increase, the number of black lawyers. 

My proposal solves both problems. Law schools that choose to admit less qualified students no longer lose reputation and applications as a result, provided they can do a good job of educating the students they admit. And students, black and white, will be in a position to decide for themselves which school will give them the best chance of passing the bar.


jimbino said...

Another way for a law school to gain would be to allow high-scoring students to do self-study and sit for the bar when they felt ready. That would improve the student/teacher ratio.

I skipped a lot of my law-school classes, especially those taught or populated by students ignorant of STEM and economics. But I got my degree back when law school was virtually free; it would be hard to put up with it while paying through the nose, as students have to do nowadays.

A law school realizes, of course, that great students add greatly to the classroom experience and to to the endless beer parties. That being the case, they properly pay well to subsidize them by merit scholarships.

Gordon said...

I suspect one problem with your proposal is that law professors would have to "teach to the exam", something they are, on the whole, unwilling to do, if they were going to improve the chances of less able students to pass the bar.

Everyone I knew in my class took a bar preparation exam. NY tests on about eight topics out of twenty or more. Without a bar prep course tailored to what you might face on the exam, it would be rather challenging for most to pass. On the other hand, the NJ exam is also about eight topics, but you already know what they will be: essentially all first year topics plus evidence. As a result, anyone planning to take both NY and NJ takes a prep course for NY and crosses his fingers for the NJ exam. (Barbri tells you that if you pass NY, you have an 80% chance of passing NJ as well.) But even the few students I knew who took the NJ exam only, also took a bar prep course.

And here is the reason (somewhat tongue-in-cheek, but only somewhat) that I formulated after my first year: what they teach in class does not prepare you for the final; studying for finals does not prepare you for the bar exam; and passing bar exam does prepare you to be a lawyer. Each of these is just a hurdle to try to thin the herd.

lelnet said...

One must, of course, also factor in the typical post-bar-passage career of the ex-student, which immensely complicates your assumptions...possibly even to the point of refuting your hypothesis entirely, because you appear to be entirely ignoring the question of signalling. If signalling were an insignificant or minor component of the value of a law degree from a particular school, this elision would be of little consequence. But that is clearly not the case in the present market.

Even if your assumption is correct, vis a vis the marginal student's differential likelihood of bar-exam passage at SCU vs Stanford, one must also consider the career options. I would contend that a rational student, even knowing that SCU would virtually guarantee that he would pass the bar while Stanford would make it likely that he would either fail the first time or else only barely pass, might very reasonably choose Stanford anyway, owing simply to the greater signalling value of the Stanford diploma, which will continue dispensing substantial utility for his entire career, while "did you pass the bar exam" is merely a pro-forma question with a binary answer, unlikely to come up after his first round of job interviews.

An arch-rationalist might argue, of course, that the market over-values that signalling. Fair enough. But the student will not, after he graduates and passes the bar, be seeking employment in a hypothetical, more-rational market designed by someone who knows what the proper value of a Stanford-signal ought to be, but in the market that actually exists in our world, irrational though it sometimes can be.

gotlucky said...

Professor Friedman:

"Thus a law school can increase its bar passage rate, quality of education held constant, by admitting students with higher LSAT's and GPA's."

Suppose that the quality of education is not held constant. Further suppose that students have no low cost way of determining the quality of education at a particular school.

After all, that is precisely the manner in which you responded to Vijay in your recent Austrian thread. The purpose of your thought experiment was to get at the principle of the matter, and for that you set up a scenario in a particular way so that you do so, even if this particular scenario does *not* actually exist in the real world.

Such was Vijay's thought experiment. If there are two identical goods, consumers will prefer the cheaper of the two ceteris paribus. You responded with a scenario where the goods were not identical. Your scenario factored the perceived price of the good into the good itself, meaning that the goods were no longer identical.

So the real question is, why can you use thought experiments in order to demonstrate a point, but Austrian school economists cannot?

For that matter, why could your father use thought experiments in order to demonstrate a point (a popular example being the Ford Pinto video on youtube), but when Austrian economists use thought experiments to demonstrate a point they are apparently using a flawed methodology?

What do you hope to learn about the real world when you use thought experiments in order to deduce certain conclusions? Especially since your thought experiment regarding the law schools is not reflective of the real world?

maurile said...

You'd also want to control for which states a school's graduates tend to sit for the bar exam in. (Stanford might have a higher bar passage rate than SCU even if the students are prepared identically if SCU grads mainly sit for the California bar while Stanford grads often go to New York or D.C., if California exam is harder.)

Ultimately, though, I don't think prospective law students should care much about a school's bar passage rate. In my experience, preparing for the bar exam was almost completely independent from going to law school. Indeed, given what law school tuition costs, I would have been rather disappointed if my law school had acted as a bar-prep course (which should be much cheaper).