King Tut, Statistics, and a Pet Peeve

I recently came across a National Geographic article describing the results from DNA analysis of a number of mummies, including King Tut. It was an interesting article, but there was one thing in it that annoyed me. In describing their results, the author said that the DNA analysis showed that there was a 99.9% probability of a particular relationship between two of the mummies.

That statement was false. I know it was false because that sort of analysis cannot produce that sort of result. Like many other people who use statistics without understanding it, the author was confusing the information the statistical analysis produced with the information he wanted it to produce.

A confidence result in classical statistics tells you how likely it is that you would get the result you got if the assumption you were testing was false—more precisely, if a particular alternative assumption, called the null hypothesis, was true. His 99.9% means that if the null hypothesis (presumably that the two mummies were not closely related—the article doesn't say) was true there is no more than a .1% chance that the genetic evidence that they were related would be as good as it is.

Unfortunately for the author of that article and many others, the probability of getting their result if their assumption is false is not the same thing as the probability that their assumption is false, given that they got their result. The latter is what they want, and what the assertion of a 99.9% probability for the relationship claimed. But it wasn't what they got.

To see the difference, consider a much simpler experiment. I pull a coin out of my pocket without looking at it. My theory is that it is a two headed coin; the null hypothesis is that it is a fair coin.

I flip it twice and it comes up heads both times. If it is a fair coin, the probability of that outcome is only 25%. If it is a two headed coin, it's 100%. If the probability of the result given the null assumption was the same thing as the probability of the null assumption given the result, that would mean that the odds were now three to one—75% probability—that the coin was double headed. I don't think so.

Readers interested in what it takes to actually generate a probability estimate for an assumption being true are invited to read up on Bayesian probability.

After writing this post, I discovered that I had mentioned the same point some time back in a different context.

Is Obama a Christian? Is Palin?

There has been a lot of coverage of the fact that, according to at least one report, about 18% of the population believe that Obama is Muslim. I know of no reason to believe they are correct. On the other hand, I find the confident assertion that he is a Christian by most of those commenting on the question somewhat naive.

For an American politician to announce that he was an atheist or agnostic would cost him a lot of votes in a national election. Obama is an ambitious and successful politician. So the fact that, at an early stage in his career, Obama announced that he had become a Christian tells us very little about his actual beliefs.

The same is true of other politicians. Sarah Palin was, I suppose, the most visibly Christian of the four candidates in the most recent presidential election. That may well represent her actual beliefs. But without knowing more than I do about when she first acquired political ambitions and what her life was like before that, I have no way of telling whether she is a devout believer or a competent pretender.

We do have pretty good evidence that neither Obama nor Palin is a Muslim, however. Islam requires of its believers visible actions, such as praying four times a day and fasting during Ramadan. I think someone would have noticed.

Why Don't Universities Sell Admissions?

Or do they?

It would be a mistake for schools, especially elite schools, to allocate places entirely on the basis of price, for at least three reasons.

1. Part of what schools are selling is a credential, and part of that credential comes from having been admitted. An employer prefers, ceteris paribus, employees able enough to have gotten into Harvard or Chicago. He has no reason to prefer ones rich enough to have bought a place at one of those schools.

2. The value to students of attending a school depends in part on the school's reputation, which depends in part on the quality of students admitted in the past. By basing admissions on measures of applicant quality the school may be able to raise average student quality, thus raise the performance of its graduates, thus raise the value of the school to future applicants.

3. Students are both customers and inputs. Smart students prefer an environment with other smart students, and probably learn better in such an environment. Put differently, a smart student provides positive externalities to fellow students and thus, indirectly, to the school, a dumb student provides negative externalities.

All of these explain why schools give some weight to measures of student quality in deciding whom to admit. But none of them explain why they give no weight at all to willingness to pay. A student is worth more to the school the more able he is, but not infinitely more. Even if student quality is the only thing schools care about, additional money could be used to offer more generous scholarships to able students who would otherwise go elsewhere, raising average quality. So one would expect schools to be willing to trade off, at some rate, money against SAT scores, agreeing to admit somewhat less qualified applicants at somewhat higher prices.

So far as I can tell, they do not do so. The reason might be internal ideology—elite schools for the most part are rich nonprofits, in a position to sacrifice financial benefits in order to act in ways that those running them approve of. It might be other people's ideology—schools may fear that the policy I have suggested would be seen as a corrupt favoring of the undeserving rich over the deserving poor. While everyone recognizes that wealth confers advantages on those who have it, many people find that fact objectionable, at least if the advantages are in things they think important, such as health care or education.

Both of those are, I think, plausible explanations, but not very interesting ones. Can anyone suggest a better alternative?

While discussing admission policies, it is worth also thinking about another puzzle: legacy admissions. While schools do not preferentially admit those who are willing to pay more—many claim, I suspect truthfully, that they do not even preferentially admit those able to pay full tuition over those who can only come if given large amounts of financial aid—many do preferentially admit the children of their alumni.

One possible explanation connects to the first part of this post. Legacy admissions can be seen as a covert and imperfect way of doing what I have just argued that schools do not do. Applicants are instructed to list on their applications any alumni among their close relatives. Alumni offices keep track of alumni donations; that information can be provided to admissions officers.

Are there other reasons for legacy admissions? One possibility is that the school thinks of itself as having a particular culture, being intended for a particular sort of people. Its alumni, having been not only selected to fit into that culture but instructed for four years in it, are particularly likely to be that sort of people, making their children more likely to fit in.

Another possibility is tribalism. Humans tend to divide the social world into ingroup and outgroup, us and them. One basis for such a division is what school one went to, a fact dramatically demonstrated at college football games. The people running a school and its almuni are part of the same ingroup, admission can be seen as a benefit given to those admitted, and people naturally prefer to allocate benefits to us instead of to them. They also prefer to include in the ingroup those most likely to be loyal to it. If Harvard admits the son of a Yale graduate, can he be trusted to cheer for the right team?

Other explanations?

My Arguments on other People's Blogs

I've been arguing about the meaning of Adam Smith's invisible hand on one blog and about whether it is obviously wrong for people such as orthodox Jews or Amish to bring criminal charges to their own authorities before reporting them to the police on another. Some readers may find one or both argument of interest.

Great Comment on Public Schooling

" This is probably because they interact socially with a far less age-segregated set of people (our public school system is really quite unique, and profoundly unnatural that way, it is as if someone read Lord of the Flies and decided it was prescriptive rather than descriptive)."

(Jehu, on Robin Hanson's blog, commenting on why he finds home schooled children much more likable than public school children)

Eggs: An English Lesson

The recent egg recalls have raised the issue of whether the FDA should require farmers to vaccinate their hens against salmonella, something it has so far declined to do. Thus the NYT writes:

Faced with a crisis more than a decade ago in which thousands of people were sickened from salmonella in infected eggs, farmers in Britain began vaccinating their hens against the bacteria. That simple but decisive step virtually wiped out the health threat.

But when American regulators created new egg safety rules that went into effect last month, they declared that there was not enough evidence to conclude that vaccinating hens against salmonella would prevent people from getting sick. The Food and Drug Administration decided not to mandate vaccination of hens — a precaution that would cost less than a penny per a dozen eggs.

The obvious implication is that the U.S. ought to imitate a wise British decision and require vaccination. Further down in the article, however, we discover that:

There are no laws mandating vaccination in Britain. But it is required, along with other safety measures, if farmers want to place an industry-sponsored red lion stamp on their eggs, which shows they have met basic standards. The country’s major supermarkets buy only eggs with the lion seal, so vaccination is practiced by 90 percent of egg producers, ...

Or in other words, Britain's success in drastically reducing the number of salmonella case is due not to regulation but to voluntary private action driven by market pressure.

Economics of Language and Courtesy

Someone commenting on my previous post mentioned the Gricean maxim of relevance. Checking the Wikipedia article on the Gricean maxims, I find the interesting comment that:

“Although Grice presented them in the form of guidelines for how to communicate successfully, I think they are better construed as presumptions about utterances, presumptions that we as listeners rely on and as speakers exploit.” (Bach 2005).

The maxims can thus be seen as an application of the economic approach to understanding behavior—the assumption that individuals have objectives and tend to choose the best way of achieving them. The objective is communication, the maxims describe how best to do it, and a listener dealing with potential ambiguity in speech—for example ambiguity in the meaning of “most”—can sometimes resolve it by assuming that the speaker is using the word with a meaning that achieves that objective. Where what is relevant is which candidate won an election, “most” is likely to mean a majority or even a plurality: “The party that got most votes was …”. Where what is relevant is whether there is a substantial minority for whom a statement is not true, “most” is likely to mean an overwhelming majority: “Most of my students understand English, so there is no need to provide translations of the readings into other languages.”

For a second application of economics, consider a scenario offered by a friend in a recent conversation on courtesy. Someone cuts into the checkout line ahead of you. One possible response is to accuse him of cutting into line. An alternative is to point out to him where the end of the line is, with the implication that he merely made a mistake.

My objective is to get him to go back to the end of the line, getting me through a little faster, and to do it with a minimum of unpleasantness. By treating his act as a mistake I lower the cost to him of doing what I want, since doing so does not require him to implicitly confess a deliberate violation of local norms. Lowering the cost to him of doing what I want makes him more likely to do it. What my friend regarded as behavior due to courtesy appears to me as a simple application of economics.

One can carry the argument one step further. If, instead of offering the norm violator an easy out, I loudly upbraid him, he will be less likely to quietly concede his error . But, since I will have raised the cost to him of cutting into line, he may be less likely to do it again. If my objective were the general good rather than my own private good, that might be the sensible choice, deterring future offenses against other people at some cost in current unpleasantness. In my friend’s view, the reason to be courteous was the benevolent desire to maintain social harmony. But courtesy, at least in this case, causes me to sacrifice the general good for my private good—precisely the behavior that economics predicts.

What does "most" mean?

In online discussions, I have come across an interesting disagreement over the meaning of a common word. To my ear, "most" means a large majority. To at least some others, it means any majority. Checking online dictionary definitions, I was surprised to find that many supported their meaning for the word rather than mine. Looking at an old Merriam-Webster's unabridged this morning, I found that the first definition included both alternatives.

To take a simple example, which of these statements is true?

"Most inhabitants of the U.S. speak English."

"Most inhabitants of the U.S. are female."

To my ear, the first statement is true, the second false. By the alternative interpretation of "most," both are true.

The situation is complicated by the fact that "most," in a different context, can be used to mean a plurality. In a three way election, the candidate with the most votes might have only 40% of the total. This usage is sometimes, but not always, signaled by the form "the most."

Comments? How do other people use the word? Does anyone have a plausible explanation of why both meanings appear to be common?