This morning I came across a news story purporting to show that vaccination against the HPV virus does not make girls more likely to be sexually active. After reading it, I have no more reason to believe the conclusion than before reading it, making it a good example of bad journalism.
The article describes an experiment in which more than 300 girls between 13 and 21 were interviewed about their sexual behavior before getting the shot and at intervals thereafter. Before getting the shot 42% of the girls said they were sexually inexperienced. Six months after, of the 99 who had been inexperienced, 20 no longer were. That is the sum total of information about whether the vaccination makes girls more likely to be active provided by the article—there is also a little more information, almost equally worthless, about the effect on the behavior of those already active.
So far as I can tell from the article, there was no control group. The article offers no information about how many girls from a group with the age distribution of those studied would be expected, absent the vaccination, to become sexually active over a period of six months. The only thing it tells us relevant to what it is supposed to be about is that the researchers say that their research produced the conclusion that, pretty obviously from the tone of the article, they wanted it to. No information by which a reader can decide for himself whether it's true.
Just for fun, I tried doing a back of the envelope calculation to see how plausible their claim was. To get an upper bound on the rate at which girls become active, assume that 100% go from inexperienced to experienced between age 13 and 21, making the average rate 11%/year. The story does not give the age distribution of the group studied, but if we assume an equal number of each age and take the group size to be 300, they should have been going from inexperienced to experienced at a rate of about 33/year. According to the article, the rate was actually 20 in six months. Insofar as the calculation tells us anything, it suggests that the rate for those vaccinated was higher than would have been expected—the opposite of the conclusion of the article and, presumably, the study.
That is not very strong evidence that the study's results were bogus, since we do not know the actual age distribution. It could have been concentrated in an age range where the average rate was as high as the rate observed. I have not read the study itself, so have no way of knowing whether it produced evidence for (or against) its conclusion. But the article on it is pure puffery.
The article mentioned, at the end, a previous study on a smaller scale that did use a control group. There too, it gave no information by which the reader could tell whether the reported conclusion was true. "That study ... found no significant differences" presumably means that the any differences found were not statistically significant, which sounds as though it means that there were no differences that matter—provided the reader does not know what "statistically significant" means.
P.S. A commenter (via Google+) provides me a link to the study. What the authors were looking at was not the relation between vaccination and sexual behavior but the effect of vaccination on perceived risk of sexually transmitted infections and the relation between that perception and sexual behavior. The authors report that for a majority of participants vaccination did not affect perception; they do not say what the effect was on the minority. They found no evidence that a reduction in perceived risk resulted in an increase in sexual activity. Their study produced no information on whether vaccination affected sexual behavior through mechanisms other than its effect on reported perception of risk—for example by implying that it was normal and accepted for girls to be sexually active.
The oddest thing in the paper is the theory they seem to be testing, which they refer to as risk homeostasis. That theory apparently holds that individuals act to maintain a constant level of risk. In the sexual context, that would imply that any reduction in the riskiness of intercourse would be exactly balanced by an increase in frequency.
From the economist's point of view, that is equivalent to assuming that the total amount a consumer spends on a good is independent of its price, that if the price falls in half the consumer doubles his consumption. It makes me wonder if risk homeostasis is the result of someone reading a comment by an economist and not understanding it.
----
P.S. A commenter (via Google+) provides me a link to the study. What the authors were looking at was not the relation between vaccination and sexual behavior but the effect of vaccination on perceived risk of sexually transmitted infections and the relation between that perception and sexual behavior. The authors report that for a majority of participants vaccination did not affect perception; they do not say what the effect was on the minority. They found no evidence that a reduction in perceived risk resulted in an increase in sexual activity. Their study produced no information on whether vaccination affected sexual behavior through mechanisms other than its effect on reported perception of risk—for example by implying that it was normal and accepted for girls to be sexually active.
The oddest thing in the paper is the theory they seem to be testing, which they refer to as risk homeostasis. That theory apparently holds that individuals act to maintain a constant level of risk. In the sexual context, that would imply that any reduction in the riskiness of intercourse would be exactly balanced by an increase in frequency.
From the economist's point of view, that is equivalent to assuming that the total amount a consumer spends on a good is independent of its price, that if the price falls in half the consumer doubles his consumption. It makes me wonder if risk homeostasis is the result of someone reading a comment by an economist and not understanding it.
14 comments:
Stephen Hawking wrote that A Brief History of Time was almost entirely bereft of mathematics because "someone told [him] that every equation included would halve [his] sales". The same principle could explain junk scientific journalism.
I think it is usually a waste of time to read summaries of scientific studies made by journalists. There will be exceptions as always...but mostly it goes something like this:
http://xkcd.com/882/
(those statistics related xkcd comic strips bookmarks of mine really come in handy pretty often :) )
If the problem of horrid science reporting was limited to regional newspapers and link baiting internet websites, the world wouldn't be such a bad place.
The sad thing is that this is the norm wherever you go. Which science journalist ever, anywhere, made a critical note about elementary things like statistical significance or control groups?
Someone misunderstands an economist's note! Not at all surprising, since lots of economists, including Nobelists like Gary Becker, have not quite mastered English grammar, making some of their writing unintelligible to the layman who dares to try to read it.
The other comments are about the sad state of science journalism, which I agree is indeed sad. But in this particular case, I think the journalism just reflects the study.
Now, the study. The study is terrible. I think they set out to answer the question, "Does HPV vaccination turn kids into raging nymphomaniacs." They found that it doesn't, at least most of them. But you can't really publish an article on raging nymphomania, at least not in a regular journal (but think of the news coverage you'd get!), so they decided to put it under a tamer headline.
Tamer, and wronger. But these are the prices you have to pay in order to publish.
(Well, you don't actually have to. But if you've gotten to the point of having designed and executed a horrible study...)
It's pretty amazing to me how much terrible, terrible science gets published. Really it's not that hard to design a controlled study, even one that's survey based rather than experimental.
dmorr: You have read the study? Or are you assuming that the study itself is bad? If so, why?
Perhaps more interesting is why anyone cares that people are having sex?
If you set aside religious ideas (fine for you, not OK to inflict on others), I don't see a problem.
There are obvious reasons why people care about other people's sexual activity, especially parents about their children. Sex risks various diseases, risks (for women) pregnancy, and perhaps most important has substantial emotional consequences. The result of going to bed with someone might be concluding you are in love with him or her, getting married, and discovering over the next five years that that was a very stupid choice. I don't think I am entitled to make sexual decisions for you, but I think it entirely reasonable for me to be concerned about such decisions made by people I care about, most obviously my children.
What strikes me about this issue is a related point. The people arguing (for all I know correctly) that the vaccination won't make girls more sexually active are trying to calm the fears of parents who are worried that it might. But a lot of those people, I suspect, share your view of the subject. That makes their reassurance unconvincing, since if they thought vaccination did make girls more sexually active they would still be in favor of it--in some cases even more in favor of it.
David: At the same time, a lot of those who publish studies with opposite results are probably also against vaccination whether it affects sexual life of the girls (in any way) or not. I think you pointed it out quite well in an earlier post - no one is unbiased (only those who don't care about the topic at all and they probably won't make any studies or research) and if they are honest and competent, bias is not such a big issue.
Of course, if this particular study is the same kind of rubbish as the journalist's report on it, then it fails (at least) the competence test. But having not read the actual study, I cannot make any conclusions about that. Perhaps it does contain a control group, reveals the distribution of those girls' ages (x girls between the ages of 13 and 21 is very vague...it could mean one 13 year old, one 21 year old and the rest 18 year olds and I would be very surprised if the distribution of the sex life start age were a flat one) and other relevant information. You can make valid research which looks like bogus if it is interpreted by someone who is incompetent and perhaps also dishonest.
I noticed the P.S. So while the study was not as great as it could be, the journalist still wildly misinterpreted it...to a point of drawing conclusions that are not part of the study and not even tested there (directly anyway).
That is probably even worse than the xkcd example. And another reason for me to disregard any news article that starts with "New study shows that..." at least as long as they do not include a link to the study itself and I am interested in the topic enough to read the study first (after which I don't need the journalist's report anyway).
Risk Homeostasis is a hypothesis first advanced by psychologist Gerald Wilde in a 1982 paper, so there's at least some empirical evidence behind it.
There's also a similar but weaker idea, the Peltzman effect, which should be uncontroversial to anyone familiar with economic theory.
Peltzman's piece on auto safety was part of what my final sentence referred to. We would expect a decrease in cost to result in an increase in quantity. But there is no reason to expect the product of price times quantity to stay fixed.
Certainly not always and everywhere, but apparently at least occasionally, assuming http://onlinelibrary.wiley.com/doi/10.1111/j.1539-6924.1982.tb01384.x/abstract is methodologically sound.
Post a Comment