If so, do you approve?
The FDA and Pfizer arranged to have the tests that ended up showing their vaccine more than 90% effective done last Wednesday, the day after the election. It's clear from the account of what happened that there were multiple decisions that could have been made a little differently and would have produced the information a little sooner.
The obvious conjecture is that the timing was deliberate, that they expected a positive result and thought that announcing it before the election would help Trump. The alternative is that this is just another example of the FDA being (I think over) cautious, making absolutely certain the vaccine works, at a cost of about a thousand lives for every day of delay.
The more interesting question, for me, is whether Biden supporters believe that if it did happen, they approve. Would such a decision count as indefensibly using powers given to the FDA for entirely different purposes to meddle in the election, or as a responsible decision to save America from another four years of Trump? How deeply is "The end does/doesn't justify the means" embedded in the value system of commenters here and on FB, where I also posted a version of this?
Also of interest is whether there are any Trump supporters who believe that, if it happened, it was a defensible, if unfortunate, decision, that they would approve if something similar had been done by someone on their side.
You can find my view of the ends/means question in the relevant chapter at:
http://www.daviddfriedman.com/…/Ideas%20I_%20A%20Book%20fro…
That's a collection of draft chapters for the book I'm currently writing.
15 comments:
I wonder why you didn't phrase this as, "Why didn't the FDA Try to Help Trump Win?"
I think the conclusion you come to derives from your frame of reference. I have no knowledge of actual decision making.
Given the context, obviously the thought of the effect on the election came to mind. And keep in mind the outcomes of the trials were unknown.
One has multiple options:
- Try to ignore the (literal) near co-incidence and follow whatever policy and procedure controls.
- Make a partisan desicion you think will favor one or the other.
- Make a decision that attempts to remove any influence.
Partisans are going to see the second no matter what you choose; the first is what, I think, 'good people' are supposed to want when viewing through the veil of ignorance. The third is effectively impossible to pull off, but what I think happened.
Releasing any results before the election feels like a more active choice, injecting information in much the same way Comey did. Some manager somewhere is going to have to answer questions about why they chose to use their professional power to influence the election.
On the other hand, whether or not people choose to believe you, doing so after offers the simple claim that you were just following the schedule. It is much harder to argue with.
So, if I were betting, I'd credit or blame (depending on your political aesthetics) the personal conservatism of the professional managerial class for the decision.
This is a situation in which people’s belief in Democracy and their political beliefs come into clash but it is difficult to explicitly reject Democracy because it is socially undesirable or subconsciously difficult. Delaying the release of information for political purposes reduces the voters’ ability to make an informed decision, one of the essential features of Democracy. However, I think most anti-Trump people believe that Trump is so harmful to the country and the world that withholding information, even at the cost of lives, is worth it. That is very difficult to say out loud.
It is similiar to why Trump having an affair or saying vulgar things does not disqualify him in Republican’s eyes. They do not like it but they care more about policy and their side winning than symbolism, just as Democrats care more about policy and their side winning than Democracy. However, Republicans do not like to awknowledge that he has said vulgar or stupid things because it is socially undesirable and harmful to one’s candidates reputation.
I am an instrumentalist. I do not value Democracy as a procedure. So, if withholding information results in the better candidate winning, that is okay. If there are deaths involved, we need to do cost benefit analysis. Obviously this should involve more than lives and deaths but I will provide a simple example. If Biden were to end the wars and save thousands of lives or pass regulations which saved many lives, then it would be worth it. If roles were reversed, but Trump was going to save lives by not escalating wars or not passing regulations and saving lives (more than a few 1000) then it would be worth it. It is extremely plausible to me that the difference between the two candidates is so large as to make a few thousand deaths worthwile as bad as that sounds.
I think almost everyone is a instrumentalist first and a proceduralist second but it is difficult to admit to.
"doing so after offers the simple claim that you were just following the schedule. It is much harder to argue with."
When the schedule was agreed on, they knew when election day was going to be. As I understand the situation, the schedule called for the samples to be tested that Wednesday. It could have been set a week earlier. They could have agreed to test as samples came in instead of waiting to test them all on one day.
As it turned out, the number of positive results was well above what had been agreed on, so testing a week earlier would also have given a positive result.
Anyone here who has a more detailed account of the process is invited to provide it.
(Mathematician, but not specifically a statistician.)
The trial was set to complete when there were 164 confirmed cases in the sample, at which point the distribution of cases between the groups would give evidence as to whether the vaccine was preventing infections. There's no way to reliably predict when that number of cases will occur, so there isn't really an option to plan to have the trial finished before vs. after the election.
However, the trial isn't actually over yet--there have only been 94 confirmed cases as of the time of Pfizer's press release, and they are giving only preliminary results. Based on the way the statistical tests are designed, we can't confidently say that it works or not until we get the data decided upon ahead of time. So, Pfizer (but not the FDA, which wasn't involved in the announcement as far as I know) could possibly have chosen to delay until after the election for the press release if they wanted to do so.
"When the schedule was agreed on, they knew when election day was going to be."
I don't understand the quibble. Of course they did. This changes nothing about the analysis of my hypothetical chain of events.
The timing is unfortunate, but it looks like a question of when they decided to the the unblinding. When they knew they had 32 cases (but not in which group), they apparently decided to wait for more cases for greater statistical power, and because there was additional trial information that wouldn't be ready until late November anyway. So they didn't know until after the election that the news was very good; whether they wanted their vaccine announcement to not be tainted by apparent political bias is harder to know. In retrospect, it would have been better to have the original unblinding date after the election anyway.
It seems under-appreciated or under-weighted that being caught violating rules itself is a very negative outcome. That's one reason why I'm inclined to agree with the notion that ethical alternatives to consequentialism are instrumentally valuable – it's too hard to estimate all consequences of some (sufficiently significant) act.
And there's no obvious privileged 'level', or moment, at which a 'full consequentialist' analysis should be performed. A general 'by any means' norm itself (arguably) has different consequences than a strict 'by the book' norm.
Mostly, these kinds of cost-benefit analyses seem to be done 'in isolation' whereas I think a useful corrective is to routinely consider the consequences of adopting possible means as general policies instead of as entirely ad-hoc options.
'Game theory' sure seems to apply to these considerations and it's hard (for me) to ignore that all of these decisions are effectively being made in a (very much) repeated game; not as 'one shot'.
I could see the FDA as carrying out the requirements of a "pre-committed" strategy not to release good or bad information in the days immediately before the election. If correct, as it turned out, such a policy hurt Trump in this case; had the result of the trial been disappointing, though, the delay policy would have helped Trump.
Now, I have no evidence that this was the FDA's rationale nor even am I certain that it would be optimal if deliberate. But committing to *any* information revelation strategy would absolve the FDA of manipulation intentions. (I also realize that I have avoided the question of whether Biden supporters would promote a don't-tell-good-news policy if that were in fact its purpose.)
A somewhat similar issue came up in this post from last year.
It's a hypothesis-testing question. I generally assume as a default, null hypothesis that professionals are doing their jobs, with the alternative hypothesis (to be asserted only if the data make the null hypothesis look implausible) that they're making decisions substantially biased by partisanship. If Donald Trump knew any statistics, he would probably start in the opposite direction: assume by default that people make every decision with a partisan eye towards who benefits from the outcome, and only if the data make that look implausible should you conclude that people are doing their jobs based on the facts.
The reality in this case is probably somewhere in between, with two important calculations. First, people are basically doing their jobs in a fact-based way, but (for reasons of individual self-preservation, with the Comey example ever-present in their minds) they don't want to be accused of releasing data just before the election in order to influence the election outcome. Second, since the people making the many small decisions are predominantly scientists, who are predominantly horrified at the thought of another four-year direct onslaught on the notion of objective reality, they especially don't want to hold themselves responsible, for the rest of their lives, for doing so in such a way that tilts the election towards Trump.
The former calculation alone would justify a decision that "we won't release any data, no matter how good or bad, in (say) the month before the election", which would be consistent with the observed data. The second would warrant a decision that "we won't release strongly favorable results in (say) the month before the election", which would also be consistent with the observed data. So I don't see a way to distinguish between these two factors, but I would guess that both played a role.
"the FDA ... making absolutely certain the vaccine works, at a cost of about a thousand lives for every day of delay."
I would point out that the "cost of about a thousand lives per day" applies to the deployment of a vaccine, not to the release of data from an effectiveness study. Even if these data had been released a week earlier, it's not clear that a vaccine would have been widely available a week earlier, because there are so many other things -- government approval, starting and ramping up a production line, developing and ramping up a distribution system, etc. -- that have to happen between this point and that.
In particular, if as James says the trial won't be officially over until there are 164 confirmed cases in the sample, and that level hasn't been reached yet, then the release of these results now rather than when the 164 threshold is reached has no impact on the date that a vaccine is available.
Yes, it appears that the info was delayed to help Biden. But would it have mattered? I don't believe that Comey changed any votes.
Trump essentially told us the info before the election, and Biden said not to believe it. See this story.
Trump: "I've spoken to Pfizer ... They can go faster than that by a lot."
Biden: "we don’t trust him at all, nor do you"
https://www.washingtonexaminer.com/news/biden-does-not-trust-trump-to-deliver-a-safe-vaccine
Clickable link to above story:
https://www.washingtonexaminer.com/news/biden-does-not-trust-trump-to-deliver-a-safe-vaccine
@Anonymous:
I wrote::
"When the schedule was agreed on, they knew when election day was going to be."
You wrote:
I don't understand the quibble. Of course they did. This changes nothing about the analysis of my hypothetical chain of events.
Earlier you wrote:
"- Try to ignore the (literal) near co-incidence and follow whatever policy and procedure controls."
Perhaps I missed it, but I have seen no explanation of why "policy and procedure" would require unblinding on that particular date. If the policy was not to reach a conclusion until they had 62 infections, which I believe it was, then testing samples as they came in and doing the analysis when there were 62 positives would have produced the information as soon as possible, which would seem desirable.
Instead, they chose an apparently arbitrary date to unblind — which just happened to be the day after the election.
It's true that if the result came in negative, that decision would help Trump, but I doubt the people involved thought that was likely. The result actually came in considerably more positive than necessary for approval.
Does that make my point clear?
The reason the release of the information has little effect on vaccine availability is that the FDA has chosen to require two months of safety data, and that wasn't yet completed at the time. I don't think that requirement makes any sense, given that if the vaccine was very unsafe we would know by now — aside from very long term effects, for which two months would be insufficient — and the first people to get it will be people at high risk from Covid.
The other issue I have seen discussed is how long it will take to produce substantial quantities of the vaccine. It's hard to believe that having the information earlier has no effect on that, but it might not be a large effect.
Post a Comment