Tuesday, April 10, 2012

Truth, Falsity, and Jobless Numbers

I recently came across a web page with the following claim:
Given the difficulty of estimating economic data it is common practice for government agencies to announce a preliminary number subject to later revision. Under the law of averages, estimates should balance out between being higher or lower than later revisions. Amazingly, though, the Obama Department of Labor’s preliminary estimates of new jobless claims have been lower than later revisions in 56 of the last 57 weeks.  
The claim was followed by what purported to be a quote from Labor Secretary Hilda Solis. 
"We feel it is better to err on the side of optimism,” she said. “The preliminary estimate is widely reported. The subsequent revisions are rarely noticed. By adding a bit of sheen to the preliminary estimate we feel we are helping to boost morale. We believe that good morale is an important building block for positive change.”“Making the economy look better will make people feel better,” Solis went on. “If people feel better they are more likely to support the policies of the Administration, which we feel is crucial if we are to be given the opportunity to continue on the path laid out by the President for another four years."
My immediate reaction was suspicion—the quote sounded too much like what a critic of the President would imagine his labor secretary saying and quite unlike what an administration official would actually say. I put a comment on the web page to that effect, adding that I didn't have an opinion on whether the initial fact was true.

Further investigation found the quote only on pages hostile to the administration, and no source other than another such page. On the other hand, the fact, initial underestimates for 56 of the past 57 weeks, is from the Wall Street Journal and so presumably true.

I conclude that the labor department has indeed been deliberately misrepresenting the evidence—erring in the same direction 56 times out of 57 is not something that has any significant probability of happening by chance. The obvious explanation is  the one given in the purported quote. But I am quite confident that the labor secretary didn't actually say what the quote asserts she said, at least not in public.


P.S. A commenter points out that the "quote" from the labor secretary actually originated as part of a longer piece, obviously intended as satire.

Pretty good satire, too.


Daniel J. Smith said...

Thanks for tracking that down, David. I was curious myself as to the veracity of the post. I did want to clarify that the post on my blog < www.danieljosephsmith.wordpress.com/ > was just a re-post from John Lott's Blog < http://johnrlott.blogspot.com/2012/04/so-what-are-odds-of-obama.html >. I actually post nothing original on my 'blog,' I just use it as a way to keep track of the posts by others that I find interesting and the posts that I intend to read later on, as well as to categorize the posts later on so I can find them easier if I want them for a lecture or research (in fact, I'm an avid fan of your blog so I am constantly linking to your posts).

I agree with what you wrote and really appreciate you taking the time to write a post clarifying the issue.

The Ghost said...

I've been tracking the first time unemployment claims for about a year (since 4/16/11) and I can verify that the revision has never been down ... the average "adjustment" has been over 3,500 up ...

this smells to statistical high heaven ...

Miko said...

Rather than the "secret pro-Obama conspiracy" explanation, I'd suggest the alternative that they just aren't very good at making predictions. Tversky and Kahneman's _Judgment under Uncertainty: Heuristics and Biases_ (for example) looks at the ability of experts in many fields to make predictions and finds that systematic errors like this are actually quite common.

jeremy h. said...

I'm also suspicious, but even more so than Dr. Friedman. The first release is not a prediction, it is a report based on the available data. Once they get more data, the data are revised the following week. And what more data would they get? Most likely, people filing claims after the fact. The only way it would go down (that I can imagine) is if fraudulent claims were detected.

There might be a case for not releasing the initial numbers. Just release the revised numbers! However the WSJ article's suggestion that "the market reacts to the initial report, but not the revisions" has no evidence to back it up. And it seems wrong, unless we think traders are stupid.

Simon said...

This seems to be the source:


It's satire. And actually rather funny. Amazing that someone would mistake it for the real thing: it's like Rachel Maddow and ChristWire.

Stephen Bloch said...

It's possible that the Obama administration might intentionally release optimistic initial numbers, on the theory that good news actually causes the economy to improve. It's also possible, as Jeremy h. suggests, that something about the data-gathering procedure produces a systematic downward bias in initial numbers.

Has anybody computed a similar statistic for other
administrations in similar situations, e.g. the first few
(recession) years of the GWB administration?


jeremy h. said...

Stephen, I had the same question about historical data, but was unable to easily find the data myself. WSJ says the source was St. Louis Fed, but I can only find the final, not preliminary, data on FRED. Of course, the data probably would not resolve the issue, since it's possible that the data has always been manipulated (or that it never has).

Anonymous said...

I work at a research department in a bank and have access to data revisions across many economic series. In general, during the recovery first releases actually under-report improvement. A case in point is the Aug 2011 payrolls release which was initially reported as adding zero jobs but has subsequently been revised to +85 thousand jobs.