This page had been redirected to a new URL, please update any bookmarks.

Font Size: A A A


« Settling In | Main | Soft, Not Too Soft »

May 16, 2007

The Wisdom Of Forecasting Crowds (Such As It Is)

Ever wonder who you should turn to for expert economic prognostications?  My colleagues Mike Bryan and Linsey Molloy (future Chicago MBA!) remind us that the answer is everybody and nobody:

... we examine economists' year-ahead growth and inflation predictions since 1983 to see whether any have distinguished themselves as particularly good (or bad) forecasters over time.

We find little evidence that any forecaster consistently predicts better than the consensus (median) forecast and, further, we find that forecasters who gave better-than-average predictions in one year were unable to sustain their superior forecasting performance—at least no more than random chance would suggest.

Not that consensus forecasts are all that great:

... we summarize the track record of the median economist’s year-ahead predictions for real GDP growth and CPI inflation since 1983. (Forecasts were compiled by the Livingston Survey.) If we arbitrarily define an accurate prediction as being within 1/2 percentage point of the realized outcome, we would say that since 1983 the median forecast was accurate in only seven years, or about 30 percent of the time... The accuracy of the median forecaster’s prediction of inflation was a bit better over the 23-year period. Inflation predictions were accurate—that is, within 1/2 percentage point of actual inflation—39 percent of the time...

... So suppose the median forecaster expects the economy to grow 3.4 percent next year (its average since 1983). You could conclude—with 90 percent confidence—that the economy will grow between a robust 5.8 percent and a sluggish 1 percent. Similarly, the RMSE of the median economist’s inflation prediction over this period was 1 percent, which means that given an average inflation rate of 3.1 percent, you could be about 90 percent confident that prices will rise between a stable 1.4 percent and an uncomfortably rapid 4.8 percent over the coming year.

Fair warning, I think.

May 16, 2007 in This, That, and the Other | Permalink


TrackBack URL for this entry:

Listed below are links to blogs that reference The Wisdom Of Forecasting Crowds (Such As It Is):

» Forecast Accuracy and Consensus Forecasts from Businomics Blog
A recent report from the Cleveland Fed by Michael F. Bryan and Lindsey Molloy confirm older results that consensus forecasts do better than any one forecaster. (Hat tip to Macroblog.) That's one reason I pay close attention to consensus forecasts, [Read More]

Tracked on May 19, 2007 1:19:48 PM


What value is any forecast if it's not based on assumptions supported by evidence? Have a majority of macro Economists even agreed that the tripling of home prices in CA (& other bubble states) in the last 10 years was NOT due to fundamentals?
For years Dean Baker's been doing his best to argue that "The story of the (housing) bubble is the sharp divergence between sale prices and rent over the last decade, which indicates that sales prices were not being driven up by fundamental factors in the housing market."
If housing contributed a VERY large portion of our growth for the last four years and that growth was not based on "fundamentals" (AND there's no other sector projecting large enough growth to replace this growth), shouldn't we expect a year or two of NO GDP growth? Shouldn't this be a goal to get our economy back on secure footing?

Posted by: bailey | May 16, 2007 at 11:16 AM

"... to get our economy back on secure footing"

What??!! This is America, the home of the free and the land of the brave. The very thought of "secure footing" is simply not welcome here since it is so... so... UN-American!

"... secure footing ..." sounds like a concept that the Europeans would be attracted to.

The hallmarks of a "healthy" American economy are "dynamic", "unpredictable", "vibrant", "disruptive change", "volatile", "constant change", "innovation", "continuous reinvention", "nimble", "agile", etc. There is simply no room in this list for "secure footing."

The one exception is monetary policy, for which we seek (and currently have) "relatively" low and stable inflation and "relatively" high employment. The emphasis remains on flexibility and tolerance for volatility and a determined effort not to be locked into a fine-tuned, mechanistic approach to "driving" or "steering" the economy.

"... secure footing ..." - No way!

We all need to do our best to avoid such a disastrous and misguided basis for economic activity.

If you ever do achieve a "secure footing", I can assure you that the infamous "invisible hand" of the market economy will eventually knock you off of that "secure" footing.

-- Jack Krupansky

Posted by: Jack Krupansky | May 16, 2007 at 01:09 PM

Buffett on economics:
"I am not a macro guy. I don't think about it. If Alan Greenspan is whispering in one ear and Bob Rubin in the other, I don't care at all. I'm watching the businesses."

"I don't read economic forecasts. I don't read the funny papers."

Posted by: RB | May 16, 2007 at 01:16 PM

Jack, Oh for the days when that hand was invisible! I'll buy what you're proposing the MINUTE I see an inflation explanation for our monetary policy that's supported by a representative population sampling.

Until then, I'll stick with my thinking that any Gov't. sponsored policies that act to defend house prices that have tripled in 10 years is unhealthy.

As far as our "home of the free", I recommend you buy some new strings for that old guitar of yours, huh? Then, before you decide upon a melody, take another look at our Governments' long-term liabilities. I suggest you practice up for "the Blues".

Posted by: bailey | May 16, 2007 at 02:19 PM

There are two important aspects to be mentioned when interpreting the results of this study. First, the failure of a number of economists to give an accurate prediction at one year horizon does not mean that such a prediction is impossible in principle. This is a banal logic - individual negative results can not deny any theory. Otherwise, hard sciences would have been extinguished due to a constant failure in the beginning of any study. Just to recall the case with quantum physics. There was an absolute consensus between physicists in the end of 19th century that physics was a self-consistent theory without any problem to solve yet, except might be couple effects (as found to be of quantum character afterwards) which everybody failed to explain. As we know now the consensus was wrong.
Second, there is a relationship that allows to predict inflation at 2.5 year horizon with the RMSFE of 0.5% for the period after 1983. For illustration, a figure comparing measured (smoothed by MA(2)) and predicted (smoothed MA(4)) GDP deflator for the period between 1958 and 2004. For the sake of synchronization, the predicted curve is shifted 1 year ahead.

The predicted curve is obtained by the following relationship:
Therefore, having better measurements of labor force level, LF, two (actually 2.5) years ago one can exactly predict GDP deflator. In the figure, only past (relative to GDP deflator) estimates of labor force are used. So, the prediction is out-of-sample and one can use the term RMSFE, not RMSE. For the annual readings, the prediction is 2.5 years ahead.
CPI, being a part of GDP deflator, is also well predicted but a bit more volatile. This is a common feature of physical systems - violability of amplitude of fluctuations increases from the level of the system as a whole to portions of the system. The smaller is the portion the large is fluctuation. This relationship also demonstrates that the assumption of the impossibility to predict inflation is not valid.

Posted by: I.O.Kitov | May 17, 2007 at 03:38 AM

"... look at our Governments' long-term liabilities."

Sorry, but it is always foolish to focus so intently on only one side or term of an equation. Liabilities? What about assets? What about the positive benefits, economic and otherwise, accrued as we incurred said "liabilities."

I remain completely unwilling to open a "short position" on the future productive capacity of The American People acting in a global economy. It may be difficult to place a value on this asset, but why assume that it is essentially zero and not even acknowledge its existence?

As to "The Wisdom of Forecasting Crowds", the only crowd that matters is all of those colllective "invisible hands" of The American People acting over the coming decades. They will in fact be creating the future and are unlikely to feel constrained by either so-called professional forecasters or "one-siders" who are effectively clueless as to the raw productive economic capacity and potential of The American People.

One other note... the so-called "liability" of the federal debt and deficit is actually an asset in some ways, such as the "value" people perceive in owning Treasury "debt". It really is hagly valued. Worried about the trade deficit with China or China owning so much Treasury "debt"? I would submit that both of those "liabilities" help to stabilize the geopolitical and cultural climate and hence deliver significant "value" to The American People. Somehow, you need to account for such positive benefits and not simply focus obsessively on the raw financial numbers.

BTW, the last numbers I saw showed that the federal budget deficit was shrinking, even as the federal government continues to grow.

-- Jack Krupansky

Posted by: Jack Krupansky | May 17, 2007 at 12:36 PM

For God's sake, Bailey. Can Dave make one post in which you don't make one of your whiny, depressing, government is stupid, the economy is out of control, mortgage brokers are evil, the world's about to end comments? Don't you get tired of making the same comment day after day, month after month, year after year? Have you no idea that your efforts affect nothing?

Posted by: cb | May 17, 2007 at 01:10 PM

Ouch CB, mea culpa! To be fair, I readily confess to being an afficionado of fine WHINE (& the cheap stuff too). But, to be fair, I've never said nor implied Gov't. is stupid, nor that mtg. Brokers are evil. I post only with the hope that sooner or later Economists on BOTH sides, right & left, will get off their soapboxes and reconsider their assumptions.
To your credit I suppose, your comment is similar to one I received a few months back on PK's shadow site.
So, I think it's time once again to head for Hawaii to sop up some sun & reconsider MY approach. Aloha, all.

Posted by: bailey | May 17, 2007 at 05:02 PM

Enjoy your trip!

Posted by: cb | May 18, 2007 at 12:39 AM

Voters have a responsibility to question their leaders, ESPECIALLY Economists who use their professional credentials to espouse political rhetoric. Beyond this I believe Fed staffers have a responsibility to inform their leader when he's being unfairly used by the current Administration.
In Mr. Bernanke's speech yesterday he reported "In 2006, 69 percent of households owned their homes; in 1995, 65 percent did." This is a GWB Administration soundbite for their "ownership society".
I think voters have every right to expect dribble from the Commerce Dept., but we deserve better from our top Economist.
I think Mr. B. should have identified "homeowners" as those who own at least 10% of their homes. At the very least he should have excluded those who have no equity and are reaping HUGE Gov't. tax breaks on mtg. loans they can walk away from with barely a smirch on their already questionable credit.
Mr. B. may be politically naive, but I don't believe it's incidental that "1995" was close to the cyclical depths for housing prices, while in "2006" home prices were still close to the highs.

Posted by: bailey | May 18, 2007 at 12:05 PM

Post a comment

Comments are moderated and will not appear until the moderator has approved them.

If you have a TypeKey or TypePad account, please Sign in