Monte Carlo Simulation: The Problem is Not Skinny Tails

Monte Carlo simulation is a convenient statistical tool by which an analyst can make inferences over the probability of rare events. Basically, the method takes a model with its estimated parameters and simulates possible sample paths under some distributional assumptions on the errors. The technique is particularly useful for understanding the tail risk of distributions. The method has been widely adopted by banks, investors, and financial advisors. And, because the models failed so horribly in the current downturn, the method is now under attack.

A lot of discussion over the failure of these models has centered on the use of normally distributed errors. The Normal distribution is the mainstay of statistically theory but the distribution places relatively little weight on tail events. In other words, the distribution treats rare events as rare.

The proposed solution is to work with fat-tailed distributions. Distributions that put more weight on tail events. These distributions raise the probability of very rare events, increasing the risk and raising the probability of observing rare events.

Monte Carlo simulation faces a far more severe problem than its distributional assumptions. To run a Monte Carlo simulation, the parameters of the distribution must be known. With a normality assumption, we need know only the mean and the covariance matrix. Yet, even assuming the model is correctly specified, we can never know these parameters. We have at best a good guess.

This problem of parameter uncertainty swamps any distributional assumptions. Rare events are rare. They are not observed in any reasonably-sized sample. Every sample is going to produce slightly different estimates of the model parameters and these slightly different parameters have huge implications for the probability of tail events.

I think the easiest way to demonstrate the problem is with an example. I will run a Monte Carlo simulation to find the probability of Industrial Production recessions. I assume the growth rate of IP is an i.i.d normally distributed process.

But, I estimate the variance and mean of the process over different periods. First, I choose the Great Moderation era, 1984 to 2006. Many statistical models of the economy restrict themselves to this period. This is a reasonable sample if you believe there was a regime shift in the early 1980s. Second, I extend the sample through March 2009. Finally, I use the post-war period and then the full sample from 1921.

The bars in the chart below show the probability that IP will fall below its starting value over any five year period. Essentially, the probability is the odds of a recession occurring over any five-year interval. The number above each bar gives the average interval between recessions.

Between 1984 and 2006, there is a fifty percent chance of a recession in any five year period with a recession occurring every 9.8 years on average. Extending the sample two additional years changes the probability significantly. Using data from 1984 to 2009, a recession occurs once every 7.4 years. This number is strikingly similar to the probability over the entire post-war era, the third bar. Finally using the entire IP sample, a recession occurs every 6.3 years on average and the odds of a recession in any five-year period is over 80 percent.
The average size of recession (shown in the next chart) also changes over the different samples. The ordering is preserved with the Great Moderation having the mildest downturns and the full sample, including the Great Depression, having the harshest recessions.

Again using the Great Moderation sample, the tail risk is also small. The following chart shows the average decline in output in a five percent recession. That is a recession that occurs only five percent of the time. The tail fall in output during the Great Moderation is less than 2 percent.

Takeaway: Monte Carlo simulation suffers far more from parameter uncertainty than from shortcomings over the choice of distribution. Slight changes in the sample used yield drastically different simulations. And, the correct sample is unknowable.

By the way, the problem of parameter uncertainty is endemic in forecasting models. We want to use history to guide our judgment of the future but history is a fickle guide.

The End of the Beginning? The Household Sector is a Problem

In a recent post, I presented some mixed signals on the prospects for a near-term recovery of the U.S. economy. In the end, I came down against a recovery being imminent but different eyes could have made the call in the other direction; indeed, once again I am in the minority and most forecasters are seeing a preponderance of green shoots in the data.

I view the world through different eyes than most economists. At least in policy circles, economists see the current downturn as driven in its entirety by a financial crisis. I don’t think this is a financial crisis (here’s why). I think this recession was caused by a shock to the manufacturing sector but it is the bad household balance sheets that have driven the dynamics of this recession: too much debt transformed a small productivity shock into a full blown solvency crisis. The asset side of the household balance sheet (household income) has fallen slightly; the high level of liabilities transforms this to a crisis.

The global economy cannot return to health until households have worked off at least part of their excess debt. So far they have made little progress.

The picture below shows the ratio of total consumer credit outstanding to personal income. From the late 1950s through the early 1990s, this ratio was stable on average between 0.14 and 0.16. Then in the 1990s the ratio took off, increasing to 0.19 by early 2000. The fall in income and no decline in debt drove the level even higher through the summer of 2003 where it peaked at 0.23.

Like the current account balance, like consumption’s share in GDP, this ratio is too high and will have to adjust before the economy can return to health. It may have to adjust to its 2000 level or it may need to go all the way back to its long-run average, but it has to move down.
But, household debt alone does not give a sufficient accounting. Government debt is essentially an off-balance sheet liability of the household sector. And government debt has grown apace over the last 10 years. In January 2001, the federal debt was $5.7 trillion. An increase in the growth rate of government spending, combined with slower revenue growth raised the growth rate of government debt. By December 2008, federal debt had reached $9.2 trillion. Since then, the debt has increased more than 20 percent, exceeding $11 trillion dollars and approaching the value of nominal GDP.
Adding the federal debt to consumer debt raises the household debt to income ratio to more than double is pre-1990 level. As bad as these numbers seem, they are actually much worse. Household demographics do not support the increase in debt. Over the next five years, the proportion of American households over the age of 65 hits a record. There is a reason people retire at 65; beyond that age morbidity rates rise sharply, implying lower productivity. Social pension programs are a side show. The true problem is the fall in labor input and the loss of human capital and making people work longer is not the answer: there is a reason people tend to retire at 65 (morbidity). The household sector is living beyond its means and its means are not going to grow.

Long-term Adjustment: Short-term growth?

So, over the long run, the household sector needs less debt (see the previous post). What about short-term indicators? Is the household on the brink of recovery despite my dire predictions? No, but let’s take a look.

Surveys provide the timeliest insight on the health of the consumer sector and consumer confidence has ticked up. Both the Michigan survey and the Conference Board are at or near record lows, but both series have stopped falling and may be poised for a rebound. But consumer confidence is volatile and does not strongly lead the cycle. The small increases in the series so far this cycle are indistinguishable from random noise.

A better direct indicator of consumer-sector health is the state of the housing market. Residential investment leads the cycle by one to four quarters and it tends to lead vigorously, falling rapidly from the peak and rising robustly from the floor. Housing starts have shown no inclination to rise and it looks like they still want to fall.

We will hit a floor in housing starts in the next few months. (There is a binding zero nominal bound for starts.) Rather than being good news for the housing market, hitting the floor in starts may trigger a more virulent phase of the downturn. If the economy can no longer adjust through quantities, it must adjust through prices. If starts can no longer absorb part of the adjustment, house prices must fall faster. If house prices fall faster, more households will opt for foreclosure, adding an independent source of pressure on prices.
And of course, household incomes are not going to support recovery. The graph below shows a stunning (and familiar) picture of continuing claims for unemployment. Continuing claims for unemployment insurance are at a record; claims as a percent of the workforce are rapidly approaching the earlier peak as well.
So, long-term fundamentals are negative for consumption and near-term indicators have yet to turn. I just don’t see hope of a full-blown recovery. What I see right now is a short-term respite from the dreariest days followed once more by a period of contraction.

Testing the Ice: The End of the Beginning

"Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning." Winston Churchill, 1942

The evidence is mounting and an increasing number of people are seeing signs of recovery, green shoots. The consensus amongst forecasters is that the worst is behind us and that the economy will return to growth in the second half, maybe even by the second quarter. And, there are some promising signs in the data. I’d say I see a glimmer of hope, but no more than that.

By far the strongest evidence of a turnaround is the most recent initial claims data. Initial claims fell 3.4 percent in the month of April. The fall itself was not large, a one standard deviation event. But the implications of the fall may be immense. Initial claims always turn down at the economic trough. In this post, I discussed the ins and outs of employment. I showed that a fall in the separation rate was far and away the timeliest indicator of recoveries. And initial claims are the closest we have to a direct measure of this rate in real time data.
If the trend in unemployment continues, this recession is over (at least for the moment—see 1981). Of course, the movement was quite small by historical standards and does not yet point to a true recovery. False dips in initial claims are quite common and in ¾ of recessions initial claims moves down by at least 4 percent before rising once again.

The picture below puts the fall in perspective. If claims fall at their April rate for the rest of the year, the level of December claims remains far above its recent levels and far above its level during the 2001 recession. Claims have to start falling a lot faster for us to be sure of recovery, but the sign is positive. This is my single favorite indicator of upward turning points.

The second most positive piece of news is the manufacturing PMI. As shown in the picture below, the manufacturing PMI for the United States has now risen three months in a row and the increases have been sizable. Any number below 50 indicates a contraction in the sector, but the upward movement is encouraging. Historically, changes in the PMI are tightly linked to both the manufacturing sector and the broader economy. This series is consistent with a return to growth as early as the end of the second quarter (June).

Working against the manufacturing PMI is the nonmanufacturing sector. The PMI for nonmanufacturing is falling again after bouncing off its end of year lows. This series looks like it wants to head down. And since the nonmanufacturing sector accounts for around 70 percent of economic activity, the series cannot be ignored.

More importantly, the data from the manufacturing sector is not nearly as positive as the PMI would indicate. Over the three months that the PMI has risen, shipments and new orders have continued to fall, albeit at a slightly slower pace than in previous months. Still, the level of both of these series remains near the peak prior to the last recession. Like unemployment insurance claims, shipments tend to turn up before the official trough of the recession.

All told, I still do not see a recovery even in the manufacturing sector, although I think there are many hints that the sector is no longer collapsing. That’s good, if it kept collapsing at the rate of the last six months, we would reenter the dark ages before two years passed by. But an end of the collapse is not the same as the beginning of a recovery.

To stick with the WWII theme, the year is 1942; the war is not yet won. But, the Germans have been stopped at the shore, and while we are a long way from total victory, absolute defeat no longer approaches.