Financial simulators–broadly grouped into a) historical return calculators and b) Monte Carlo simulators–are popular tools for financial planning. But it’s important to recognize their limitations.
Historical Return Calculators
Historical return simulators (e.g. FireCalc) allow you to test a given strategy against historical returns to see how often it would have worked. For example, you can check how often a 4% starting withdrawal rate would have been successful over a 30-year retirement given various stock/bond allocations.
Such calculators are useful for showing what has not worked in the past. Showing that a strategy has worked only occasionally tells us that we should have little confidence that it will work in the future. That’s why, for example, we know that it’s unwise to plan to withdraw 7% of your portfolio every year during retirement.
Monte Carlo Simulations
Monte Carlo simulators allow you to perform similar tests. But instead of testing a proposed strategy using actual historical sequences of returns, they ask you to provide statistical descriptors of investment returns (average return, standard deviation of returns, correlation to other investments, etc.), then they test the proposed strategy against numerous return sequences generated using those descriptors.
Monte Carlo simulations are especially useful for testing how much a plan’s probability of success will change as a result of changing assumptions. (For example, if stocks end up being 10% more volatile over annual periods than they’ve been historically, will that be a major problem?)
Are Historical Returns Meaningful?
Consider this analogy: You’re trying to determine the average height of a group of people (as well as other facts such as the standard deviation of heights among the group). With every additional person from the group that you measure, your data set grows and you can be more confident in your conclusions.
We try to do the same thing with historical returns–collect an ever-growing pile of data and use it to determine things like average annual stock market return.
But there’s a problem here: As our sample size grows, our population could be changing. For example, I’d assert that the financial markets and world economies are meaningfully different from, say, 50 years ago in several ways (examples: instantaneous information on stock, bond, and commodity prices; automated trading in very large amounts by institutional investors).
What effect will those changes have on investment returns in the future? I don’t know. But I don’t think we can simply assume that such changes will have no effect.
As such, any data older than 50 years is of limited value. As we continue to collect more data, we have to keep throwing our old data out as it becomes less and less relevant. Even today’s data may not be particularly relevant if you’re concerned with returns several decades into the future.
Conclusion: The predictive value of any simulations based purely on historical data must be taken with a healthy dose of skepticism.