05 August 2012

A taxonomy of uncertainty, "WARNING: Physics Envy May Be Hazardous To Your Wealth!"

From a paper by Andrew W. Lo and Mark T. Mueller, "WARNING: Physics Envy May Be Hazardous To Your Wealth!":

Level 1: Complete Certainty

Level 2: Risk without Uncertainty

randomness governed by a known probability distribution for a completely known set of outcomes

Level 3: Fully Reducible Uncertainty

risk with a degree of uncertainty, an uncertainty due to unknown probabilities for a fully enumerated set of outcomes that we presume are still completely known

Level 4: Partially Reducible Uncertainty

situations in which there is a limit to what we can deduce about the underlying phenomena generating the data. Examples include data-generating processes that exhibit: (1) stochastic or time-varying parameters that vary too frequently to be estimated accurately; (2) nonlinearities too complex to be captured by existing models, techniques, and datasets; (3) non-stationarities and non-ergodicities that render useless the Law of Large Numbers, Central Limit Theorem, and other methods of statistical inference and approximation; and (4) the dependence on relevant but unknown and unknowable conditioning information. ...

Under partially reducible uncertainty, we are in a casino that may or may not be honest, and the rules tend to change from time to time without notice. In this situation, classical statistics may not be as useful as a Bayesian perspective, in which probabilities are no longer tied to relative frequencies of repeated trials, but now represent degrees of belief. Using Bayesian methods, we have a framework and lexicon with which partial knowledge, prior information, and learning can be represented more formally. ...

At this level of uncertainty, modeling philosophies and objectives in economics and finance begin to deviate significantly from those of the physical sciences. Physicists believe in the existence of fundamental laws, either implicitly or explicitly, and this belief is often accompanied by a reductionist philosophy that seeks the fewest and simplest building blocks from which a single theory can be built. Even in physics, this is an over-simplification, as one era’s “fundamental laws” eventually reach the boundaries of their domains of validity, only to be supplanted and encompassed by the next era’s “fundamental laws”. ...

It is difficult to argue that economists should have the same faith in a fundamental and reductionist program for a description of financial markets (although such faith does persist in some, a manifestation of physics envy). Markets are tools developed by humans for accomplishing certain tasks—not immutable laws of Nature—and are therefore subject to all the vicissitudes and frailties of human behavior. While behavioral regularities do exist, and can be captured to some degree by quantitative methods, they do not exhibit the same level of certainty and predictability as physical laws. Accordingly, model-building in the social sciences should be much less informed by mathematical aesthetics, and much more by pragmatism in the face of partially reducible uncertainty. We must resign ourselves to models with stochastic parameters or multiple regimes that may not embody universal truth, but are merely useful, i.e., they summarize some coarse-grained features of highly complex datasets.

While physicists make such compromises routinely, they rarely need to venture down to Level 4, given the predictive power of the vast majority of their models. In this respect, economics may have more in common with biology than physics. As the great mathematician and physicist John von Neumann observed, “If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is”.

Level 5: Irreducible Uncertainty

Irreducible uncertainty is the polite term for a state of total ignorance; ignorance that cannot be remedied by collecting more data, using more sophisticated methods of statistical inference or more powerful computers, or thinking harder and smarter. Such uncertainty is beyond the reach of probabilistic reasoning, statistical inference, and any meaningful quantification. This type of uncertainty is the domain of philosophers and religious leaders, who focus on not only the unknown, but the unknowable.

Stated in such stark terms, irreducible uncertainty seems more likely to be the exception rather than the rule. After all, what kinds of phenomena are completely impervious to quantitative analysis, other than the deepest theological conundrums? The usefulness of this concept is precisely in its extremity. By defining a category of uncertainty that cannot be reduced to any quantifiable risk—essentially an admission of intellectual defeat—we force ourselves to stretch our imaginations to their absolute limits before relegating any phenomenon to this level.

Level ∞ : Zen Uncertainty

Attempts to understand uncertainty are mere illusions; there is only suffering.

(HT Steven Strogatz)