The Misbehaviour of Markets — Benoit Mandelbrot on Fractals in Finance

The Misbehaviour of Markets — Benoit Mandelbrot on Fractals in Nature and Finance

Financial Eden and the blackbox

There are two vantage points from which to see the world, writes Benoit Mandelbrot in The Misbehaviour of Markets. The first is through the Garden of Eden—one of deterministic cause-and-effect. The idea being that if we had enough data and information, perhaps we could plot the world like clockwork. Quantum mechanics and chaos theory shows us, however, that nature is uncertain in fundamental ways. Eden, in its entirety, is out of reach.

What remains is the alternative: to see the world through a black box. That is, to have a rough sense of the ins and outs, but not of its deeper machinations. Thermodynamics, for example, cannot track the path of every particle in its study of heat and work. But there remains many things it can say about the system and its probable states using statistical mechanics. When it comes to black boxes, probability and statistics are our friends.

Finance, Mandelbrot writes, “is a black box covered by a veil.” It is worsened in part by imperfect data, noisy reporting, and theoretical fads. The “most confounding factor of all”, Mandelbrot adds, “is anticipation.” Beliefs and expectations coil the past, the present, and the future into a complex ball of yarn of which we are all apart. What we need in finance is “a contrary approach” that is “macroscopic instead of microscopic”, and “stochastic instead of deterministic.”

Laminar flows and turbulence

For starters, Mandelbrot says it helps to think of randomness as having many states. If we draw a parallel to the solid, liquid, and gaseous states of matter, then randomness can be said to be either mild, slow, or wild. Mild randomness is akin to the randomness of radio static or flips of a coin. It is stable and predictable. Wild randomness, by contrast, is more irregular and disorderly, much like the craggy variations on our coastlines. Slow randomness sits somewhere in between these two extremes.

Fluid flows in a wind tunnel are good examples of randomness. “When the rotor at the tunnel’s head spins slowly, the wind inside blows nice and smoothly”, Mandelbrot explains. Such laminar flows “glide in unison in long, steady lines, planes and curves.” But “as the rotor accelerates, the wind inside the tunnel picks up speed and energy. Here and there, it suddenly breaks into gusts—sharp, intermittent.” We get turbulence. And turbulence of this sort—from the “billows of a cumulus cloud… [to] Jupiter’s celebrated red eye… [to] the pattern of sunspots”—appear everywhere and in all shapes and sizes.

To Mandelbrot, many aspects of financial markets “are analogous to physical systems of turbulence in a sunspot or eddies in a river.” They vault from calm to frenzy and back again. These expressions aren’t new, of course. Mandelbrot’s point, rather, is to see the underlying math that each of these black box systems share. Namely, that the math of fractal geometry—which finds applications in geology, meteorology, ecology, and elsewhere—may have descriptive power in finance and economics as well.

Coarseness, self-similarity, and fractals

To conform to their theories and models, economists will treat irregularities as tolerable errors or outliers. Techniques like the method of least squares in regression analysis make it easy to focus on broad relations. But in fields like finance or meteorology, the outliers are often of most consequence, even if they occur infrequently. Fractal geometry, by contrast, recognizes that “roughness is no mere imperfection from some ideal”, Mandelbrot writes. Irregularity is “the every essence of many natural objects—and of economic ones.” The trick is to find the regularity hidden inside the serpentine of disorder.

One approach is to search for “symmetries, or invariances—the fundamental properties that do not change from one object under study to another.” Fractals, in particular, tend to exhibit “a pattern or shape whose parts echo the whole”—a simple wellspring from which bewildering complexity can emerge. “The frond of a fern, for instance,… is made up of smaller fronds that, in turn, consist of even-smaller leaf clusters”, Mandelbrot shows. The case is similarly true of some varieties of cauliflower. “Each floret can be broken off and is, itself, a cauliflower in miniature.” Fractals, you see, tend to build themselves recursively. Their scaling patterns are self-similar.

Self-similarity in finance

To Mandelbrot, “the very heart of finance is fractal” (a self-affine multifractal to be exact). Imagine, for instance, that you are handed thousands of price charts of stocks, commodities and indices. But all the ticker names, timestamps, and price levels have been removed. All you see are bolded lines that move up and down. Some lines are trending while others are going nowhere. Some prices appear steady while others are erratic. With nothing else to go on, you’d have a hard time telling which from which, and when from when. Each chart looks different yet similar at the same time. Here, fractals and self-similarity are found not in the way ferns or lungs build out, but in the way prices change and volatilities vary.

Moreover, you may have noticed, as Mandelbrot did, that in contrast to orthodox theory, volatility in market prices “never settle down.” And that “there [are] too many big price jumps to fit the bell curve.” To reconcile such misbehaviour, Mandelbrot suggests that “a power law applies to positive or negative price movements of many financial instruments. It leaves room for many more big price swings than would the bell curve.” (It’s also worth noting that power laws are scale-invariant measures. If a distribution conforms to a power law, it will look the same at any two proportionately scaled intervals. Power laws and self-similarities tend to go hand-in-hand for this reason.)

Fractal worlds and illusory order

So price changes are not only of the “mild” coin-flipping sort as assumed by random-walk models in finance. Like fluids in a wind tunnel, volatility is sometimes slow and sometimes wild. We have to pay attention to “abrupt changes” and “almost-trends”—a self-similar gallery of booms-and-busts on multiple scales. Wild volatility tends to come in spurts and bunches as well. Mandelbrot parallels this to flood patterns in the Nile river, where dry and wet spells tend to cluster together.

It follows that if you want to build a high enough dam, it is “not just the size of the floods” that matters, “but also their precise sequence.” It also suggests that simple proxies for risk, like volatility and beta, might be too crude a yardstick. Market participants, after all, are heterogeneous and operate on many time scales. The variances and potentialities a day trader and long-only investor encounters will be different.

If abrupt-changes and almost-trends are manifesting across multiple scales and factors, we cannot say with exactness what will happen. It is a simple reminder that “the market is very risky—far more risky than if you blithely assume that prices meander around a polite Gaussian average”, Mandelbrot writes.

“You can spot intervals in which the motion appears to trend upwards, or slide downwards. Mere chance, of course… Pictures can deceive as well as instruct. The brain highlights what it imagines as patterns; it disregards contradictory information. Human nature yearns to see order and hierarchy in the world. It will invent it where it cannot find it.”
Benoit Mandelbrot and Richard Hudson. (2004). The Misbehaviour of Markets. A Fractal View of Risk, Ruin, and Reward.

A brief word of caution

For some of you, this is obvious stuff. Clearly, wild volatility must come in clusters because economic crises, political turmoil, social upheaval, and environmental disasters do too. But I hope you’ll forgive Mandelbrot for being a purist that prefers not to involve himself with the underlying drivers of price fluctuations. He admits that he prefers “to keep theory under control and stick to the data… [and] the mathematical tools [he] devised.”

However, the mathematician goes on to say that: “to drive a car, you do not need to know how it goes; similarly, to invest in markets, you do not need to know why they behave the way they do.” This, I think, deserves more disagreement for driving and investing are two different things. For one, the information you need to drive a car safely are usually within your periphery. Financial decision-making, on the other hand, is far more uncertain and nebulous. The multifractal models that Mandelbrot advocates, for instance, much like their neighbors in orthodox finance, associate risk with price volatility and related measures of price changes. Practitioners know, of course, that prices are an imperfect and sometimes unhelpful indicator of systemic and idiosyncratic risk.

Multiple mental models

Since much remains in the qualitative and the unseen, relying on one mental model alone in finance is conceptually dangerous. It leaves the user vulnerable to intellectual blind spots and the reflexiveness of competition. As Ed Thorp writes in A Man for All Markets, “learning is like adding programs, big and small, to this computer.” So until we have a complete description of economics, which we may never find, it seems unwise to confine ourselves to one incomplete mode of thinking.

So it is perhaps just as ironic that the economics establishment has been slow to incorporate Mandelbrot’s findings. This is especially so when empirical power laws are turning up everywhere we look—from word frequencies (Zipf’s law) to income distributions (Pareto’s income curve) to city sizes and insurance claims. Of course, not everything in nature or society is fractal. Lungs, ferns, cauliflowers, and market prices are not endlessly self-similar. Far from it. But the fact that power laws seem to manifest themselves across disparate domains suggests that there are deep structures for us to uncover and understand.

“Psychologically, we must keep all the theories in our heads, and every theoretical physicist who is any good knows six or seven different theoretical representations for exactly the same physics. He knows that they are all equivalent, and that nobody is ever going to be able to decide which one is right at that level, but he keeps them in his head, hoping that they will give him different ideas for guessing.”
Richard Feynman, The Character of Physical Law, 1964

Sources and further reading

Mandelbrot, Benoit., and Hudson, Richard. (2004). The Misbehaviour of Markets. A Fractal View of Risk, Ruin, and Reward.
Thorp, Edward. (2017). A Man For All Markets: From Las Vegas to Wall Street, How I Beat The Dealer And The Market.

Footnote: Scaling matters

As an aside, Mandelbrot asks readers to imagine a ball of thread. Sure, in your hands, it looks like a simple, three dimensional ball. But if you look at it from a distance, it starts to resemble a dimensionless point. And if you look at it very closely, then the individual threads begin to take shape and form. And if you zoom in further into its atomic structure, you will see something else altogether. So what then are the shapes and dimensions of a simple ball of thread?

Well, “it depends on your point of view”, Mandelbrot says. “For a complex natural shape, dimension is relative… Think of [it], not as an inherent property, but as a tool of measurement.” That is, the lengths we measure can change as the scales and rulers we use change. Scientists refer to this property or ratio-of-change as a fractal dimension.

Straight lines, for example, have a fractal dimension of 1. The lengths we measure do not change with magnification. The British coastline, by contrast, “has a fractal dimension of about 1.25.” As you zoom in, more and more “intricate inlets, promontories, cliffs, and crannies” present themselves. They get more complicated the closer you look.

Human lungs, likewise, are topsy-turvy tree-like networks with a “fractal dimension [that] is very close to 3.” From the trachea (windpipe) to the alveoli (air sacs), there are eleven orders of branching. It is a classic example of natural fractals—of “iterative division” and self-similar hierarchies—in action.

Scaling matters.