The (mis)behavior of risk managers: recognizing our limitations

Citation

Mainelli, M. (2005), "The (mis)behavior of risk managers: recognizing our limitations", Journal of Risk Finance, Vol. 6 No. 2. https://doi.org/10.1108/jrf.2005.29406baf.002

The (mis)behavior of risk managers: recognizing our limitations

The (mis)behavior of risk managers: recognizing our limitations

Many happy returns?

This column was inspired by reading a book, meeting its two authors, and having a breakfast. Benoit Mandelbrot, inventor of fractal geometry and one of the progenitors of Chaos Theory, is Sterling Professor of Mathematical Sciences at Yale University and a Fellow Emeritus at IBM’s Thomas J. Watson Laboratory. Richard L. Hudson is former managing editor of The Wall Street Journal’s European edition. Their new book, The Mis(behavior) of Markets: A Fractal View of Risk, Ruin and Reward (Mandelbrot and Hudson, 2004), is meant to challenge financial people to re-evaluate the standard tools and models of modern financial theory. I recently had the pleasure of discussing the book over breakfast with the authors and a number of leading financial thinkers. Some of the book recapitulates other writings by Mandelbrot and quite a bit of the book’s material has appeared in similar form earlier either by Mandelbrot himself or via other authors (e.g. Peters, 1991, 1994). Nevertheless, it is wonderful to see Mandelbrot return to finance again to remind us how shaky some of our risk management foundations are.

Despite paraphrasing the title of their book for this column, this is not a book review. The book is worth reading for its insights into the non-normal (non-“bell curve” or non-Gaussian) distributions encountered in finance and criticisms of the core assumptions of financial theory, e.g. Markowitz’ Random Walk, as well as noting the limitations and dangers in tools such as the Capital Asset Pricing Model, Sharpe Ratios and Black-Scholes option pricing. At the breakfast promoting their book, it seemed strange that the fundamental criticisms they made seemed, at times, less important to the small audience than how the authors’ conclusions might help improve financial prediction. In other words, “I don’t care whether it works; how can you help me make money?” I found myself more interested in what should my approach be if I believe that the foundations of risk management are wobbly.

Misbehavior of risk tools

If distinguished authors, supported by copious research, are criticizing many fundamental risk management tools such as Beta and Value-at-Risk and labeling GARCH or FIGARCH as “patches”, perhaps we should pause for reflection before proceeding. Risk as a subject has witnessed advancing taxonomy in traditional Aristotelian form. While we have defined huge trees of risk, we still roll ahead of us a pile of un-taxonomized risk. Formerly we believed we understood “market” risk and “credit” risk, so “operational” risk was the pile of remainders. Today we can assume we are understood as we refer to “market”, “credit”, or “operational” risk, but our rolling pile still exists. We are being slightly disingenuous when we pretend that everyone surely understands “liquidity” risk. Only recently have we been paying attention to “reputational” risk or “governance” risk.

Problems posed by chaos, complexity, and predictability question whether a systems approach to risk management is appropriate or possible in whole or in part. Chaos Theory is not a theory, but a way of approaching problems. Chaos Theory assembles a set of techniques and viewpoints that seem to recur in problems. These techniques try to take account of boundary conditions between order and disorder, between the easily modeled problem and the impossible to model problem. Although earlier work by Cantor and Sierpinski has been a great influence, “chaos” as an informal intellectual movement began in the early 1960s, largely kicked off by the work of Lorenz and Mandelbrot. “Many systems have now been found where randomness and determinism, or chance and necessity, integrate and coexist” (Peters, 1991, p. 41).

It’s the models, stupid!

Non-linear dynamic models are the posters of the Chaos Theory movement, as evidenced in numerous coffee table books of striking images produced from graphs of the models (as well as book covers themselves, such as that of Mandelbrot and Hudson’s). The initial recognition was that simple models can produce apparent complexity. Further, this apparent complexity seems, in many cases, to resemble apparent complexity found in nature, such as trees, clouds or coastlines. The exploration of fractals rose markedly in the mid-1970s, made significantly easier through the ready availability of computers. Graphs of deterministic non-linear models that appear to generate unpredictable behavior are popular in Chaos Theory because they illustrate well the existence of a region of models bounded by chaos on one side and the organized simplicity of continuous models on the other. These models tend to have the characteristics of being aperiodic and having forms that change structurally given small changes to the model variables. At the same time these models exhibit an underlying order, with resonances, symmetries and attractors that people perceive. Several authors make a point that linear modeling is a reduced case of non-linear modeling and that fractals, for instance, are truly ubiquitous in a way that linear approaches cannot be. Chaos Theory and its illustrative companions, fractals, are seen to encompass a new science of wholeness. On a cautionary note, and echoing William of Ockham:

1. 1.

   Although it is true that it is the goal of science to discover rules which permit the association and foretelling of facts, this is not its only aim. It also seeks to reduce the connections discovered to the smallest possible number of mutually independent conceptual elements. It is in this striving after the rational unification of the manifold that it encounters its greatest successes, even though it is precisely this attempt which causes it to run the greatest risk of falling a prey to illusions (Einstein, Nature 146, 1940, 605, as taken from van den Beukel, p. 83).

So beware of fractals as “illusions”. While there are elements of romanticism within Chaos Theory literature, concrete analytical work sits alongside. Chaos Theory’s distinctions among chaos, order and the boundary between chaos and order is particular useful as a metaphor. One large theme in Chaos Theory is said to originate with Poincaré:

1. 1.

   […] sensitive dependence on initial conditions (Peters, 1991, p. 135).

1. 1.

   Chaos theory is about limitations on predictability in classical physics, stemming from the fact that almost all classical systems are inherently unstable. The “instability” in question has nothing to do with any tendency to behave violently or disintegrate. It is about an extreme sensitivity to initial conditions (Deutsch, 1997, p. 201).

In one direction, we frequently examine financial-mathematical models and compare them with reality in hopes of finding a lasting comparison from which the existence of a theory may be inferred. However, this “can be confused with ‘data mining’ or torturing the data until it confesses … Actual results depend on many numerical experiments with varying test parameters. If this sounds unscientific, it is” (Peters, 1991, pp. 171-2). Another direction is to take a theory, develop a model and compare it with reality.

De Grauwe and Vansenten (1990) built a model of the foreign exchange market from generally accepted theory. They demonstrated the chaotic properties of the model and contrasted the model with actual exchange rate data showing numerous statistical similarities with the model outputs. The results illustrate that models can be constructed which may be perceived to mimic actual market behavior, but which are not predictive. However, minute changes to the model inputs resulted in new outputs that bore little statistical similarity to actual exchange rate data. The model was sensitive to initial conditions to such a degree that no data could ever be accurate enough to begin forecasting. The degree of variance in normal input data was more than sufficient to change the model output structurally. Trivial facts or events can completely alter the model output. “Exact” knowledge of the environment would be necessary to use the model predictively and knowledge to that accuracy is unlikely. Moreover, the model is untestable as there are no means of obtaining data of sufficient quality to test it.

It’s the predictions, stupid!

Niels Bohr’s famous wry comment was, “prediction is extremely difficult, especially about the future”. Working empirically, Makridakis (1986) concludes that the answer to “can the future be predicted?” is, “it depends”, pointing out the many problems with setting realistic expectations for forecasts. The Economist (1992, p. 91) proffers an old rule: “predict a figure or a date, but never both”. Many people, including this columnist, are less interested in deterministic models than predictions from historic statistics. We are interested in seeing whether, while we may not be able to model the system, we can work from input measures such as key risk indicators (KRIs) for banks to predict losses, or from historic prices to identify anomalous trades. We have taken on board some of Mandelbrot’s comments over the years and do include fractal dimensions as coefficients, just as we do volatility or weighted moving averages. We spend our time building correlation engines between inputs and outputs without concerning ourselves overly much with the many sub-systems interacting and creating the outputs from the inputs. “Predictions from models without understanding”, if you will.

Interestingly, however improbable prediction may seem as a measure, most organizations feel that a successful plan or system should predict future performance, with a bias to believing that outcomes result from action, not inaction. Ironically, inaction or the initial disposition of forces is a decisive factor in many victories. Sun Tzu (1963) indicates that there is a strong element of quiet victory in strategic success and that true success is the victory won before an obvious battle: “and therefore the victories won by a master of war gain him neither reputation for wisdom nor merit for valor”. It can be difficult to distinguish positive inaction from luck, or negative action from misfortune. When organizations try to apply objective measures of success to predictive systems, the results are disappointing, particularly as detailed in Sherden (1998) or Mintzberg (1994, pp. 91-152). The concept of prediction from historic data entails a number of philosophical problems, such as being unable to predict discontinuities, let alone paradox:

1. 1.

   But it is logically impossible to predict future knowledge: if we could predict future knowledge we would have it now, and it would not be future; if we could predict future discoveries they would be present discoveries. From this it follows that if the future contains any significant discoveries at all it is impossible to predict it scientifically... if the future were scientifically predictable it could not, once discovered, remain secret, since it would in principle be rediscoverable by anybody. This would present us with a paradox about the possibility/impossibility of taking evasive action … With the collapse of the notion that the future is scientifically predictable the notion of the totally planned society goes down as well. This is also shown to be logically incoherent in other ways: first, because it cannot give a consistent answer to the question “Who plans the planners?” (Magee, 1985, p. 100).

Are risk managers misbehaving?

So, we reach a tight spot. Leading mathematicians, such as Mandelbrot, point out that most of our financial data distributions are non-normal, thus questioning the popular tools in our toolkit. Our taxonomies are brittle and immature and cannot handle “sets of risk” (Leitch, 2004). Our deterministic models are potentially chaotic and unverifiable. Our statistical models can help us in “steady state” situations or at identifying anomalies, but are worthless at predicting discontinuities. Our approach to designing risk management systems assumes that cause (our new, improved risk management system) results in success (reduced risk), but we cannot prove it. We have few tools at all for handling the largest operational risk factor, people (Howitt et al., 2004; Mainelli, 2004).

People sometimes ask how financial market risk varies from natural risk such as earthquakes or hurricanes. In my opinion, the key distinction is that financial markets incorporate people’s perceptions. People add a very strong feed-forward loop to financial systems. Talking a market up or down frequently moves a market up or down. If people change their perceptions of a risk, e.g. terrorism recently, then that perception change alters future behaviors. My suspicion is that the feed-forward loop of people’s perceptions might well explain the non-normal distributions we encounter in finance.

There is an old Groucho Marx joke that Woody Allen recycled to explain the inevitability of amorous relationships:

1. 1.

   Doctor, my brother thinks he’s a chicken. Can you help?

   Why don’t you stop him?

   We need the eggs!

In some ways, risk management systems, checklists, or models are inevitable. These risk tools may not work but, as humans, we need the eggs.

So what should risk managers do? Well, first, proceed with caution around any certainties in approaches to risk. Second, be honest with ourselves and others about the limitations of top-down taxonomies, ticklists, models and systems. Third, keep risk management close to issues of human behavior; risk management is mostly about changing future behaviors. Fourthly, look for hard evidence that will help you reinforce risk management approaches that are working and revise or remove those that are not – find KRIs that really are related to losses; rethink ticklists until their scores really do reveal higher risks; search for questions and processes that demonstrably add value.

Finally, we need to recognize that we are only in the early stages of research into risk management. Many of today’s accepted tools and techniques will be shown in future to be useless or wrong or even dangerous. We need to avoid erecting such an elaborate establishment that we cannot dismantle large portions for rebuilding. In short, we need to keep our minds open, our spirits inquiring and retain a sense of fun and adventure as we try to solve some of the biggest problems in finance – aligning risk and reward.

AcknowledgementsThe author would like to thank the Centre for the Study of Financial Innovation (http://www.csfi.org.uk/) for sponsoring the breakfast with Benoit Mandelbrot and Richard Hudson that inspired this column, and also Matthew Leitch for his helpful comments.

Michael MainelliDirector, Z/Yen Limited (Michael_Mainelli@zyen.com). Z/Yen Limited is a risk/reward management firm helping organizations make better choices. Z/Yen undertakes strategy, finance, systems marketing and intelligence projects in a wide variety of fields (www.zyen.com), such as developing a risk/reward prediction engine, helping a global charity win a good governance award or benchmarking transaction costs across global investment banks.)