May 7, 2009

Future Proof:Catching Black Swans

Recently an earthquake hit the town of L'Aquila in Italy, collapsing a number of buildings, killing more than a hundred people and leaving several thousands stranded. Last year Hurricane Ike slammed into Galveston, Texas, leaving many parts of the coastal community submerged. In the news of late have been the "one-time write-offs" that banks are taking because of the extraordinary credit crisis.


The one thing that all three of these things have in common is that they appear to be "Black Swans", a term that economist and mathematician Nassim Taleb used to describe events that seem wildly improbable, yet nonetheless do occasionally happen. The term derives from the saying that, as all swans are white, the term "as rare as a black swan" means something that is so improbable that there is no way that it could happen.. Of course, eventually a species of black swan was eventually discovered in Australia, which was Taleb's point - beware of assuming that simply because events are rare, they will never happen - and when they do happen, they tend to cluster.

To understand why black swans are not as rare as one may think (and to anticipate their appearance), it's worth going back to the abstraction model proposed a couple of posts ago. Almost all complex systems exist at multiple levels of abstraction. One way of thinking about this is to envision the human body as being made up of subsystems - organs - each of which in turn are made up of tissues. which are in turn made up cells. A person in peak health has all of her subsystems (organs) operating more or less optimally.

However, one day a carcinogenic agent enters the body - tobacco smoke, asbestos, environmental steroids, the list is rather depressingly long. A cell in her breast mutates in the presence of the carcinogen, losing the ability to "shut off and die", which most cells do after they've reached a point where their internal mechanisms are no longer sufficient to do the job efficiently. The cancer spreads, every time the cell would undergo normal meiosis. If the woman is lucky, a routine examination would find this cancer when it is fairly small, at which point the best solution is to remove the cancerous tissue.

If the woman is unlucky, the cancer would grow until it found a conduit (typically a lymph node or a blood vessel). Cancer cells that broke free from the mass would be transported by the conduit until it ends up somewhere else in the body, at this point it would attach itself to other tissue and continue spreading. The cancer cells crowd out other cells, choking off access to blood vessels or waste channels, and other cells either become cancerous in turn or they become necrotic - dying but not being removed by the body's defenses. The woman becomes tired more easily as energy that would normally be going to maintenance of the body is increasingly co-opted by the cancer cells. Tissue becomes tender and inflamed, and pain caused by cancer cells crowding in on nerve endings becomes more endemic.

If the cancer spreads to the lungs, then breathing becomes more difficult and becomes an emphysema. If it spreads into the lymph system then the woman has more trouble fighting off infections, and becomes sicker more often. If it spreads into the bone then normal stresses may cause the bone to snap.

The broken bone is a black swan event - it seems unlikely that a bone would normally break under typical stress actions, but in point of fact, this isn't a typical stress action. The system has been compromised, and the cancer has spread out in a spider-web like fashion through much of the body. An aggressive fight against the cancer by irradiating it or using poisons (chemotheraphy) may be able to remove the tissue, but typically it does so by further weakening the ability of the body to function.

The body does not die because of the cancer. Instead, the cancer causes each system in turn to become less efficient, and ultimately to fail because it can't get the energy necessary to continue. Once one organ fails, it increases the likelihood that other systems that are dependent upon that organ will also fail. The unfortunate woman dies of system failure.

This rather detailed and morbid description still serves as a metaphor for other systems. Complex systems are made up of simpler ones, which are made up of simpler ones still. Corruption usually occurs fairly far down in a given system, but most complex systems are generally fairly effective at catching and eliminating the obvious points of corruption (or worst case scenario, sequestering them off in isolation) . Corruption here simply means a subversion of the normal functions of that particular abstraction - an bridge inspector signing off an inspection report with only a peremptory check of the bridge, an employee stealing supplies from the supply cabinets, a student cheating on an exam or a businessman cheating on his taxes, a radical publishing seditious tracts, to name a few of the many, many examples.

Physical system analogs would be areas of snowfall on a mountain that gets more sun than normal but is also supporting other areas of snowfall, a particularly warm, dry, dust-laden wind coming off the Sahara into the Atlantic Ocean, the gradual creep of increasing temperatures in an area that hasn't faced them pushing flowering and bee pollenation behaviors slowly out of synch.

The point about most such corruption (i.e., regions of potential instability) is that, for systems in quasi-equilibrium, the corruption usually has comparatively little impact over the short term. Most systems have regulating mechanisms that tend to correct for such instabilities - the office manager notices that one department is using more supplies than the other, and a bit of surveillance reveals that one employee is using dramatically more than he should be. At that point, the employee is summarily fired, and a new employee hired to replace him, and the message is made clear - you steal, you're gone. This tends to move the system back into equilibrium.

The regulation and action is not a normal event - it is only undertaken when corruption is noticed. It's a small "collapse", one that may result in some disruption of activities and hence impact the efficiency of the abstraction - and for the employee it results in a significant disruption from the way things were. However, such feedback cycles normally keep the system relatively stable.

However, over time, the corruption can become more endemic, and at a higher level of abstraction. The managing bridge inspector is lax in checking on reports, and the inspectors under him avoid looking at those places on the bridge that are awkward to get to or would require getting especially dirty, the comptroller in a company works with one or two accounts to falsify the books, the teacher at a university starts accepting bribes and sexual favors for grades, a company provides campaign donations to a politician in order to give them a tax break or special legislative consideration.

Note in this case that there are two levels of abstraction involved in all of these scenarios. Generally the role of a manager is to act as a governor or regulator on the actions of others, to provide negative (damping) feedback to minimize corruption in a system. When that feedback is subverted, it amplifies the corruption rather than reducing it, and it makes it increasingly likely that the feedback will start to compromise the stability of the abstraction layer.

One of the more interesting phenomena that takes place in systems of abstraction is the paradox that the longer a system remains in equilibrium, the more likely that it will become unstable. In order to understand why, consider that most corrective feedback occurs only after a problem has reached a crisis point - the office manager finds that supplies she just ordered are gone, and she can't think of any legitimate reason why they would be. As she is responsible for her budget, she knows that she has less lattitude if excess pilferage is reducing her budget (and that she could face "corrective action" if such thefts continue to be unexplained.

However, as the organization gets larger, the office manager has more responsibilities, and tracking down pilferage drops down the list. The attitude begins to form that office supplies are fair game, and people begin more inclined to take supplies whether they need them or not - and those office supplies begin to move towards bigger ticket ideas like computers and projectors. Expense accounts start to become padded, and pretty soon begin to become a significant part of a person's income. Eventually the amounts begin to become high enough that it impacts the bottom of the line of the company, particularly if the comptroller and his friends in accounting are in on it (getting kickbacks for equipment that's disappearing).

What makes this worse is that it has gone from being an isolated instance to becoming pervasive and endemic. You can't fire everyone without bringing the company to its knees. Eventually you are forced to fire the comptroller, establish a new-tight accounting system for all internal goods and services, alienate a number of your employees who had come to see the office supplies as a right, and then spend several months searching for a new accounting team.

Stability breeds complacency, which breeds instability. Hyman Minsky, an economist in the 1960s, laid out this hypothesis for financial systems, but it holds in most complex multi-layered abstraction systems. Deregulation of the banking industry, low interest rates on the part of the Federal Reserve and a push towards home-ownership in the early 2000s meant that bankers could make higher risk mortgage loans to increasingly unqualified buyers then sell these loans to other financial institutions. These financial companies would combine these mortgages in novel (and dangerous) ways and sell them as financial vehicles to investors. The investors would then use these securities as collateral to build increasingly unsustainable leverages, while insurance companies sold "black swan" insurance that they never expected to pay off in order to make these securities palatable to accountants. Meanwhile, the real estate agents worked with the appraisal firms and builders in order to get the largest payback in fees, and homeowners in turn found themselves forced to take out ever larger loans for the same properties.

Low initial rates on loans were reset after a specific period to a much higher rate, and people began to fall behind on their payments, in time, the cascade of defaults and jungle mail cascaded through the system. The rapidly receding value of these assets caused a Minsky Moment in September 2009, as a key investment bank, Lehman Brothers, was allowed to go bankrupt. Because of the unwinding of the positions that Lehman had held, this created the financial equivalent of a heart attack as credit disappeared from the market overnight.

Starved for credit, companies could no longer sustain regular payroll, watched their energy supply (cash inflow) dry up as consumers pulled back abruptly in spending and soon were forced into rapid liquidation. Abstractions were unwound as energy (in the form of credit) disappeared from the system. Unemployment shot up as millions of people were forced out of work, accelerating the crisis, while attempts to recapitalize the banks have so far fallen short of solving the systemic problems.

Instabilities - turbulence - disrupted each layer of abstraction. This process is still ongoing, and will likely take one to two more years in order for the turbulence to dissipate to a level that new structures can start to form again, at a lower energy level.

The lesson for futurists - first, disruptions do not happen without reason. Most disruptions occur when a stable (complacent) abstraction becomes corrupt and brittle. In a recently stable scenario, external stimulae impinge upon the system all the time, but the system is resilient enough to ward them off. As systems become more mature they become more fragile, and their ability to adapt becomes increasingly compromised. Eventually, a stimulus occurs that causes a breakdown of a particular part of the system, and the system has become so interdependent that this shock then gets passed on, destructively, to other subsystems. The subsequent loss of system integrity can prove fatal, and the system will unwind to a less energy intensive state as energy bleeds into turbulence.

In general, you cannot predict what the shock will be that will ultimately send a system over the edge, and its futile to try. What's important is to examine whether, given a shock, a system is resilient enough to absorb it, or whether the shock will prove devastating. The role of both analysts (who are futurists) and regulators is to do the research to determine what organizations are too fragile, and then to examine the consequences that a shock to the system will have.

One final note here: a good place to look are organizations that are deemed "to big to fail". Most likely once a company (or a government) reaches that point, it is definitely overdue for an earthquake.

No comments: