Drift into failure, p.30
Drift Into Failure, page 30
Diversity of opinion here can be one way to make the system stop and think when it thinks it can least afford to do so. This can be done by having an actor, or group of actors, in the organization who have the authority, the credibility and the courage to say "no" when everybody says "yes." European courts once used to engineer this diversity into the very circle of advisers that royalty relied On for strategic decisions. Knowing that advisers might agree with royal assessments or decisions just to save their own skin or serve their own interests, a jester would be part of the royal retinue, and cast his or her dissent in poetic or musical tones. This form of delivery not only immunized the jester from the expulsion or persecution that would have befallen normal advisers (the dissent and mockery, after all, Could be dismissed as a "joke"), it also represented a pointed, memorable way to make the king or queen think twice. When everybody said "yes," the jester could say "no" and get away with it. Shakespeare, as did others, made jesters feature large in some of his plays. In his Twelfth Night, the jester was described as "wise enough to play the fool." Their dissent had effect. King James VI of Scotland was known to be very lazy about reading things before signing them. Exploiting this, his jester George Buchanan (1506–1582) tricked him into abdicating in favor of George for 15 consecutive days. James got the point.8
Sensitive Dependency and Small Steps
Because there is no central designer, or any part, that knows the entire complex system, local actors can change their conditions in one of its corners for a very good reason and without any apparent implications. To them, it's simply no big deal, and in fact it may bring immediate gains on some local goal trade-off that people routinely face (for example, getting the System out the door versus checking it once more). Changing these conditions, however, may generate reverberations through the interconnected webs of relationships. Classifying the BP Liberty project in the Beaufort Sea north of Alaska as onshore rather than offshore even before drilling had started (see Chapter 1) was such a change in the initial conditions of the system. The effects of that classification would reverberate in unforeseeable ways through the design, regulation and operation of everything that followed after. The reclassification of foam debris events from flight safety to maintenance issues in the case of Space Shuttle Columbia (see Chapter 4) was another such change in conditions. A difference in an initial condition like this can get amplified (or suppressed) as its influences and post-conditions meander through the system.
These changes in condition (a redefinition or reclassification of a risk event, or a particular denotation of the status of a pre-operational system, for example) are not momentous in themselves. Which is why they are relatively easy to achieve, and hard to detect or stop. These are, in fact, often small steps. They are small steps that help the system locally optimize or rationalize a corner of its operations. Between-flight turnaround times of Space Shuttles become quicker if scratches and gouges in the insulating foam are pasted over with a mix of base chemicals rather than repaired, and if the problems that are noted are seen as belonging to maintenance rather than flight safety. Getting to the approval stage with the Minerals Management Service where oil may be pumped is easier for an onshore operation than an offshore one. These are rational, optimizing steps.
Drift into failure is marked by such small steps, or decrementalism. Constant organizational and operational adaptation around goal conflicts, competitive pressure and resource scarcity produces small, step-wise normalizations. Each next step is only a small deviation from the previously accepted norm, and continued operational success is relied upon as a guarantee of future safety. Small steps can become a lever for managing the complexity of drift. There are two advantages to small steps, after all.
One is that small steps happen all the time as complex systems adapt to their environment and produce new interpretations and behaviors as a result. This means that possible opportunities for reflection on practice offer themselves up quite frequently. Why did we just make that decision? Remember, in a complex system any action can influence almost everything. This means that we must promote a solicitude over the possible impacts of small steps. A local optimization may become a global disaster. Ask around about what the step might mean in different corners of the complex system. Go deeper or wider than you may have done before. And then ask some more.
The second advantage is that the steps are small. Calling on people to reflect on smaller steps probably does not generate as much defensive posturing as challenging their more momentous decisions. Retracting a small step, should people conclude that that is necessary, may also not be as expensive. In other words, small steps could mean that there is political and organizational space for critically reflecting on them, and perhaps ultimately space for not taking one of them.
Doing this may require some managerial action, however. Small steps and the normalization they can entail are often no longer be visible or seen as significant by insiders. Outsiders might think about this very differently, and they may represent a resource that managers could capitalize. Having people come in from other operational workplaces does at least two things. It brings fresh perspectives that can help insiders recalibrate what they consider "normal." It also forces insiders to articulate their ideas about running their system, as they engage in activities to train or induct the newcomer. A question such as "why do you guys do that?" can be taken not just as a call for explanation and training, but as an invitation for critical reflection on one's own practices.
This was one of the findings of high-reliability research: the 18-month job rotations on naval aircraft carriers, for example, demanded not only that constant training go on. It also offered the system a diversity of perspectives from people who came onboard after having been at a variety of other places. This was found to be one of the main ingredients for creating high-reliability operations in a risky and safety-critical setting.9 Rotating personnel from different places (but similar operations) for shorter periods can in principle be done in a variety of worlds. It represents an important lever for assuring the kind of diversity that can make small steps visible and open for discussion.
Unruly Technology
Unruly technology, which introduces and sustains uncertainties about how and when things may fail, can also be turned into a lever for managing the complexity of drift. Recall how technology can remain unruly. Even though parts or subsystems of a technology can be modeled exhaustively in isolation, their operation with each other in a dynamic environment generates the unforeseeabilities and uncertainties typical of Complexity.
We can actually use this to our advantage, Technology in a complex world, whether it is a Gaussian copula or a 15-storey high external Space Shuttle fuel tank, is never "finished." Declaring a technology operational has much more to do with us, and with our organizational and political constraints, than with the technology and how and where it operates. The entire Space Shuttle program, of course, has shown the perils of declaring a technology operational (and no longer developmental or experimental). That small change of a label carried enormous implications in terms of political, budgetary and societal expectations. And through this, that small change of a label contributed to large events later on: the losses of two Space Shuttles.
To capitalize on unruly technology, rather than be vexed by it, we should invert our perspective. We may traditionally have seen unruly technology as a pest that needs further tweaking before it becomes optimal. This is a Newtonian commitment: we can arrive at an exhaustive description of how the technology works, and we can optimize its functioning (that is, find one best way of deploying it), Complexity theory, in contrast, suggests that we see unruly technology not as a partially finished end-product, but as a tool for discovery. The feature of drift discussed above, small steps, is actually a wonderful property here. Small steps can mean small experiments. Many small steps means many small experiments and opportunities for discovery. These are small experiments without necessarily devastating consequences, but at the same time small experiments with the potential to create hugely important results and insights.
There is a beauty in unruliness. Unruly technology, and its violations of our expectations opens up little windows not just on the workings of a complex system and its environment. It also shows the extent our own calibration. If we are surprised by the effects of technology (NASA's initial surprise about the foam strikes, David Li's surprise and dismay about the enormous popularity of a Gaussian copula he invented), than that generates information both about our environment and about us. The questions we should reflect on (even if we may not get clear answers) are two-fold: why did this happen, and why are we surprised that it happened? The first question is about the workings of the complex system and its environment. The second is about our (lack of) understanding of those workings.
Now, separating these two questions of course risks instantiating and reproducing the Newtonian belief that there is a world (of technology-in-context) that is objectively available and apprehensible. Knowledge is nothing more than a mapping from object (technology out there) to subject (our minds). It assumes that our discovery about the workings of unruly technology is not in itself a creative process. It is merely an uncovering of distinctions that were already there and simply waiting to be observed. It rests on the belief that observer and the observed are separable and separated. The idea of our knowledge as an objective representation of a pre-existing order is very Newtonian. In it, the job of an analyst who looks at the data coming from unruly technology is to create representations or constructs that mimic or map the world – their "knowledge." When these copies, or facsimiles, do not match reality, it is due to limitations of the analyst's perception, rationality, or cognitive resources, or due to limitations of methods of observation. More data (and more lines of evidence, cleverer experiments) mean better knowledge: better copies, better facsimiles. This stance represents a kind of aperspectival objectivity, It assumes that we are able to take a "view from nowhere," a value-free, background-free, position-free view that is true. This reaffirms the classical or Newtonian view of nature (an independent world exists to which we as researchers, with proper methods, can have objective access).
But does this work? A central pre-occupation of critical perspectives on science and scientific rationality (as offered by Nietzsche, Weber, Heidegger, Habermas, to name a few) for at least a century-and-a-half is that our (scientific) knowledge is not an objective picture of the world as it is. Isn't what we see and what we can know very much a product of, and constrained by, the knowledge and language we already possess? Complexity science does not see the observed and the observer as separable. The observer is not just the contributor to, but in many cases the creator of, the observed. If there is an objective world of unruly technology out there, then we couldn't know it. As soon as an event has happened (for example, a foam strike), whatever past events can be said to have led up to it, undergo a whole range of transformations. The small steps that were taken by an organization (launching a Space Shuttle in low ambient temperature, for example) can get interpreted as contributing to particular outcomes in the behavior of unruly technology, but can also be constructed as irrelevant (as many believe was done by some of the telcon participants on the eve of the launch of Space Shuttle Challenger). Recall how actors or observers in a complex system are intrinsically subjective and uncertain about their environment and future, even if global organization emerges out of their local interactions with each other and the environment. What each of these agents knows, or can know, has little or nothing to do with some objective state of affairs, but is produced locally as a result of those interactions – themselves suspended within a vast and complex and non-linear network of relationships and processes.
The very act of separating important or contributory events from unimportant ones by a few decision-makers, then, is an act of construction, of the creation of a story. It is not the reconstruction of a story that was already there, ready to be uncovered. Any sequence of events or list of contributory or causal factors already smuggles a whole array of selection mechanisms and criteria into the supposed "re"-construction. There is no objective way of doing this – all our choices are affected, more Or less implicitly, by the backgrounds, preferences, experiences, biases, beliefs and purposes that we have. "Events" are themselves defined and delimited by the stories with which we configure them, and are impossible to imagine outside this selective, exclusionary, narrative fore-structure.
This, again, is where diversity matters. Diversity means importing different narrative fore-structures, different ways of seeing different things. Diversity means the ability to bring entirely different worlds into being, worlds of observation and experience that would perhaps not have existed within the framework available to the organization itself. The notion that there is not "one" true story of what happened with unruly technology, but rather multiple possible narratives, is an aspect of complexity that can be harnessed. Rather than making contradiction and paradox between different stories go away (by politically repressing them, by not listening to them through reporting schemes or employee empowerment), we could see a great opportunity. The space that is left open between different stories becomes a landscape for exploration and learning – not to find the final, true account, but to see how different interpretations lead to different repertoires of action and countermeasure.
Contribution of the Protective Structure
The last feature of drift into failure is the contribution of the entire protective structure. This structure can include parts of the organization itself (for example, internal controls or quality assurance programs), but also an external regulator and other forms of oversight. These structures are set up and maintained to ensure safety (or at least ensure regulatory compliance that is in turn believed to help ensure safety).
Protective structures themselves typically consist of complex webs of players and interactions, and are exposed to an environment that influences it with societal expectations, resource constraints, and goal interactions. This affects how it condones, regulates and helps rationalize or even legalizes definitions of "acceptable" system performance. It means that there is often something inescapably paradoxical and corruptible about the role of a protective structure. In principle, the idea of a protective structure is the addition of more diversity to the system. Bringing in outsiders is an obviously good idea – it gets done when consultants come in, but also when a regulator comes in. But if we look at the way in which workers from the federal Minerals Management Service were gradually co-opted by the oil industry they were overseeing then this is an example of a decline in diversity. Viewpoints between operator and regulator about what was risky and what was okay started to overlap, even while promoting an image of control and diversity.
The problem for a protective structure, of course, is that those outsiders cannot be complete outsiders. They have to have some affinity with the industry or organization they want to understand and influence. They have to have some insider knowledge, in other words. But not so much that they will become acculturated, usurped by the worldview they are supposed to assess. This goes for regulators too. Inspectors have to be outsiders and insiders at the same time. If they are just outsiders, they won't know what to look for and miss subtle signs and weak signals. They will also lack credibility in telling the operator, or their own regulator-superiors, what might need to be done. But if inspectors become insiders only, or if all inspection is basically done by the organization itself, the lack of diversity can lead to a halt in exploration of better ways, safer ways, to do work.
Perhaps the terms most closely associated with protective structures–regulation, compliance, oversight and inspection – are all fundamentally mismatched to complexity. Complex systems cannot in principle be regulated. Regulation, in its bare-bones form, means controlling or maintaining particular parameters so as to keep things operating properly. The idea of regulation is locked in a machine metaphor (and therefore Newtonian assumptions) of how an organization works. This includes not only an image of an organization as a collection of parts and interconnections, but the idea that we can arrive at a complete description of how the system works versus how it is supposed to work. This, of course, is where compliance-based approaches come in. If regulators discover gaps between rules and practice, they may typically try to close those gaps by pressing for practice that is more compliant with the rules.
Complex systems, however, produce behaviors that are more akin to living systems than to machines. Self-organization and emergence, let alone creative evolution, are impossible in a machine, And these behaviors are all very difficult to hold up against a set of rules for how the system is supposed to work. Rules cannot even accommodate creative evolution and emergence, because it would mean that somebody or some agency has designed, in advance, the emergence and creative evolution and self-organization and then condensed it all into rules that reflect some ideal-type against which actual evolving practice can be matched. Emergence, creative evolution and self-organization cannot be designed beforehand. That is the whole point of complexity. Remember how this led, in the Case of BP, to an approval of deep-sea drilling on the basis of rules for shallow-water drilling. Practice had evolved ahead of the rules. The regulator, entrapped by its compliance-based approach, could do little more than apply a set of rules it already had in hand (even though these rules had little to do with the emerging practices and unruly technology of deep-sea drilling). This is why the literature on unruly technology posits how even non-compliant practice is rule-following (but those rules are informal and unwritten), and that formal rules follow practice (see Chapters 1 and 3).
