But Which Human?
Given the probabilistic, chaotic nature of the firing of nerve cells that leads to human thought and action, error is an inevitable result of our interactions with the world.
Put another way, to err is human. Often in the wake of some presumably avoidable disaster (say, a plane crash), some official proclaims that the disaster was due to “human error.” Such official explanations typically focus on operator error.
This focus misses other important types of human errors: by the people who design the systems, those who maintain the systems, and even those who make management decisions.
Fundamentally, all disasters have human error at their base (except for natural disasters we can’t anticipate), because human involvement is critical to their occurrence.
But, who made the error? Did the pilot ignore a warning light? Maybe the warning light should have been replaced.
Perhaps the designer should have included a back-up warning system, to cover for a failed warning light.
Or maybe the designer specified that a back-up warning system was needed, but the product manager decided it was too costly.
Thus, following a disaster, the first critical question to ask is, Which humans made the errors?
The second and third questions may be even more important – Why did these humans make the error?, and How can we avoid or correct it in the future?
This article does not address legal liability. The assessment of liability is fraught with concerns such as precedents and contractual agreements, and well beyond the purview of a paper such as this, with an R&D focus.
Two Examples of Catastrophic “Human Error”
Some of the most spectacular news stories of the last 50 years exemplify the view that human error in operation, design, maintenance, and/or management underlies disasters.
Many of these examples involve multiple errors. For instance, the proximal cause of the Chernobyl accident was a poorly designed test (a design error) during a scheduled shut-down of a reactor. Safety planning – which simply specified that test operations required the approval of the plant foremen and that in the event of an emergency, operators should follow local instructions – was inadequate for the test.
The decision to run the test without an appropriate safety plan involved an error in management. In addition, the plant operators committed an error by deviating from the test protocols.
One of the most famous of all disasters, the Space Shuttle Challenger explosion, illustrates the potential complexity of “human error.”
No operator error was ever implied in the Challenger disaster. In dramatic testimony, Nobel physicist Richard Feynman demonstrated the ultimate cause of the explosion: fuel leakage past a cold-hardened O-ring (a design error).
Flight managers were aware of some dangers due to cold, but managers, perhaps influenced by media ridicule and political pressures, decided to launch in the cold weather anyway.
Taking it a step further, Tufte (1997) argues convincingly that errors in deciding how to display historical data on O-ring performance in cold weather was critical, concluding “Had the correct scatterplot or data table been constructed, no one would have dared to risk the Challenger in such cold weather” (p. 52).
Why do people
Both errors and error-free actions result from the same basic processes of perception, knowledge and memory, reasoning and decision-making, and response production. As an example, consider this common perceptually-based error. A driver sits at a red light next to a large truck, and the truck begins to inch backward.
Typically, a driver will first make the perceptual error that the car has begun to roll forwards, and then, acting on that erroneous perception, press hard on the brake. Why do we make the perceptual error? Because human perception requires us to compare events in both a spatial and a temporal context.
So, when the space around us moves backward (this is why a big truck that takes up our visual field is more compelling than a little sports car), that usually means we are moving forward.
It’s just this sort of event that leads an operator to misread the absolute value of an indicator on a display, based on the values of indicators nearby in space surrounding that indicator.
Another frequent cause or error is our ability to access our knowledge from memory. For example, most readers will have had the experience of intending to go to one location (say the mall), then continuing past the turn off and going towards the office (a more frequently traveled route). We have the correct route stored in long-term memory, but simply don’t retrieve it at the right time.
Our ability to retrieve information is based largely on the availability of cues that bring the knowledge out of long-term memory into active consciousness.
Thus, a maintenance worker following a memorized set of procedures might skip over a procedure if appropriate, timely cues are missing.
Reasoning and decision-making also depend on having access to the right information at the right time. A common error in problem-solving, the “cognitive set error,” occurs when a set of actions leads to success several times, usually under one set of circumstances.
Then, when the circumstances change and we don’t notice, we continue to apply this set of actions, now inappropriately. A common example might be the homeowner who succeeds in fixing several leaky faucets by replacing worn washers, but who then attempts to replace the washers on a washerless faucet.
Designers may fall victim to this cognitive set. If a design approach has been successful in the past, it often makes sense to try it again. However, if previous designs had experts as the target users but a new design is intended for novice users, a previously-successful design approach might not work.
Finally, even the most expert, skilled performers of a task are prone to action slips - the unconscious (“silly,” “stupid”) errors that we all make. Typically, we produce responses when two conditions are met - when a general response system is activated and when the environment contains the right trigger for the specific response.
Thus, we greet our spouse with the appropriate name if we are thinking about him or her (activation) and he or she is in front of us (trigger). An action slip can occur if the wrong response system is activated, if the activation level even of the right system is very high, of if an incorrect triggering event occurs that closely resembles the correct trigger.
Thus, we may greet our spouse with the wrong name if we are thinking about our manager at work when the spouse appears in front of us.
A highly experienced operator tends to make these action slips — witness the many typos that power typists make, especially in cases like typing “numb” on the way to “numerous,” due to activation of the more familiar “number.”
What can be
done to reduce errors?
There are various ways product and process developers and users can try to reduce or eliminate error. Some bad approaches include blaming the most proximal person (“A good pilot would have recognized that airport to be Cleveland, regardless of what the instruments said.”) and mandating error-free activity (like telling a pitcher “Don’t walk in the tying run.”).
A better approach might be called “training for error”. Regardless of how convoluted the operator interface is (“Hit control-alt-delete to reboot”), if the operators are trained sufficiently, the wondrous human mind can comprehend and recall a huge amount of instruction, encompassing all the correct actions.
An even better approach has been called “designing for error.” Lewis and Norman (1986) encouraged designers to expect user errors, and to design with errors in mind. They suggested that designers should understand the causes of errors, design so as to minimize the causes, make errors easy to discover and correct, and to change their attitudes about errors. “Don’t think of the user as making errors; think of the actions as approximations of what is desired” (Norman, 1990, p. 131).
Good design acknowledges the likelihood of a user error (“Are you sure you want to delete file X”?), makes it easy for the user to recognize that an error has been made (“File X has been deleted.”), and affords the user an easy way to recover from an error (“Undo file deletion.”).
And so . . .
Considering the complex human being as an integral part of any system, and the problematic nature of human perceptual, memory, reasoning, and action subsystems, we are obliged to carry out product and process designs differently.
We assume that human operators will make errors, and design accordingly. Taking it a step further, we assume that humans who build systems, humans who maintain systems, humans who make management decisions about systems, even humans who draw charts of data about systems, will also make errors.
The goal, it would seem, is not to avoid all error (which we cannot), but rather to ensure that the results of a human error can be corrected before they produce disaster.
Lewis, C., & Norman, D. A. (1986). Designing for error. In D. A. Norman & S. W. Draper (Eds.), User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Erlbaum Associates.
Norman, D. A. (1990). The design of everyday things. New York: Doubleday Currency.
Tufte, E. R. (1997). Visual explanations. Cheshire, Connecticut: Graphics Press.
Randolph G. Bias, Ph.D. is a Manager of Usability Engineering BMC Software, Inc. and Douglas J. Gillan, Ph.D. is an Associate Professor of Psychology at New Mexico State University in Las Cruces, NM.
DR World Main Index | Return to DRJ's Homepage
Disaster Recovery World© 1999, and Disaster Recovery Journal©
1999, are copyrighted by Systems Support, Inc. All rights reserved. Reproduction in whole or
part is prohibited without the express written permission form Systems Support, Inc.