How have checklists been used in high risk environments? And what human factors studies form the foundation for the use of checklists?
The concept of the checklist is so simple, and is used in so many ways today that talking of “critical” checklists does not leave much of an impression. Incorporating checklists in high-hazard environments has been one of the most influential innovations to enhance safety in recent times.
The defining moment leading to the innovation of using a formal checklist occurred in 1935.
The Army Air Corps was to award a contract for the next generation bomber, and three companies were bidding for the contract: The Douglas Aircraft Company with the Douglas DB-1, the Glenn L. Martin Company with the Martin 146, and Boeing with the Model 299. It was acknowledged that the Boeing four-engine plane with many new features was by far the better machine and the competition was purely academic. The Model 299 could fly farther, faster, and carry more payload than either of the other two entries.
However, during the demonstration trials the Boeing plane crashed and was destroyed, killing the two expert pilots and seriously injuring the engineers on board. This resulted in Boeing’s disqualification. Boeing’s internal investigation determined the pilots had made an error by not unlocking the wind gust-lock. The gust-lock, one of the new features added, is engaged while on the ground to prevent elevator damage from high-wind situations, but must be released prior to take-off. When the Model 299 took off with the gust-lock engaged the elevators were inoperable.
When the competition demonstration flights had to be re-run because of a technicality, Boeing reentered the Model 299 with only one alteration: Boeing experts developed a series of normal checklists for the pilots to use to ensure that critical tasks were accomplished. As a result, Boeing won the competition and more than 12,000 of what became the B- 17 Flying Fortress aircraft were sold.
This paper examines the use of checklists in various high risk environments, and pre- and post-checklist implementation comparisons. We will also discuss human factors studies that form the foundation for the use of checklists, as well as draw parallels between what implementation teams in other fields have discovered when integrating checklists and the positive impact checklists can have on safety. The issues examined and conclusions documented can be used to support the development of critical checklists throughout any industrial application.
The following categories are indicative of the multiple uses of checklists:
- Procedural checklists: Excessively long, complex, or critical tasks performed only occasionally require “read-and-do” checklists (those checklists not done from memory, but the checklist is read and the various tasks performed sequentially) to ensure specific tasks in both normal and non-normal situations are accomplished.
- Preparation checklists: Multiple-step situations require checklists to ensure all of the variables are performed as desired; e.g. shopping lists, trip planning, group or individual communication sessions.
- Problem-solving checklists: Multiple-point, question type checklists used for troubleshooting complex procedures or tasks to pinpoint what went wrong, areas of difficulty, or solutions to barriers blocking forward movement.
- Prevention checklists: Critical checklists address errors, mistakes, mishaps in high-hazard work areas that can result in injury or death to users or neighbors, destruction of property, or impact to reputation or continued business success.
Each of these different uses has a particular purpose, and each has value and meaning as intended. While the use of checklists is present in almost every industry, extensive studies examining their effectiveness and the science of how to develop effective checklists are almost non-existent except in the medical field, nuclear power, and the aviation industry. Our primary focus in this paper is to examine the effectiveness of prevention checklists in these industries, as well as in mining and rail transportation.
For example, in only the last two decades has the medical industry adopted a serious stance in preventing surgical errors by widespread adoption of using checklists.
In response to a shocking study published in 2000 that reported an estimated 44,000 Americans died each year as a result of preventable medical errors, a medical team at Johns Hopkins Hospital introduced several improvements, included and guided by a checklist termed the Comprehensive Unit Based Safety Program.
The program resulted in a reduction of what is known as central-line associated bloodstream infection, a life-threatening condition, to nearly zero over four years. The Johns Hopkins results has led other medical institutions to implement similar patient safety programs. Additionally, in 2008, when the World Health Organization deployed a Surgical Safety Checklist program, various healthcare systems, cultures and operating venues reported a 53 percent reduction in postoperative mortality and a 64 percent reduction in morbidity.
The following implications, drawn from this analysis of processes undertaken to build a safety culture and in the use of checklists in numerous industries, provides insightful principles and practical strategies on how the development of critical checklists can have a positive impact on the development of a safety culture:
- Understanding checklist use
- The use of checklists will not eliminate all accidents in the lab, test facility, production line or wherever they are implemented. Accidents will still happen even when users make decisions consistent with best practices because risk cannot be completely eliminated.
- Training classes and feedback can positively change personal behavior. Practical issues encountered during the implementation of checklists can be minimized by effective training.
- All employees must make decisions as to what steps, if any, they need to take based on a mental representation of the factors known or hypothesized in the ever-changing or semi-constant state of their equipment at any given point in time.
- “No repetition of incidents” must be a guiding principle, and if an incident occurs it is necessary to learn from it to prevent the same incident from happening in the future.
- Internalization and generalization is central to the process of change. A change in communication can get people talking about safety differently than they communicated before an effective safety awareness program was implemented.
- Emphasis must be placed on the importance of checklist use with reminders of situations where deviations from checklists occur and how they can be misused.
- Organizational changes that support checklist use
- It is crucial that performance evaluation and theoretical rhetoric proclaim the same message.
- Engaged senior leadership is essential to sustaining a culture of safety and the involvement of all levels of staff is critical to a successful safety and checklist initiative roll-out.
- Safety and checklists must be ingrained in organizational culture until everyone can say that checklists are just “a part of how we do business.”
- One key to organizational success in the use of checklists is to continue to search for errors, know their error rates, monitor them after implementing safety innovations, and give feedback to all staff so they know the science behind the checklists is valid. The use of checklists is not the end-game; reduced error rates are.
- A key feature of a successful safety and checklist initiative is its rejection of a command-and-control regime where workers are simply told they are to use a checklist and expected to go and do it. Instead, they need to be encouraged to develop checklists that fit their own unique challenges and culture.
- To improve safety, it is critical that we shift from a perception where the prime cause of accidents is user errors to thinking of errors as the consequence of many factors that combine to create conditions for accidents, and how to reduce those conditions.
- A zero mindset to injuries, accidents, and occupational illnesses is necessary, and a belief that all injuries and occupational illnesses are preventable.
- Committed peer champions placed as “safety expert” in labs, test facilities, or manufacturing plants will stimulate discussions on safety.
- Benefits of checklists
- Ensuring that all critical tasks are carried out.
- Encouraging a non-hierarchical team-based approach.
- Enhancing communication, catching near misses or potential complications early enough to correct them, and encouraging the use of technologies to manage anticipated and unanticipated complications.
- Workers will feel more empowered to control their own safety, more comfortable with their safety-related choices, and are more likely to confront anyone asking them to perform an unsafe procedure when a safer one is available.
Checklists are a good way of making certain that tasks get done, but determining the best way of proceeding in a complex operational setting is to acknowledge expertise and experience as an essential foundation to the use of checklists.
Pre-planned procedures and checklists cannot replace the necessity of users bringing to bear diagnostic and response strategies in real-time based on their experience, collaboration and attentiveness when the confluence of influencing factors on any given piece of equipment, test or facility requires it.
To read and download the complete Boeing Technical Journal paper titled “An Analysis of the Effectiveness of Checklists when combined with Other Processes, Methods and Tools to Reduce Risk in High Hazard Activities,” click here.
By By William Y. Higgins and Daniel J. Boorman