The Federal Aviation Administration (FAA) took another step toward a new safety culture by reducing the emphasis on blame in the reporting of operational errors by air traffic controllers. “We’re moving away from a culture of blame and punishment,” said FAA Administrator Randy Babbitt. “It’s important to note that controllers remain accountable for their actions, but we’re moving toward a new era that focuses on why these events occur and what can be done to prevent them.” Effective immediately, the names of controllers will not be included in reports sent to FAA headquarters on operational errors…. Removing names on the official report will allow investigators to focus on what happened rather than who was at fault. Link to FAA press release.
As described in the linked article, workers at Electricite de France are raising concerns about conflicting pressures to work faster, achieve higher capacity factors and provide competitive electricity. EdF has long held a very high reputation for its nuclear operations, in part attributed to the national government’s central ownership and operating responsibilities. While it remains to be seen the extent of such concerns, it is apparent that central ownership does not provide a shield against many of the same pressures experienced by U.S. plants. The article also highlights the potential complications of heavy reliance on subcontractors if it leads to the loss of core competencies in the host organization.
As mentioned in a prior post we will be highlighting some of the work of the late Bernhard Wilpert, a leading figure in research on the role of human behavior in high reliability organizations.
Professor Wilpert emphasized the interaction of human, technology, and organizational dynamics. His tools for human factors event analysis have become the standard practice in German and Swiss nuclear plants. He is the author of several leading books including Safety Culture in Nuclear Power Operations; System Safety: Challenges; Pitfalls of Intervention; Emerging Demands for Nuclear Safety of Nuclear Power Operations: Challenge and Response; and Nuclear Safety: A Human Factors Perspective. Professor Wilpert was also a principal contributor to the LearnSafe project conducted in Europe from 2001 – 2004. See the following link for information about the project team and its results and look to us for future posts on the LearnSafe research. Link to LearnSafe project.
To delve a little deeper into this question, on Slide 10 of the NEI presentation there is a typical summary graphic of assessment results. The chart catalogs the responses of members of the organization by the eight INPO principles of safety culture. This summary indicates a variety of responses to the individual principles – for 3 or 4 of the principles there seems to be a fairly strong consensus that the right things are happening. But 5 of the 8 principles show greater than a 20 score negative responses and 2 of the principles show greater than a 40 score negatives.
First, what can or should one conclude about the overall state of safety culture in this organization given these results? One wonders if these results were shown to a number of experts, whether their interpretations would be consistent or whether they would even purport to associate the results with a finding. As discussed in a prior post, this issue is fundamental to the nature of safety culture, whether it is amenable to direct measurement, and whether assessment results really say anything about the safety health of the organization.
But the more particular question for this post is whether an assessment can detect complacency in an organization and its potential for latent risk to the organization’s safety performance. In a post dated July 30, 2009 I referred to the problems presented by complacency, particularly in organizations experiencing few operational challenges. That environment can be ripe for a weak culture to develop or be sustained. Could that environment also bias the responses to assessment questions, reinforcing the incorrect perception that safety culture is healthy? It may be that this type of situation is of most relevance in today’s nuclear industry where the vast majority of plants are operating at high capacity factors and experiencing few significant operational events. It is not clear to this commentator that assessments can be designed to explicitly detect complacency, and even the use of assessment results in conjunction with other data (data likely to look normal when overall performance is good) may not be credible in raising an alarm.
The famous words of Gertrude Stein are most often associated with the notion that when all is said and done, a thing is what it is. We offer this idea as we continue to look at the meaning of safety culture assessment results – are the results just the results, or do they signify some meaning or interpretation beyond the results?
To illustrate some of the issues I will use an NEI presentation made to the NRC on February 3, 2009. On Slide 2 there is a statement that the USA methodology (for safety culture surveys and assessments) has been used successfully for five years. One question is what does it mean that an assessment was successful? The intent is not to pick on this particular methodology but to open the question of exactly what is the expected result of performing an assessment.
It may be that “successful” means that the organizations being assessed have found the process and results to be useful or interesting, e.g., by stimulating discussion or furthering exploration of issues associated with the results. There are many, myself included, who believe anything that stimulates an organization to discuss and contemplate safety management issues is beneficial. On the other hand it may be that organizations (and regulators??) believe assessments are successful because they can use the results to make a determination that a safety culture is “acceptable” or “strong” or “needs improvement”. Can assessments really carry the weight of this expectation? Or is a rose just a rose?
Slide 11 highlights these questions by indicating a validation of the assessment methodology is to be carried out. “Validation” seems to suggest that assessments mean something beyond their immediate results. It may also suggest that assessment results can be compared to some “known” value to determine whether the assessment accurately measured or predicted that value. We will have to wait and see what is intended and how the validation is performed. At the same time we will be keeping in mind the observation of Professor Wilpert in my post of August 17, 2009 that “culture is not a quantifiable phenomenon”.
A topic that we will visit regularly is the use of safety culture assessments to assign quantitative values to the condition of a specific organization and even the individual departments and working groups within the organization. One reason for this focus is the emphasis on safety culture assessments as a response to situations where organizational performance does not meet expectations and “culture” is believed to be a factor. Both the NRC and the nuclear industry appear aligned on the use of assessments as a response to performance issues and even as an ongoing prophylactic tool. But, are these assessments useful? Or accurate? Do they provide insights into the origins of cultural deficiencies?
One question that frequently comes to mind is, can safety culture be separated from the manifestation of culture in terms of the specific actions and decisions taken by an organization? For example, if an organization makes some decisions that are clearly at odds with “safety being the overriding priority”, can the culture of the organization not be deficient? But if an assessment of the culture is performed, and the espoused beliefs and priorities are generally supportive of safety, what is to be made of those responses?
The reference material for this post comes from some work led by the late Bernhard Wilpert of the Berlin University of Technology. (We will sample a variety of his work in the safety management area in future posts.) It is a brief slide presentation titled, “Challenges and Opportunities of Assessing Safety Culture”. Slide 3 for example revisits E. H. Schein’s multi-dimensional formulation of safety culture which suggests that assessments must be able to expose all levels of culture and their integrated effect.
Two observations from these slides seem of particular note. They are both under Item 4, Methodological Challenges. The first observation is that culture is not a quantifiable phenomenon and does not lend itself easily to benchmarking. This bears consideration as most assessment methods being used today employ some statistical comparisons to assessments at other plants, including percentile type ranking. The other observation in the slide is that culture results from the learning experience of its members. This is of particular interest to us as it supports some of the thinking associated with a systems dynamics approach. A systems view involves the development of shared “mental models” of how safety management “works”; the goal being that individual actions and decisions can be understood within a commonly understood framework. The systems process becomes, in essence, the mechanism for translating beliefs into actions.
