Monday, August 30, 2010

Experiencing Decisions

With this post we’re back to a topic that is high on our list of important issues for ensuring a strong safety culture - and one that doesn’t seem to get a lot of attention by the industry and regulators.  It is the critical importance of how safety values are integrated into decision making processes, particularly for decisions that involve balancing other business priorities and where the safety implications are not clear cut.   

An article in the Wall Street Journal on August 23, 2010* provides the perspective of educators at a top business school on teaching values to their students.  The concern arises when values are taught in the abstract and reliance is placed on commitments to high ethics without the contextual conflicts that will arise in the real world.  “The power of the situation, and our too frequent disregard for it, is an overarching lesson from sociology and social psychology. Situational forces drive behavior to a surprising extent, much more than expected by those who believe character determines all.”

Isn’t this also true for the nuclear industry?

The authors, professors at Yale’s School of Management, observe, “This [traditional leadership courses] leaves the connection between values, leadership and action underdeveloped.”  Students “...must be brought face-to-face with the pressures that profit-maximization will create for them.”

How can students appreciation of decision making challenges be made more realistic and compelling?  The authors believe “We must make them aware that these decisions will challenge their values…... We need to make sure they engage in a continuing dialogue….We have found this is best achieved through experiential learning.”

The term “experiential learning” certainly caught our eye as we believe it is consistent with our own focus on developing simulation tools to provide a realistic decisionmaking environment - where trainees can practice managing the conflicts they will inevitably confront on the job. 

*"Promises Aren't Enough: Business Schools Need to Do a Better Job Teaching Students Values," Rodrigo Canales, B. Cade Massey and Amy Wrzesniewski, The Wall Street Journal, August 23, 2010.

Friday, August 27, 2010

Safety Climate Surveys (Part 2)

On August 23, 2010 we posted on a paper* reporting on a safety climate survey conducted at a number of off-shore oil facilities.  We noted that paper presented a rigorous analysis of the survey data and also discussed the limitations of the data and the analysis.  Our Bob Cudlin has been in contact with the paper’s lead author who provided a candid assessment of how survey data should be used.

In a private message to Bob, Dr. Mearns gave a general warning against over-inference from survey data and findings.  She says, “I think safety climate surveys have their place but they need to be done properly and unfortunately, many attempts to measure safety climate are poorly executed.   The data obtained from surveys are simply numbers but they don’t tell you much about what is actually going on within the organisation or the team regarding safety.  I see the survey results as a ‘temperature check’ but it needs a more detailed diagnosis to find out what really ails the safety culture.” 

We couldn’t have said it better ourselves. 

*  Mearns K, Whitaker S & Flin R, “Safety climate, safety management practices and safety performance in offshore environments.”  Safety Science 41(8) 2003 (Oct) pp 641-680.

Monday, August 23, 2010

Safety Climate Surveys (Part 1)

In our August 11, 2010 post we quoted from a paper* addressing safety culture on off-shore oil facilities.  While the paper is a bit off-topic for SafetyMatters (the focus is more on industrial safety and individual, as opposed to group, perceptions), it provides a very good example of how safety climate survey data should be collected and rigorously analyzed, and hypotheses tested.  In addition, one of the findings is quite interesting.

The researchers knew from survey data which respondents had experienced an accident at a facility (not just those facilities where they were currently working), and which respondents had not.  They also knew which of the surveyed facilities had a historically higher proportion of accidents and which had a lower proportion.  “In this case, . . . respondents who had not experienced an accident provided significantly less favorable scores on installations with low accident proportions. Additionally, respondents who had experienced an accident provided significantly less favorable scores on installations with high accident proportions.” (p. 656)  In other words, workers who had been through an accident recognized a relatively safer (riskier) environment better than workers who had not.  While this is certainly more evidence that experience is the best teacher, we think it might have an implication for the commercial nuclear industry.

Unlike offshore oil workers, the overwhelming majority of nuclear power plant employees have never experienced a significant incident (we’re excluding ordinary personnel mishaps).  Thus, their work experience is of limited use in helping them assess just how strong their safety culture actually is.  Does this make these employees more vulnerable to complacency or slowing running off the rails a la NASA?  

*  Mearns K, Whitaker S & Flin R, “Safety climate, safety management practices and safety performance in offshore environments.”  Safety Science 41(8) 2003 (Oct) pp 641-680.

Thursday, August 19, 2010

NRC Chairman on Safety Culture

NRC Chairman Jaczko recently gave a speech* where he presented three ongoing challenges to the nuclear industry: knowledge management, safety culture and public outreach.  We applaud the chairman for continuing to emphasize safety culture.  He indicated it was important to have a strong safety culture, all industry participants need to have a consistent focus on safety, and complacency is a potential pitfall.

Unfortunately, he plowed no new ground in his remarks.  We think the chairman should have mentioned the potential for lessons learned from recent disasters in other industries.  In addition, with the U.S. nuclear renaissance trying to get underway, we believe he could have pointed out the need for a strong safety culture at all the players who will be involved in building new plants: designers, fabricators, suppliers, contractors, owners, and many others.

*  Jaczko, G.B., “Focus on Regulation.” Prepared Remarks for the Goizueta Directors Institute, Atlanta, GA, (NRC S-10-031) August 10, 2010.

Monday, August 16, 2010

SafetyMatters

“Jim Reason defines a safety culture as consisting of constellations of practices, most importantly, concerning reporting and learning. A safety culture, he argues, is both a reporting culture and a learning culture. Where safety is an organisation’s top priority, the organisation will aim is to assemble as much relevant information as possible, circulate it, analyse it, and apply.” *

This quote comes from a paper [see cite below] we recently posted about and reminds us of our purpose in writing this blog.

We have posted about Reason’s insights into safety culture and find his work to be particularly useful.  We hesitate to exploit such insights for our own purposes but we can hardly resist noting that one of the core purposes of our SafetyMatters Blog is to share and disseminate useful thinking about safety management issues.  We wonder if nuclear operating organizations take advantage of these materials and assure their dissemination within their organizations.  Ditto for nuclear regulators and other industry organizations.  We observe the traffic to the blog and note that individuals within such organizations are regular visitors.  We hope they are not the exceptions or the only people within their organizations to have exposure to the exchange of ideas here at SafetyMatters. 

Comments are always welcome.

*  Andrew Hopkins, "Studying Organisational Cultures and their Effects on Safety," paper prepared for presentation to the International Conference on Occupational Risk Prevention, Seville, May 2006 (National Research Centre for OHS Regulation, Australian National University), p. 16.

Wednesday, August 11, 2010

Down Under Perspective on Surveys

Now from Australia we have come across more research results related to some of the key findings we discussed in our August 2, 2010 post “Mission Impossible”. Recall from that post that research comparing the results of safety surveys prior to a significant event at an offshore oil platform with post-event investigations, revealed significant differences in cultural attributes.

This 2006 paper* draws on a variety of other published works and the author’s own experience in analyzing major safety events. Note that the author refers to safety culture surveys as “perception surveys”, since they focus on people’s perceptions of attitudes, values and behaviors.

“The survey method is well suited to studying individual attitudes and values and it might be thought that the method is thereby biased in favour of a definition of culture in these terms. However, the survey method is equally suited to studying practices, or ‘the way we do things around here’. The only qualification is that survey research of “the way we do things around” here necessarily measures people’s perceptions rather than what actually happens, which may not necessarily coincide.” (p.5) As we have argued, and this paper agrees, it is actual behaviors and outcomes that are most important. The question is, can actual behaviors be discerned or predicted on the basis of surveys? The answer is not clear.

“The question of whether or how the cultures so identified [e.g., by culture surveys] impact on safety is a separate question. Mearns and co-workers argue that there is some, though rather limited, evidence that organisations which do well in safety climate surveys actually have fewer accidents” (p. 14 citing Mearns et al)**

I kind of liked a distinction made early on in the paper, that it is better to ascertain an organization’s “culture” and then assess the impact of that culture on safety, then to directly assess “safety culture”. This approach emphasizes the internal dynamics and the interaction of values and safety priorities with other competing business and environmental pressures. As this paper notes, “. . .the survey method tells us very little about dynamic processes - how the organisation goes about solving its problems. This is an important limitation. . . .Schein makes a similar point when he notes that members of a culture are most likely to reveal themselves when they have problems to solve. . . .(p. 6)

*  Andrew Hopkins, "Studying Organisational Cultures and their Effects on Safety," paper prepared for presentation to the International Conference on Occupational Risk Prevention, Seville, May 2006 (National Research Centre for OHS Regulation, Australian National University).

**  Mearns K, Whitaker S & Flin R, “Safety climate, safety management practices and safety performance in offshore environments”. Safety Science 41(8) 2003 (Oct) pp 641-680.

Monday, August 2, 2010

Mission Impossible

We are back to the topic of safety culture surveys with a new post regarding an important piece of research by Dr. Stian Antonsen of the Norwegian University of Science and Technology.  He presents an empirical analysis of the following question:

    “..whether it is possible to ‘predict’ if an organization is prone to having major accidents on the basis of safety culture assessments.”*

We have previously posted a number of times on the use and efficacy of safety culture assessments.  As we observed in an August 17, 2009 post, “Both the NRC and the nuclear industry appear aligned on the use of assessments as a response to performance issues and even as an ongoing prophylactic tool.  But, are these assessments useful?  Or accurate?  Do they provide insights into the origins of cultural deficiencies?”

Safety culture surveys have become ubiquitous across the U.S. nuclear industry.  This reliance on surveys may be justified, Antonsen observes, to the extent they provide a “snapshot” of “attitudes, values and perceptions about organizational practices…”  But Antonsen cautions that the ability of surveys to predict organizational accidents has not been established empirically and cites some researchers who suspect surveys “‘invite respondents to espouse rationalisations, aspirations, cognitions or attitudes at best’ and that ‘we simply don’t know how to interpret the scales and factors resulting from this research’”.  Furthermore, surveys present questions where the favorable or desired answers may be obvious.  “The risk is, therefore, that the respondents’ answers reflect the way they feel they should feel, think and act regarding safety, rather than the way they actually do feel, think and act…”  As we have stated in a white paper** on nuclear safety management, “it is hard to avoid the trap that beliefs may be definitive but decisions and actions often are much more nuanced.”

To investigate the utility of safety culture surveys Antonsen compared results of a safety survey conducted of the employees of an offshore oil platform (Snorre Alpha) prior to a major operational incident, with the results of detailed investigations and analyses following the incident.  The survey questionnaire included twenty questions similar to those found in nuclear plant surveys.  Answers were structured on a six-point Likert scale, also similar to nuclear plant surveys.  The overall result of the survey was that employees had a highly positive view of safety culture on the rig.

The after incident analysis was performed by the Norwegian Petroleum Safety Authority  and a causal analysis was subsequently performed by Statoil (the rig owner) and a team of researchers.  The findings from the original survey and the later incident investigations were “dramatically different” as to the Snorre Alpha safety culture.  Perhaps one of the telling differences was that the post hoc analyses identified that the rig culture included meeting production targets as a dominant cultural value.  The bottom line finding was that the survey failed to identify significant organizational problems that later emerged in the incident investigations.

Antonsen evaluates possible reasons for the disconnect between surveys and performance outcomes.  He also comments on the useful role surveys can play; for example inter-organizational comparisons and inferring cultural traits.  In the end the research sounds a cautionary note on the link between survey-based measures and the “real” conditions that determine safety outcomes.

Post Script: Antonsen’s “Mission Impossible” paper was published in December 2009.  We now have seen another oil rig accident with the recent explosion and oil spill from BP’s Deepwater Horizon rig.  As we noted in our July 22, 2010 post, a safety culture survey had been performed of that rig’s staff several weeks prior to the explosion with overall positive results.  The investigations of this latest event could well provide additional empirical support for the "Mission Impossible" study results. 

* The study is “Safety Culture Assessment: A Mission Impossible?”  The link connects to the abstract; the paper is available for purchase at the same site.

**  Robert L. Cudlin, "Practicing Nuclear Safety Management" (March 2008), p. 3.