Safety Culture in the DOE Complex

This post reviews a Department of Energy (DOE) effort to provide safety culture assessment and improvement tools for its own operations and those of its contractors.

Introduction

The DOE is responsible for a vast array of organizations that work on DOE’s programs.  These organizations range from very small to huge in size and include private contractors, government facilities, specialty shops, niche manufacturers, labs and factories.  Many are engaged in high-hazard activities (including nuclear) so DOE is interested in promoting an effective safety culture across the complex.

To that end, a task team* was established in 2007 “to identify a consensus set of safety culture principles, along with implementation practices that could be used by DOE . . .  and their contractors. . . . The goal of this effort was to achieve an improved safety culture through ISMS [Integrated Safety Management System] continuous improvement, building on operating experience from similar industries, such as the domestic and international commercial nuclear and chemical industries.”  (Final Report**, p. 2)

It appears the team performed most of its research during 2008, conducted a pilot program in 2009 and published its final report in 2010.  Research included reviewing the space shuttle and Texas City disasters, the Davis-Besse incident, works by gurus such as James Reason, and guidance and practices published by NASA, NRC, IAEA, INPO and OSHA.

Major Results

The team developed a definition of safety culture and described a process whereby using organizations could assess their safety culture and, if necessary, take steps to improve it.

The team’s definition of safety culture:

“An organization’s values and behaviors modeled by its leaders and internalized by its members, which serve to make safe performance of work the overriding priority to protect the workers, public, and the environment.” (Final Report, p. 5)

After presenting this definition, the report goes on to say “The Team believes that voluntary, proactive pursuit of excellence is preferable to regulatory approaches to address safety culture because it is difficult to regulate values and behaviors. DOE is not currently considering regulation or requirements relative to safety culture.” (Final Report, pp. 5-6)

The team identified three focus areas that were judged to have the most impact on improving safety and production performance within the DOE complex: Leadership, Employee/Worker Engagement, and Organizational Learning. For each of these three focus areas, the team identified related attributes.

The overall process for a using organization is to review the focus areas and attributes, assess the current safety culture, select and use appropriate improvement tools, and reinforce results. 

The list of tools to assess safety culture includes direct observations, causal factors analysis (CFA), surveys, interviews, review of key processes, performance indicators, Voluntary Protection Program (VPP) assessments, stream analysis and Human Performance Improvement (HPI) assessments.***  The Final Report also mentioned performance metrics and workshops. (Final Report, p. 9)

Tools to improve safety culture include senior management commitment, clear expectations, ISMS training, managers spending time in the field, coaching and mentoring, Behavior Based Safety (BBS), VPP, Six Sigma, the problem identification process, and HPI.****  The Final Report also mentioned High Reliability Organization (HRO), Safety Conscious Work Environment (SCWE) and Differing Professional Opinion (DPO). (Final Report, p. 9)  Whew.

The results of a one-year pilot program at multiple contractors were evaluated and the lessons learned were incorporated in the final report.

Our Assessment

Given the diversity of the DOE complex, it’s obvious that no “one size fits all” approach is likely to be effective.  But it’s not clear that what the team has provided will be all that effective either.  The team’s product is really a collection of concepts and tools culled from the work of outsiders, combined with DOE’s existing management programs, and repackaged as a combination of overall process and laundry lists.  Users are left to determine for themselves exactly which sub-set of tools might be useful in their individual situations.

It’s not that the report is bad.  For example, the general discussion of safety culture improvement emphasizes the importance of creating a learning organization focused on continuous improvement.  In addition, a major point they got right was recognizing that safety can contribute to better mission performance.  “The strong correlation between good safety performance with good mission performance (or productivity or reliability) has been observed in many different contexts, including industrial, chemical, and nuclear operations.” (Final Report, p. 20)

On the other hand, the team has adopted the works of others but does not appear to recognize how, in a systems sense, safety culture is interwoven into the fabric of an organization.  For example, feedback loops from the multitude of possible interventions to overall safety culture are not even mentioned.  And this is not a trivial issue.  An intervention can provide an initial boost to safety culture but then safety culture may start to decay because of saturation effects, especially if the organization is hit with one intervention after another.

In addition, some of the major, omnipresent threats to safety culture do not get the emphasis they deserve.  Goal conflict, normalization of deviance and institutional complacency are included in a list of issues from the Columbia, Davis-Besse and Texas City events (Final Report, p. 13-15) but the authors do not give them the overarching importance they merit.  Goal conflict, often expressed as safety vs mission, should obviously be avoided but its insidiousness is not adequately recognized; the other two factors are treated in a similar manner. 

Two final picky points:  First, the report says it’s difficult to regulate behavior.  That’s true but companies and government do it all the time.  DOE could definitely promulgate a behavior-based safety culture regulatory requirement if it chose to do so.  Second, the final report (p. 9) mentions leading (vs lagging) indicators as part of assessment but the guidelines do not provide any examples.  If someone has some useful leading indicators, we’d definitely like to know about them. 

Bottom line, the DOE effort draws from many sources and probably represents consensus building among stakeholders on an epic scale.  However, the team provides no new insights into safety culture and, in fact, may not be taking advantage of the state of the art in our understanding of how safety culture interacts with other organizational attributes. 


*  Energy Facility Contractors Group (EFCOG)/DOE Integrated Safety Management System (ISMS) Safety Culture Task Team.

**  J. McDonald, P. Worthington, N. Barker, G. Podonsky, “EFCOG/DOE ISMS Safety Culture Task Team Final Report”  (Jun 4, 2010).

***  EFCOG/DOE ISMS Safety Culture Task Team, “Assessing Safety Culture in DOE Facilities,” EFCOG meeting handout (Jan 23, 2009).

****  EFCOG/DOE ISMS Safety Culture Task Team, “Activities to Improve Safety Culture in DOE Facilities,” EFCOG meeting handout (Jan 23, 2009).

BP Exec Quit Over Safety Before Deepwater Disaster

Today’s Wall Street Journal has an interesting news item about a BP Vice President who quit prior to the Deepwater Horizon disaster because he felt BP "was not adequately committed to improving its safety protocols in offshore drilling to the level of its industry peers." The full article is available here.

“what people do, not why they do it…”

Our perseverance through over three hours of the web video of the Commission meeting on the proposed safety culture policy statement was finally rewarded in the very last minute of discussion.  Commissioner Apostolakis reiterated some of his concerns with the direction of the policy statement, observing that the NRC is a performance-based agency and:

“...we really care about what people do and maybe not why they do it….”

Commissioner Apostolakis was amplifying his discomfort with the inclusion of values along with behaviors in the policy as values are inherently fuzzy, not measurable, and may or may not be a prerequisite to the right behaviors.  Perhaps most of all, he believed omitting the reference to core values would not detract from the definition of safety culture. 

Earlier in the meeting Commissioner Apostolakis had tried to draw out the staff on whether the definition of safety culture needed values in addition to behaviors [at time 2:34:58], and would it be a fatal flaw to omit “core values”.  The staff response was illuminating.  The justification offered for retaining values was “stakeholder consensus”, and extensive outreach efforts that supported inclusion.  (But why was it so important to stakeholders?)  The staff went on to clarify: “culture does not lend itself to be inspectable”, but “having values with behaviors is what culture is all about”.   Frankly we’re not sure what that means, but we do know that safety culture behaviors are inspectable because they are observable and measurable.

That much of the staff’s justification for including values in the policy statement seemed to reside in the fact that all the stakeholders had agreed received positive endorsement by Chairman Jaczko when he observed:  “...Commissioner Magwood I think made a profound point that there was value in this process here that may be tremendously more important than the actual policy statement was the fact that people got together and started talking about this and realized that across this wide variety of stakeholders, there was pretty good agreement about the kinds of things that we were talking about.”

Chairman Jaczko also weighed in on the values-behaviors contrast, coming down firmly on the inclusion of values and offering the following justification:

“...not all entities with a good safety culture will have necessarily the right values…”

Respectfully, we believe at a minimum this will further confuse the NRC’s policy on safety culture, and in all likelihood places emphasis in exactly the wrong place.  Is the Chairman agreeing all that matters is what people do?  Or is he suggesting that the NRC would find fault with a licensee that was acting consistent with safety but did not manifest the “right” values.  And how would the NRC reach such a finding?  More fundamentally, isn’t Commissioner Apostolakis correct in his blunt statement - that we [NRC] don’t care why they [licensees] do it?

More Hope

Our prior post highlighted a comment early in the January 24, 2011 Commission meeting to review the proposed policy statement on nuclear safety culture. 

In the context of her advocacy for regulations in addition to a policy statement, attorney Billie Garde stated she “hoped” that proceeding with just a policy statement was the right decision.  We thought her warning of the fallout from a possible future nuclear event would get some attention.  It did, at least with Commissioner Svinicki who sought some clarification of Garde’s concern.  Just prior to this clip, Svinicki had observed that in her mind a policy statement can’t supplant an appropriate regulatory framework in terms of compelling certain behaviors.  No matter what you think about the appropriateness of a policy statement versus other regulatory actions, Garde is certainly correct that the question will be asked in the future: Did the NRC do enough?

“I Hope For All Our Sakes This is Right”

On January 24, 2011 the NRC Commissioners met to review the proposed policy statement on nuclear safety culture developed by the NRC staff. This most recent effort was chartered by the Commission more than 3 years ago and represents the next step in the process to publish the proposed statement for public comment.

“25 years is long enough to build a policy statement…” for nuclear safety culture. This observation by Billie Garde* in her opening remarks to the Commissioners, with her timeline referring to the Chernobyl and space shuttle Challenger accidents in 1986. She also emphasized that the need was to now focus on implementation of the policy statement. She maintained her position that a policy statement alone would not be sufficient and that regulation would be necessary to assure consistent and reliable implementation.

In that regard she lays claim to one of the more disconcerting observations made at the meeting, the gist of which can be summed up as, “I hope for all our sakes this is right…”

Here’s the video clip with the exchange between Garde and Commissioner Apostolakis.

We will be following up with additional posts with highlights from the Commission session.

*  Billie Garde is an attorney in Washington, D.C.  Her NRC website bio is here.

Culture de la Sécurité

If you are paying attention you noticed we’re using French words.  And the reason is the current news regarding Air France and its safety practices and safety culture.

The principal finding of an independent study of the airline’s operations was a lack of “strong safety leadership at all levels of management" as reported in a January 26 Wall Street Journal (WSJ) article.*

While Air France has refused to make the report public, a review by the WSJ stated that the study was “sharply critical of broad aspects of the safety culture”.

Over the previous several days there have been articles in the WSJ preceding and attending the completion of the independent study.  The first of the articles** previewed some of the findings and provided favorable commentary based on an understanding that Air France would be making the report public (“Air France to Disclose…”).  This was characterized as a move toward greater openness on the part of the airline and commented, “safety experts said it was unusual for a large company, especially an airline, to give outsiders such latitude publicly to expose gaps in safety systems.”  And quoting Bill Voss, president of the Flight Safety Foundation, a global air-safety advocacy organization based in Alexandria, Va., "It's extraordinary that they are willing to release the results," said Mr. Voss. The process "gives me confidence there will be follow-through."

Well, in a follow-up article the next day, when receipt of the study results was announced by Air France, the reaction was disappointment as the airline indicated it would not release the report or its recommendations.  Nonetheless the airline trumpeted its actions:

“According to an Air France news release, the report said that creating the outside review team ‘in a public manner and providing it a broad charter’ to examine flight safety ‘was a courageous act’ and an example of safety leadership ‘rarely seen in today's international aviation industry.’” ***

In deciding on whether Air France should be awarded the Légion d'honneur, one might also consider that the current safety study was undertaken in the wake of four serious crashes between 1999 and 2009 (the most recent being the flight from Rio to Paris that went down mid-Atlantic).  In addition, it follows a prior safety study,

“Finished in 2006, that report was distributed to more than 4,000 company pilots and was widely praised for its frankness about shortcomings within the carrier. Although Air France has said its executive committee made formal decisions to implement the report's recommendations, critics of the airline continue to maintain management didn't aggressively pursue the changes.” (Jan 24, 2011)

I think we’ve seen this before.  Think about the safety studies after the Challenger crash, but not really implemented and then the disintegration on reentry of the Columbia.  Or BP and the Texas oil refinery fire followed by the Deepwater Horizon last summer.  Obviously safety assessments, no matter how strong and how independent, ultimately require the subject organization to implement changes.  We think the current Air France report correctly fingers safety leadership by management “starting at the top”.  And it never fails in these situations that top management describes as its highest priority…...can anybody guess……that’s right, it’s “safety first”.  (Jan 26, 2011)

Perhaps if it was safety first Air France might be releasing the study and its recommendations.  Wouldn’t that help make real its safety priority and wouldn’t such transparency help ensure that the recommendations are actually implemented?   We have commented in prior posts on transparency and we will continue to emphasize its importance to safety culture across all industries.


*  A. Pasztor, D. Michaels and D. Gauthier-Villars, “Air France Panel Cites Wide Safety Deficiencies,” WSJ.com (Jan 26, 2011).

** A. Pasztor and D. Michaels, “Air France to Disclose Review's Criticisms,” WSJ.com (Jan 24, 2011).

***  A. Pasztor, “Air France Enhances Safety Efforts,” later re-headlined “Air France Withholds Key Report,” WSJ.com (Jan 25, 2011).

A Nuclear Model for Oil and Gas

The President’s Commission has issued its report on the Deepwater Horizon disaster.* The report reviews the history of the tragedy and makes recommendations based on lessons learned.  This post focuses on the report’s use of the nuclear industry, in particular the role played by INPO, as a model for an oil and gas industry safety institute and auditor.

The report provides an in-depth review of INPO’s role and methods and we will not repeat that review in this space.  We want to highlight the differences between the oil and gas and nuclear industries, some recognized in the report, that would challenge a new safety auditor. 

First, “The oil and gas industry is more fragmented and diversified in nature. . . .” (p. 240)  The industry includes vertically integrated giants, specialty niche firms and everything in-between.  Some are global in nature while others are regional firms.  In our view, it appears that oil and gas industry participants cooperate with each other in certain instances and compete with each other in different cases.  (In contrast, most [all?] U.S. nuclear plants are not in direct competition with other plants.)  Obtaining agreement to create a relatively powerful industry auditing entity will not be a simple matter.    

Second, “concerns about potential disclosure to business competitors of proprietary information might make it harder to establish an INPO-like entity in the oil and gas industry.” (p. 240)  Oil and gas firms regard technology as an important source of competitive advantage.  “[A]n INPO-like approach might run into problems if companies perceived the potential for inspections of offshore facilities to reveal ‘technical and proprietary and confidential information that companies may be reluctant to share with one another.’” (p. 241)  Not only will it be difficult to get a firm to share its proprietary technology if it may lose competitive advantage by doing so, but this will make it more difficult for the auditing organization to promote the industry-wide use of the most effective, safest technologies

Third, and this could be a potentially large problem, INPO operates in almost total secrecy.  “[INPO] assessment results are never revealed to anyone other than the utility CEOs and site managers, but INPO formally meets with the NRC four times a year to discuss trends and information of “mutual interest.” And if INPO has discovered serious problems associated with specific plants, it notifies the NRC.”  (p. 236)  INPO claims, probably realistically, that maintaining member confidentiality is key to obtaining full and willing cooperation in evaluations. 

However, this secrecy contributes zero to public understanding of and support for nuclear plant operations and owners.  At this point in its evolution, the oil and gas industry needs more transparency in its auditing and oversight functions, not less.  After all, and forgive the bluntness here, very few people have died at U.S. commercial nuclear power plants (and those were in non-nuclear incidents) while the oil and gas industry has suffered numerous fatalities.  We think a government auditor, whose evaluations of facilities and managements would be made public, is the better answer for the oil and gas industry at this time.


*  National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling, “Deep Water: The Gulf Oil Disaster and the Future of Offshore Drilling,” Report to the President (Jan 2011).