Thursday, March 3, 2011

Safety Culture in the DOE Complex

This post reviews a Department of Energy (DOE) effort to provide safety culture assessment and improvement tools for its own operations and those of its contractors.

Introduction

The DOE is responsible for a vast array of organizations that work on DOE’s programs.  These organizations range from very small to huge in size and include private contractors, government facilities, specialty shops, niche manufacturers, labs and factories.  Many are engaged in high-hazard activities (including nuclear) so DOE is interested in promoting an effective safety culture across the complex.

To that end, a task team* was established in 2007 “to identify a consensus set of safety culture principles, along with implementation practices that could be used by DOE . . .  and their contractors. . . . The goal of this effort was to achieve an improved safety culture through ISMS [Integrated Safety Management System] continuous improvement, building on operating experience from similar industries, such as the domestic and international commercial nuclear and chemical industries.”  (Final Report**, p. 2)

It appears the team performed most of its research during 2008, conducted a pilot program in 2009 and published its final report in 2010.  Research included reviewing the space shuttle and Texas City disasters, the Davis-Besse incident, works by gurus such as James Reason, and guidance and practices published by NASA, NRC, IAEA, INPO and OSHA.

Major Results

The team developed a definition of safety culture and described a process whereby using organizations could assess their safety culture and, if necessary, take steps to improve it.

The team’s definition of safety culture:

“An organization’s values and behaviors modeled by its leaders and internalized by its members, which serve to make safe performance of work the overriding priority to protect the workers, public, and the environment.” (Final Report, p. 5)

After presenting this definition, the report goes on to say “The Team believes that voluntary, proactive pursuit of excellence is preferable to regulatory approaches to address safety culture because it is difficult to regulate values and behaviors. DOE is not currently considering regulation or requirements relative to safety culture.” (Final Report, pp. 5-6)

The team identified three focus areas that were judged to have the most impact on improving safety and production performance within the DOE complex: Leadership, Employee/Worker Engagement, and Organizational Learning. For each of these three focus areas, the team identified related attributes.

The overall process for a using organization is to review the focus areas and attributes, assess the current safety culture, select and use appropriate improvement tools, and reinforce results. 

The list of tools to assess safety culture includes direct observations, causal factors analysis (CFA), surveys, interviews, review of key processes, performance indicators, Voluntary Protection Program (VPP) assessments, stream analysis and Human Performance Improvement (HPI) assessments.***  The Final Report also mentioned performance metrics and workshops. (Final Report, p. 9)

Tools to improve safety culture include senior management commitment, clear expectations, ISMS training, managers spending time in the field, coaching and mentoring, Behavior Based Safety (BBS), VPP, Six Sigma, the problem identification process, and HPI.****  The Final Report also mentioned High Reliability Organization (HRO), Safety Conscious Work Environment (SCWE) and Differing Professional Opinion (DPO). (Final Report, p. 9)  Whew.

The results of a one-year pilot program at multiple contractors were evaluated and the lessons learned were incorporated in the final report.

Our Assessment

Given the diversity of the DOE complex, it’s obvious that no “one size fits all” approach is likely to be effective.  But it’s not clear that what the team has provided will be all that effective either.  The team’s product is really a collection of concepts and tools culled from the work of outsiders, combined with DOE’s existing management programs, and repackaged as a combination of overall process and laundry lists.  Users are left to determine for themselves exactly which sub-set of tools might be useful in their individual situations.

It’s not that the report is bad.  For example, the general discussion of safety culture improvement emphasizes the importance of creating a learning organization focused on continuous improvement.  In addition, a major point they got right was recognizing that safety can contribute to better mission performance.  “The strong correlation between good safety performance with good mission performance (or productivity or reliability) has been observed in many different contexts, including industrial, chemical, and nuclear operations.” (Final Report, p. 20)

On the other hand, the team has adopted the works of others but does not appear to recognize how, in a systems sense, safety culture is interwoven into the fabric of an organization.  For example, feedback loops from the multitude of possible interventions to overall safety culture are not even mentioned.  And this is not a trivial issue.  An intervention can provide an initial boost to safety culture but then safety culture may start to decay because of saturation effects, especially if the organization is hit with one intervention after another.

In addition, some of the major, omnipresent threats to safety culture do not get the emphasis they deserve.  Goal conflict, normalization of deviance and institutional complacency are included in a list of issues from the Columbia, Davis-Besse and Texas City events (Final Report, p. 13-15) but the authors do not give them the overarching importance they merit.  Goal conflict, often expressed as safety vs mission, should obviously be avoided but its insidiousness is not adequately recognized; the other two factors are treated in a similar manner. 

Two final picky points:  First, the report says it’s difficult to regulate behavior.  That’s true but companies and government do it all the time.  DOE could definitely promulgate a behavior-based safety culture regulatory requirement if it chose to do so.  Second, the final report (p. 9) mentions leading (vs lagging) indicators as part of assessment but the guidelines do not provide any examples.  If someone has some useful leading indicators, we’d definitely like to know about them. 

Bottom line, the DOE effort draws from many sources and probably represents consensus building among stakeholders on an epic scale.  However, the team provides no new insights into safety culture and, in fact, may not be taking advantage of the state of the art in our understanding of how safety culture interacts with other organizational attributes. 


*  Energy Facility Contractors Group (EFCOG)/DOE Integrated Safety Management System (ISMS) Safety Culture Task Team.

**  J. McDonald, P. Worthington, N. Barker, G. Podonsky, “EFCOG/DOE ISMS Safety Culture Task Team Final Report”  (Jun 4, 2010).

***  EFCOG/DOE ISMS Safety Culture Task Team, “Assessing Safety Culture in DOE Facilities,” EFCOG meeting handout (Jan 23, 2009).

****  EFCOG/DOE ISMS Safety Culture Task Team, “Activities to Improve Safety Culture in DOE Facilities,” EFCOG meeting handout (Jan 23, 2009).

3 comments:

  1. DOE is off the beam in many aspects here [not unusual] an important defect being the definition of [org?] safety culture that is applied - a variant of the [ineffective, non-operational] definition INPO developed in the mid 1990's. I have a collected about 20 similar ineffective, non-operational definitions; attempts by various groups that did not begin with a clear understanding of what they were attempting to define. If you can't define something properly, you can't measure, manage, or regulate it properly. This is the main problem. Proper definitions exist, but they are not applied, because most orgs [DOE et al] do not understand them.

    ReplyDelete
  2. To follow up my previous comment, I sent detailed info on the above concern [and the proper solutions] to:

    Don Nichols, Chief of Defense Nuclear Safety
    National Nuclear Security Administration
    Department of Energy

    John Ordaz, Acting Director,
    Office of Environment, Safety and Health,
    National Nuclear Security Administration
    Department of Energy

    Richard Lagdon, Chief of Nuclear Safety
    Department of Energy

    William Eckroade, Deputy Chief for Operations
    Office of Health, Safety and Security
    Department of Energy

    DOE is [of course] free to ignore this input from a "member of the public", in fact historically, it is almost a certainty that they will. Like TEPCO and Japan NSA prior to Fukushima, they don't see any problem, or any need to discuss this sort of feedback in any length or in any detail.

    ReplyDelete
  3. If someone has some useful leading indicators, we’d definitely like to know about them.

    I believe I have some useful experience with Leading Indicators over the past 19 years with the DOE complex, at Hanford first, and now Savannah River.

    A few salient points:

    Leading indicators are better thought of as "how to build a better future", not as "predictors of the future". All sorts of excess baggage and circular logic gets in the way if you believe they are predictors.

    Second, it is much more important WHAT YOU DO with leading indicator data, rather than WHAT THE INDICATOR IS DEFINED AS. Even rudimentary data can server as great cultural and "leading" indicators, if properly analyzed. Dr. Deming used Sick Day data as his barometer of a company he first visited. I've used a "chart of how many charts" are requested as an organizational indicator (less chart requests = an organization growing afraid of its data, afraid of bad news). Numerical goals and Targets are exceptionally harmful to leading indicators. I've used the same answer Dr. Deming came up with - SPC.

    See also http://www.efcog.org/wg/esh_es/Statistical_Process_Control/index.htm and INPO 07-007.

    ReplyDelete

Thanks for your comment. We read them all. We'd like to display them under their respective posts on our main page but that's not how Blogger works.