DNFSB Hearings on Safety Culture, Round Three

DNFSB Headquarters

On October 7, 2014 the Defense Nuclear Facilities Safety Board (DNFSB) held its third and final hearing* on safety culture (SC) at Department of Energy (DOE) nuclear facilities.  The original focus was on the Hanford Waste Treatment Plant (WTP) but this hearing also discussed the Waste Isolation Pilot Plant (WIPP), the Pantex plant and other facilities.  There were three presenters: DOE Secretary Moniz and two of his top lieutenants.  A newspaper article** published the same day reported key points made during the hearing and you should read that article along with this post.  This post focuses on items not included in the newspaper article, including the tone of the hearing and other nuances.  The presenters used no slides and the hearing transcript has not yet been released.  The only current record of the hearing is a DNFSB video.

Secretary Moniz

Moniz has been Secretary for about a year-and-a-half.  In his view, the keys to improving SC are training, consistent senior management attention, and procurement modifications, i.e., DOE’s intent to revise RFP and contracting processes to include SC expectations.  He also said fostering the consideration of SC in all decisions, including resource allocation, is important.  Board member Sullivan asked about the SC issues at Pantex and Moniz provided a generic answer about improving self-assessments and sharing lessons learned but ultimately punted to the next presenter, Ms. Creedon.

Principal Deputy Administrator Creedon, National Nuclear Security Administration (NNSA)

Creedon has been in her position for two months.  She believes NNSA employees get the job done in spite of bureaucracy but they need greater trust in senior management who, in turn, must work harder to engage the workforce.  Returning to the Pantex*** issues, Sullivan asked why the recommendations of the plant’s outside technical advisors had been ignored for years.  Creedon said she would work to improve communications up and down the organization.  In a separate exchange, she provided an example of positive reinforcement where NNSA employees can receive cash awards ($500) for good work. 

Creedon’s  prior position was in the Department of Defense.  To the extent she has the warfighter mentality (“Anything, anywhere, anytime…at any cost”)**** then balancing mission and safety may not be natural for her.  Her response to a question on this topic was not encouraging; she claimed the motto du jour for NNSA (“Mission First, People Always”) adequately addresses safety's prioity but it obviously doesn’t even mention safety.

Acting Assistant Secretary for Environmental Management Whitney

Whitney is also new in his job but not to DOE, coming from DOE Oak Ridge.  He laid out his goals of establishing trust, a questioning attitude and mutual respect.  He was asked about a SC assessment finding that DOE senior managers don’t feel responsible for safety, rather it belongs to the site leads or one of the EM mission support units.  Whitney said that was unacceptable and described the intent to add SC factors to senior management evaluations.  He also repeated the plan to upgrade the WTP contractor evaluation to include SC factors.  He noted that most employees stay at one site for their entire career, making it hard to transfer SC from site to site.

Our Perspective

The overall tone of the hearing was collegial.  The Board expressed support and encouragement for the presenters, all of whom are relatively new in their jobs.  The presenters all stayed on message and reinforced each other.  For example, for WTP one message is “We know there are still significant SC issues at WTP but we have the right team in place and are taking action and making progress.  Changing a decades-old culture takes time.”  Whitney received more of a (polite) grilling probably because the WTP and the WIPP are under his purview.

We are totally supportive of DOE’s stated intent to add SC factors to contracts and senior management evaluations.  When players have skin in the game, the chances of seeing desired behavioral changes are greatly increased.  We are equally supportive of Secretary Moniz’ desire to create a culture that incorporates safety considerations in all decisions.

DOE is trying to make its employees more conscious of safety’s importance; two thousand mangers have gone through SC training and there’s more to come.  Now we’re starting to worry about the drumbeat of SC creating a Weltanschauung where a strong SC is sine quo non for good outcomes and a weak SC is always present when bad outcomes occur.  Organizational reality is more complicated.  An organization with a mediocre SC can achieve satisfactory results if other effective controls and incentives are in place; an organization with a strong SC can still make poor decisions.  And luck can run good or bad for anyone.

*  DNFSB Oct. 7, 2014 Safety Culture Public Meeting and Hearing.  We posted on the first hearing on June 9, 2014 and the second hearing on Sept. 4, 2014.

**  A. Cary, “Moniz says safety culture at Hanford vit plant led to problems,” Tri-City Herald (Oct. 7, 2014).

***  NNSA's responsibilities include Pantex which has recognized SC issues.

****  See the third footnote in our Sept. 4, 2014 post.

DNFSB Hearings on Safety Culture, Round Two

DNFSB Headquarters

On August 27, 2014 the Defense Nuclear Facilities Safety Board (DNFSB) convened the second of three hearings “to address safety culture at Department of Energy defense nuclear facilities and the Board’s Recommendation 2011–1, Safety Culture at the Waste Treatment and Immobilization Plant.”*  The first hearing was held on May 28, 2014 and heard from industry and federal government safety culture (SC) experts; we reviewed that hearing on June 9, 2014.  The second hearing received SC expert testimony from the U.S. Navy, the U.S. Chemical Safety and Hazard Investigation Board and academia.  The following discussion reviews the presentations in the order they were made to the board. 


Adm. Norton's (Naval Safety Center) presentation** on the Navy’s SC programs was certainly comprehensive with 32 slides for a half-hour talk (plus 22 backup slides).  It appears the major safety focus has been on aviation but the Center’s programs also address the afloat communities (surface, submarine and diving) and Marines.  The programs make heavy use of surveys and unit visits in addition to developing and presenting training and workshops.  Not surprisingly, the Navy stresses the importance of leadership, especially personal involvement and commitment, in creating a strong SC.  They recognize that implementing a strong SC faces a direct challenge from other organizational values such as the warfighter mentality*** and softer challenges in areas such as IT (where there are issues with multiple systems and data problems).

Program strengths include the focus on leadership (leadership drives climate, climate drives cultural change) and the importance of determining why mishaps occurred.  The positive influence of a strong SC on decision making is implied.

Program weaknesses can be inferred from what was not mentioned.  For example, there was no discussion of the importance of fixing problems or identifying hard-to-see technical problems.  More significantly, there was no mention of High Reliability Organization (HRO) attributes, a real head-scratcher given that some of the seminal work on HROs was conducted on aircraft carriers. 

Adm. Eccles' (Navy ret.) presentation**** basically reviews the Navy’s SUBSAFE program and its focus on compliance with program requirements from design through operations.  Eccles notes that ignorance, arrogance and complacency are challenges to maintaining an effective program.

Mr. Griffon's (Chemical Safety Board Member) presentation***** illustrates the CSB’s straightforward approach to investigating incidents, as reflected in the following quotes:

“Intent of CSB investigations are to get to root cause(s) and make recommendations toward prevention.” (p. 3)

While searching for root causes the CSB asks: “Why conditions or decisions leading to accident were seen as normal, rational, or acceptable prior to the accident.” (p. 4)

CSB review of incident-related artifacts includes two of our hot button issues, Process Safety Management action item closure (akin to a CAP) and the repair backlog. (p. 5)  Griffon reviews major incidents, e.g., Texas City and Deepwater Horizon.  For Deepwater, he notes how certain decisions were (deliberately) incompletely informed, i.e., did not utilize readily available relevant information, and thus are indicative of an inadequate SC. (p. 16)  Toward the end Griffon observes that “Safety culture study/change must consider inequalities of power and authority.” (p. 19)  That seems obvious but it doesn’t often get said so clearly.

We like the CSB’s approach.  There is no new information here but it’s a quick read of what basic SC should and shouldn’t be.

Prof. Meshkati's (Univ. of S. Cal.) presentation^ compares the cultures at TEPCO’s Fukushima Daiichi plant and Tohoku Electric’s Onagawa plant.  It is mainly a rehash of the op-ed Meshkati co-authored back in March 2014 (and we reviewed on March 19, 2014.)  The presentation adds something we pointed out as an omission in that op-ed, viz., that TEPCO’s Fukushima Daini plant eventually managed to shut down safely after the earthquake and tsunami.  Meshkati notes approvingly that Daini personnel exhibited impromptu, but prudent, decision-making and improvisation, e.g., by flexibly applying emergency operation procedures. (p. 37)

Prof. Sutcliffe (John Hopkins Univ.) co-authored an important book on High Reliability Organizations (which we reviewed on May 3, 2013) and this academically-oriented presentation^^ draws on her earlier work.  It begins with a familiar description of culture and how its evolution can be influenced.  Importantly it shows rewards (including money) as a key input affecting the link between leaders’ philosophy and employees’ behavior. (p. 6) 

Sutcliffe discusses how failure to redirect action (in a situation where a change is needed) can result from failure of foresight or sensemaking, or being overcome by dysfunctional momentum.  She includes a lengthy example featuring wildland firefighters that illustrates the linkages between cues, voiced concerns, search for disparate perspectives, situational reevaluation and redirected actions.  It’s worth a few minutes of your time to flip through these slides.

Our Perspective

For starters, the Naval Safety Center's activities may be too bureaucratic, with too many initiatives and programs, and focused mainly on compliance with procedures, rules, designs, etc.  It’s not clear what SC lessons can be learned from the Navy experience beyond the vital role of leadership in creating a cultural vision and attempting to influence behavior toward that vision.

The other presenters added nothing that was not already available to you, either through Safetymatters or from observing SC tidbits in the information soup that flows by everyone these days.

Subsequent to the first hearing we reported that Safety Conscious Work Environment (SCWE) issues exist at multiple DOE sites (see our July 8, 2014 post).  This should increase the sense of urgency associated with strengthening SC throughout DOE.  However, our bottom line remains the same as after the first hearing: “The DNFSB is still trying to figure out the correct balance between prescription and flexibility in its effort to bring DOE to heel on the SC issue.  SC is a vital part of the puzzle of how to increase DOE line management effectiveness in ensuring adequate safety performance at DOE facilities.” 


*  DNFSB Aug. 27, 2014 Public Hearing on Safety Culture and Board Recommendation 2011-1.  There is a video of the hearing available.

**  K.J. Norton (U.S. Navy), “The Naval Safety Center and Naval Safety Culture,“ presentation to DNFSB (Aug. 27, 2014).

***  “Anything, anywhere, anytime…at any cost”—desirable warfighter mentality perceived to conflict with safety.” (p. 11)

****  T. J. Eccles (U.S. Navy ret.), “A Culture of Safety: Submarine Safety in the U. S. Navy,” presentation to DNFSB (Aug. 27, 2014).

*****  M.A. Griffon (Chem. Safety Bd.), “CSB Investigations and Safety Culture,” presentation to DNFSB (Aug. 27, 2014).

^  Najm Meshkati, “Leadership and Safety Culture: Personal Reflections on Lessons Learned,” presentation to DNFSB (Aug. 27, 2014).  Prof. Meshkati was also the technical advisor to the National Research Council’s safety culture lessons learned from Fukushima report which we reviewed on July 30, 2014.

^^  K.M. Sutcliffe, “Leadership and Safety Culture,” presentation to DNFSB (Aug. 27, 2014).

Vit Plant Safety Culture Update

Hanford Waste Treatment Plant

DOE released a June 2014 follow-up assessment* on safety culture (SC) at the Hanford Waste Treatment Plant (WTP or the Vit Plant).  This post provides our perspective on the assessment.  We will not review every facet of the report but will focus on aspects that we think are important to understanding the current state of SC at the WTP project.

Overview

Back in 2011, the Defense Nuclear Facilities Safety Board (DNFSB) called the WTP safety culture (SC) “flawed.”  Following the DNFSB report, DOE conducted an assessment of the WTP SC and concluded “that a significant number of staff within ORP [DOE’s Office of River Protection] and BNI [Bechtel National Inc.] expressed reluctance to raise safety or quality concerns for various reasons.” (p. 1) 

Like DOE’s 2011 report, the current one is based on multiple data sources: structured interviews, focus groups, observations and a culture survey.  The report identifies many SC-related interventions that have been initiated, and lists positive and negative findings from the data collected.  Detailed assessment results are reported separately for ORP and BNI.**
 
WTP’s Safety Conscious Work Environment (SCWE)—Voicing Concerns, Challenging Decisions, Fear of Retaliation

The 2011 DNFSB critique focused on the treatment of project personnel who raised technical issues.  Some of these personnel complained about retaliation for bringing up such issues.  These issues can be raised in the Employee Concerns Program (ECP), the Differing Professional Opinions (DPO) process and challenging management decisions.  In what is arguably the report’s most significant finding, perceptions of conditions in these areas are worse than they were in 2011 for ORP and unchanged for BNI. (pp. 4-5)

Supporting Details

Although ORP senior management pointed to recent reductions in ECP concerns, “[s]ome interviewees indicated that they perceived a chilled environment at ORP and they did not believe that ECP concerns and DPO issues were always addressed or resolved in a timely manner.  Additionally, some interviewees described being told by supervision not to write a DPO because it would be a career limiting decision.” (p. B-21)  Interviewees from both ORP and BNI said they lacked trust in the ECP. (pp. B-22/23)

Most ORP employees believe that constructive criticism is not encouraged. (p. 4)  Within ORP, only 30% of all ORP survey respondents (and 65% of managers) feel that they can openly challenge decisions made by management. (p. B-21)  In BNI/URS, the numbers are 45% of all respondents and 75% of managers. (p. B-22)

“The statement that management does not tolerate retaliation of any kind for raising concerns was agreed to by approximately 80% of the ORP, . . . 72% of the BNI and 80% of the URS survey respondents.” (pp. B-21-22)  In addition, “Anonymous PIERs [Project Issues Evaluation Reports] are used a lot because of fear of retaliation.” (p. B-23)  

All in all, hardly a ringing endorsement of the WTP SCWE.

Decision Making, Corrective Action and Compensation

Safetymatters readers know of our long-standing interest in how SC is reflected in these key artifacts.

Decision Making

“Interviewees provided some examples of where decision making was not perceived to reflect the highest commitment to safety”  Examples included downgrading or elimination of assessment findings, the margin of safety in corrective action plans and the acceptable level of risk for the project.  (p. B-9)  Looks like there's some room for improvement in this area.

Corrective Action

Within BNI, there are positive comments about the corrective action process but the assessment team “observed a lack of accountability for a backlog of corrective actions at a PIRB [Performance Improvement Review Board] meeting.  There is a perceived lack of accountability for corrective actions in timeliness, ownership, and quality, ...” (p. B-16)

“[O]nly about 18% of all ORP interviewee respondents believed that employees are encouraged to notify management of problems they observe and that there is a system that evaluates the problem and makes a determination regarding future action.” (p. 13)

Plenty of room for improvement here.

Compensation

In the area of compensation, it appears some rewards for Bechtel are based on SC behaviors. (p. B-8)  We consider that a very positive development.

Concerns Over ORP Working More Collaboratively with Bechtel

On the surface this looks like a positive change: two entities working together to achieve a common goal.  However, this has led to at least two concerns.  First, as described in the report, some ORP personnel believe ORP is abdicating or compromising its responsibility to perform oversight of Bechtel, in other words, ORP is more of a teammate and less of an umpire. (pp. B-4, -9, -14)  Second, and this reflects our perspective, changing the relationship between the entities can result in revised system dynamics, with old performance-oversight feedback loops replaced by new ones.  The rules of engagement have changed and while safety may still be the number one priority, the cultural milieu in which safety is achieved has also changed.

Another complication is caused by the role of Bechtel Corporate.  The report says corporate’s values and goals may not be well-aligned with BNI’s need to prioritize SC attributes and behavior.  This can lead to a lack of transparency in BNI decisions. (pp. 5, 7)  That may be a bit of weasel wording in the report; in more direct terms, corporate’s number one priority is for the money train to keep running.

Bottom Line on SC at WTP 

A strong SC is, in some ways, about respect for the individual.  The concerns that WTP personnel  express about using the ECP or DPO process, or challenging management decisions suggest that the WTP project has a ways to go to inculcate an adequate level of such respect.  More importantly, it doesn’t appear they have made any significant process toward that goal in the last few years.

Morale is an aspect of the overall culture and at the WTP, morale is arguably low because of lack of progress and missed schedules. (pp. 6, B-10)  There is undoubtedly plenty of work to do but on a day to day basis, we wouldn’t be surprised if some people feel their work is not meaningful.


*  DOE Office of Environment, Safety and Health Assessments (now the Office of Independent Enterprise Assessments), “Independent Oversight Follow-up Assessment of Safety Culture at the Waste Treatment and Immobilization Plant” (June, 2014).

**  ORP has line management responsibility for the WTP, BNI is the primary contractor and URS Corporation (URS) is a major subcontractor.

Catching Up on DOE’s SCWE Extent of Condition Review

Hanford Waste Treatment Plant

On May 29, 2014 DOE submitted its partial response to the Defense Nuclear Facilities Safety Board (DNFSB) Recommendation 2011-1 in a report* on DOE’s Safety Conscious Work Environment (SCWE) extent of condition review and recommended actions for ongoing safety culture (SC) improvement at DOE facilities.

(Quick history: The June 9, 2011 DNFSB report on DOE’s Hanford Waste Treatment Plant (WTP or the Vit Plant) said the WTP SC was “flawed.”  The report’s recommendations included that DOE should conduct an extent of condition review to determine whether WTP SC weaknesses existed at other DOE facilities.  DOE agreed to perform the review but focused on SCWE because, in DOE’s view, the issues at WTP were primarily SCWE related.)

This post summarizes the report’s findings then parses the details and provides our perspective.

 Report Findings and Recommendations

The report was based on data from eleven independent SC assessments and thirty-one SCWE self-assessments conducted by individual organizations.  The DOE review team processed the data through their analyzer and homogenizer to identify four primary SC attributes** to focus on for continuous improvement in DOE:

• Demonstrated safety leadership
• Open communication and fostering an environment free from retribution
• Teamwork and mutual respect
• Credibility, trust and reporting errors and problems

Further processing through the bureaucratizer yielded three recommended actions to improve the SC attributes:

• Form a DOE SC Improvement Panel to ensure leadership and focus on DOE's SC initiatives
• Incorporate SC and SCWE concepts and practices into DOE training
• Evaluate contract language to incorporate clear references to SC  (pp. 3-4)

Our Perspective

We reviewed the DOE independent assessments on January 25, 2013 and the self-assessments on March 31, 2014.  From the former we concluded that issues similar to those found at the WTP existed at other DOE facilities, but to a lesser degree than WTP.  The self-assessments were of such varying quality and credibility that we basically couldn’t infer anything.***

The Analyzer

The DOE team reviewed all the assessments to identify specific issues (problems).  The team binned issues under the SC attributes in DOE's Integrated Safety Management System Guide and then counted the number of issues under each attribute; a higher count meant a more serious problem.  They performed a similar exercise to identify positive organizational trends (strengths) mentioned in the assessments. 

We could be picky and ask if all the issues (or strengths) were of essentially the same importance or magnitude but the team had a lot of data to review so we’ll let that slide.  Concurrently identifying strengths was a good idea; it harkens back to Peter Drucker who advised managers to build on strengths.****

The Homogenizer

Here’s where we begin to have problems.  The team focused on identifying SC attributes and developing recommendations that applied to or affected the entire Department, essentially boiling their results down to a one-size-fits-all approach.  However, their own data belies that approach.  For example, the Leadership attribute “Open communication and fostering an environment free from retribution” was identified as both an issue AND a strength. 

In plain English, some organizations don’t exhibit the desired communication attribute and others do.  One proper fix is to identify who is doing it right, define what exactly they’re doing, and develop a method for transferring that approach to the problem organizations.  The report even says this attribute “is an area in which management can learn lessons from those sites where it was deemed a strength so that best practices can become commonplace” but this statement is buried at the end of the report. (p. 22)

The DOE entities exhibit a wide variety of scale, scope, mission and organizational and technical complexity.  The Department’s goal should be to recognize that reality, develop it as an overall strength and then build on it to create site- or organization-specific interventions.

The Bureaucratizer

The proposed fixes would basically create a top-level coordinating and oversight group, enhance SC and SCWE training and modify contracts.  The recommendations reflect a concerted search for the lowest common denominator and a minimum amount of real change.

If the SC Improvement Panel is established, it should focus on setting or refining SC policy and ensuring those policies are implemented by line management, especially field management.  They should also be involved in evaluating major SC issues.  If things aren’t going well, this group should be the first to ask the hard “Why?” questions.  But most of the panel’s proposed tasks, viz., maintaining SC visibility, providing a forum for evaluating SC status and overseeing training improvements, are low-value make work.

“[S]afety culture training for all personnel, up and down the management chain, will be updated and/ or developed to ensure that roles and responsibilities are understood and personnel have the capabilities needed to play their part in continuously improving DOE's safety culture; . . .” (p. 24)  This is a standard fix for almost any perceived organizational problem.  It doesn’t require managers to do anything different.

Modifying contracts to incorporate clear references to SC is only a beginning.  What are the carrots and the sticks to incent the contractors to actually develop, measure and maintain an effective SCWE and strong SC?

Bottom Line

This report comports completely with an organization that resembles a fifty foot sponge.  You can kick it as hard as you like, your foot goes in deep and you think you’ve had an effect, but when you withdraw your foot, the organization fills in the hole like your kick never happened.  I thought I heard a loud pop on May 29.  I now realize it was likely DNFSB Chairman Winokur’s head exploding when he read this report.

But Wait, There’s More

Remember the question the DNFSB initially asked in 2011: Do WTP SC issues exist elsewhere in DOE?  Well, the answer is: “Review of assessment results from both [independent and self assessments] indicated there is a SCWE extent of condition that requires additional and ongoing actions to improve performance.” (p. 28, the penultimate page of the report)


*  J. Hutton (DOE) to P.S. Winokur (DNFSB) May 29, 2014 letter transmitting DOE Consolidated Report for DNFSB Recommendation 2011-1, Actions 2-8 and 2-9 (May 2014).

** DOE considers SC in three focus areas: Leadership, Employee Engagement and Organizational Learning.  Each focus area has a set of associated attributes that describe what a positive SC should look like. (pp. 4-5)

***  The report puts the self-assessments in the best possible light by describing them as learning experiences for the organizations involved. (p.9)

****  Drucker was referring to individuals but, in this case, we’ll stretch the blanket to cover organizations.  For individuals, weaknesses should not be ignored but the energy required to fix them, if it’s even possible, is often too great.  However, one should keep an eye on such weaknesses and not allow them to lead to performance failure.

DNFSB Observations on Safety Culture

DNFSB Headquarters

The Defense Nuclear Facilities Safety Board (DNFSB) has been busy in the safety culture (SC) space.  First, their Chairman’s May 7, 2014 presentation on preventing major accidents provides a window into how the DNFSB views safety management and SC in the DOE complex.  Second, the DNFSB’s meeting on May 28, 2014 heard presentations on SC concepts from industry and government experts.  This post reviews and provides our perspective on both events. 

Chairman’s Presentation

This presentation was made at a DOE workshop.*  Chairman Winokur opened with some examples of production losses that followed incidents at DOE facilities and concluded the cost of safety is small compared to the cost of an accident.  He went on to discuss organizational factors that can set the stage for accidents or promote improved safety performance.  Some of these factors are tied to SC and will be familiar to Safetymatters readers.  They include the following:

Leadership

The presentation quotes Schein: “The only thing of real importance that leaders do is to create and manage culture.” (p. 13)  This quote is used by many in the nuclear industry to support a direct and complete connection between leadership and an organization’s culture.   While effective leadership is certainly necessary, we have long argued for a more nuanced view, viz., that leaders influence but do not unilaterally define culture.  In fact, on the same page in Organizational Culture, Schein says “Culture is the result of a complex group learning process that is only partially influenced by leader behavior.” **

Budget and production pressures and
Rewards that favor mission over safety 

As Winokur pointed out, it is unfortunately true that poor safety performance (accidents and incidents) can attract resources while good safety performance can lead to resources being redirected.  Good safety performance becomes taken for granted and is largely invisible.  “Always focus on balancing mission and safety.  There will always be trade-offs, but safety should not get penalized for success.” (p. 19) 

On our part, we feel like we’ve been talking about goal conflicts forever.  The first step in addressing goal conflicts is to admit they exist, always have and probably always will.  The key to resolving them is not by issuing a safety policy, it is to assure that an entity’s decision making process and its reward and compensation system treat safety with the priority it warrants. 

Decision making

Winokur says “Understand the nature of low-probability, high-consequence accidents driven by inadequate control of uncertainty, not cause-effect relationships . . .” (p. 14) and “Risk-informed decision making can be deceptive; focus on consequences, as well as probabilities.” (p. 16)  These observations are directly compatible with Nicholas Taleb: “This idea that in order to make a decision you need to focus on the consequences (which you can know) rather than the probability (which you can’t know) is the central idea of uncertainty.”***  See our June 18, 2013 post for a discussion of decisions that led to high-consequence (i.e., really bad) outcomes at Crystal River, Kewaunee and San Onofre.

There is no additional material in the presentation for a few important factors, so we will repeat earlier Safetymatters commentary on these topics.    

Complacency and
Accumulated residual risks that erode the safety margin

We have pointed out how organizations, especially high reliability organizations, strive to maintain mindfulness and combat complacency.  Complacency leads to hubris (“It can’t happen here”) and opens the door for the drift toward failure that occurs with normalization of deviance, constant environmental adaptations, “normal” system performance excursions, group think and an irreducible tendency for SC to decay over time.

Lack of oversight

This refers to everyone who has the responsibility to provide competent, timely, incisive assessment of an entity’s activities but fails to do so.  Their inaction or incompetence neither reinforces a strong SC nor prods a weak SC to improve. 

DNFSB Hearing with SC Expert Presentations

This was "the first of two hearings the Board will convene to address safety culture at Department of Energy defense nuclear facilities and the Board’s Recommendation 2011–1, Safety Culture at the Waste Treatment and Immobilization Plant."****  This hearing focused on presentations by SC experts: Sonya Haber (an SC consultant to DOE), NRC and NASA.  The experts’ slide presentations and a video of the hearing are available here.

Haber hit the right buttons in her presentation but neither she nor anyone else mentioned her DOE client's failure to date to integrate the SC assessments and self-assessments DOE initiated at various facilities in response to Recommendation 2011-1.  We still don’t know whether WTP SC problems exist elsewhere in the DOE complex.  We commented on the DOE’s response to 2011-1 on January 25, 2013 and March 31, 2014.

Winokur asked Haber about the NRC's "safety first" view vs. the DOE's "mission/safety balance."  The question suggests he may be thinking the "balance" perspective gives the DOE entities too much wiggle room to short change safety in the name of mission.

The NRC presenter was Stephanie Morrow.  Her slides recited the familiar story of the evolution of the SC Policy Statement and its integration into the Reactor Oversight Process.  She showed a new figure that summarized NRC’s SC interests in different columns of the ROP action matrix.  Chairman Winokur asked multiple questions about how much direction the NRC gives the licensees in how to perform SC assessments.  The answer was clear: In the NRC’s world, SC is the licensee's responsibility; the NRC looks for adequacy in the consideration of SC factors in problem resolution and SC assessments.  Morrow basically said if DNFSB is too prescriptive, it risks ending up "owning" the facility SC instead of the DOE and facility contractor.

Our Perspective

The Chairman’s presentation addressed SC in a general sense.  However, the reality of the DOE complex is a formidable array of entities that vary widely in scope, scale and missions.  A strong SC is important across the complex but one-size-fits-all approaches probably won’t work.  On the other hand, the custom fit approach, where each entity has flexibility to build its SC on a common DOE policy foundation doesn’t appear to lead to uniformly good results either.  The formal hearing to receive presentations from SC industry experts evidences that the DNFSB is gathering information on what works in other fields.  

Bottom line: The DNFSB is still trying to figure out the correct balance between prescription and flexibility in its effort to bring DOE to heel on the SC issue.  SC is an vital part of the puzzle of how to increase DOE line management effectiveness in ensuring adequate safety performance at DOE facilities.


*  P.S. Winokur, “A User’s Guide to Preventing Major Accidents,” presentation at the 2014 Nuclear Facility Safety Programs Annual Workshop (May 7, 2014).  The workshop was sponsored by the DOE Office of Environment, Health, Safety, and Security.  Thanks to Bill Mullins for bring this presentation to our attention.

**  E. Schein, Organizational Culture and Leadership (San Francisco, CA: Jossey-Bass, 2004), p. 11.

***  N. Taleb, The Black Swan (New York: Random House, 2007), p. 211.

****  DNFSB May 28, 2014 Public Hearing on Safety Culture and Board Recommendation 2011-1.

WIPP - Release the Hounds

(Ed. note: This is Safetymatters’ second post on the Phase 1 WIPP report.  Bob and I independently saw the report, concluded it raised important questions about DOE and its investigative process and headed for our keyboards.  We will try to get an official response to our posts—but don’t hold your breath.) 

Earlier this week the DOE released its Accident Investigation Report on the Radiological Release Event at the Waste Isolation Pilot Plant.  The report is a prodigious effort in the just over two months since the event.  It is also a serious indictment of DOE’s management of WIPP and arguably, the DOE itself.  There is however a significant flaw in the investigation and report: the investigators were kept on too tight a leash.  Itemization of failures, particularly pervasive failures, without pursuing how and why they occurred is not sufficient.  It also highlights the essence and value of systems analysis - identifying the fundamental dynamics that produced the failures and solutions that change those dynamics.

At first blush the issuance of yet another report on safety issues and safety management performance at a DOE facility would hardly merit a rush to the keyboard to dissect the findings.  Yet we believe this report is a tipping point in the pervasive and continuing issues at DOE facilities and should be a call for much more aggressive action.  It doesn’t take long for the report to get to the point in the Executive Summary:

“The Board identified the root cause of Phase 1 of the investigation of the release of radioactive material from underground to the environment to be NWP’s and CBFO’s management failure to fully understand, characterize, and control the radiological hazard.” [emphasis added] (p. ES-6)  NWP is Nuclear Waste Partnership, the contractor with direct management responsibility for WIPP operations, and CBFO is the Carlsbad Field Office of the DOE.

To complete the picture the investigation board also found as a contributing cause, that DOE Headquarters oversight was ineffective.  So in sum, the board found a total failure of the management system responsible for radiological safety at the WIPP. 

Interestingly there has been a rather muted response to this report.  The DOE issued the report with a strikingly neutral press release quoting Matt Moury, Environmental Management Deputy Assistant Secretary, Safety, Security, and Quality Programs: “The Department believes this detailed report will lead WIPP recovery efforts as we work toward resuming disposal operations at the facility.”  And Joe Franco, DOE’s Carlsbad Field Office Manager: “We understand the importance of these findings, and the community’s sense of urgency for WIPP to become operational in the future.”*  (We note that both statements focus on resumption of operations versus correction of deficiencies.)  New Mexico’s U.S. Senators Udall and Heinrich called the findings “deeply troubling” but then simply noted that they expected DOE management to take the necessary corrective actions.**  If there is any sense of urgency we would think it might be directed at understanding how and why there was such a total management failure at the WIPP.

To fully appreciate the range and depth of failures associated with this event one really needs to read the board’s report.  Provided below is a brief summary of some of the highlights that illustrate the identified issues:

-    Implementation of the NWP Conduct of Operations Program is not fully compliant with DOE policy;
-    NWP does not have an effective Radiation Protection Program in accordance with 10 Code of Federal Regulations (CFR) 835, Occupational Radiation Protection;
-    NWP does not have an effective maintenance program;
-    NWP does not have an effective Nuclear Safety Program in accordance with 10 CFR 830 Subpart B, Safety Basis Requirements;
-    NWP implementation of DOE O 151.1C, Comprehensive Emergency Management System, was ineffective;
-    The current site safety culture does not fully embrace and implement the principles of DOE Guide (G) 450.4-1C, Integrated Safety Management Guide [note: findings consistent with findings of the 2012 SCWE self assessment results]; and DOE oversight of NWP was ineffective;
-    Execution of CBFO oversight in accordance with DOE O 226.1B was ineffective; and
-    As previously mentioned, DOE Headquarters (HQ) line management oversight was ineffective. (pp. ES 7-8)

Many of the specific deficiencies cited in the report are not point in time occurrences but stem from chronic and ongoing weaknesses in programs, personnel, facilities and resources. 

Losing the Scent

As mentioned in the opening paragraph we feel that while the report is of significant value it contains a shortcoming that will likely limit its effectiveness in correcting the identified issues.  In so many words the report fails to ask “Why?”  The report is a massive catalogue of failures yet never fully pursues the ultimate and most relevant question: Why did the failures occur?  One almost wonders how the investigators could stop short of systematic and probing interviews of key decision makers.

For example in the maintenance area, “The Board determined that the NWP maintenance and engineering programs have not been effective…”; “Additionally, configuration management was not being maintained or adequately justified when changes were made.”; “There is an acceptance to tolerate or otherwise justify (e.g., lack of funding) out-of-service equipment.” (p. 82)  And that’s where the analysis stops. 

Unfortunately (but predictably) what follows from the constrained analysis are equally unfocused corrective actions based on the following linear construct: “this is a problem - fix the problem”.  Even the corrective action vocabulary becomes numbingly sterile: “needs to take action to ensure…”, “needs to improve…”, “need to develop a performance improvement plan…”,  “needs to take a more proactive role…”.

We do not want to be overly critical as the current report reflects a little over two months of effort and may not have afforded sufficient time to pull the string on so many issues.  But it is time to realize that these types of efforts are not sufficient to understand, and therefore ultimately correct, the issues at WIPP and DOE and institutionalize an effective safety management system.


*  DOE press release, “DOE Issues WIPP Radiological Release Investigation Report” (April 24, 2014)  Retrieved May 5, 2014.

**  Senators Udall and Heinrich press release, “Udall, Heinrich Statement on Department of Energy WIPP Radiological Release Investigation Report” (April 24, 2014).  Retrieved May 5, 2014.

DOE Report on WIPP's Safety Culture

On Feb. 14, 2014, an incident at the Department of Energy (DOE) Waste Isolation Pilot Plant (WIPP) resulted in the release of radioactive americium and plutonium into the environment.  This post reviews DOE’s Phase 1 incident report*, with an emphasis on safety culture (SC) concerns.

From the Executive Summary

The Accident Investigation Board (the Board) concluded that a more thorough hazard analysis, coupled with a better filter system could have prevented the unfiltered above ground release. (p. ES-1)

The root cause of the incident was Nuclear Waste Partnership’s (NWP**, the site contractor) and the DOE Carlsbad Field Office’s (CBFO) failure to manage the radiological hazard. “The cumulative effect of inadequacies in ventilation system design and operability compounded by degradation of key safety management programs and safety culture [emphasis added] resulted in the release of radioactive material . . . and the delayed/ineffective recognition and response to the release.” (pp. ES 6-7)

The report presents eight contributing causes, most of which point to NWP deficiencies.  SC was included as a site-wide concern, specifically the SC does not fully implement DOE safety management policy, “[t]here is a lack of a questioning attitude, reluctance to bring up and document issues, and an acceptance and normalization of degraded equipment and conditions.”  A recent Safety Conscious Work Environment (SCWE) survey suggests a chilled work environment. (p. ES-8)

The report includes 31 conclusions, 4 related to SC.  “NWP and CBFO have allowed the safety culture at the WIPP project to deteriorate . . . Questioning attitudes are not welcomed by management . . . DOE has exacerbated the safety culture problem by referring to numbers of [problem] reports . . . as a measure of [contractor] performance . . . . [NWP and CBFO] failed to identify weaknesses in . . . safety culture.” (pp. ES 14-15, 19-20)

The report includes 47 recommendations (called Judgments of Need) with 4 related to SC.  They cover leadership (including the CBFO site manager) behavior, organizational learning, questioning attitude, more extensive use of existing processes to raise issues, engaging outside SC expertise and improving contractor SC-related processes. (ibid.)

Report Details

The body of the report presents the details behind the conclusions and recommendations.  Following are some of the more interesting SC items, starting with our hot button issues: decision making (esp. the handling of goal conflict), corrective action, compensation and backlogs. 

Decision Making

The introduction to section 5 on SC includes an interesting statement:  “In normal human behavior, production behaviors naturally take precedence over prevention behaviors unless there is a strong safety culture - nurtured by strong leadership.” (p. 61)

The report suggests nature has taken its course: WIPP values production first and most.  “Eighteen emergency management drills and exercises were cancelled in 2013 due to an impact on operations. . . .Management assessments conducted by the contractor have a primary focus on cost and schedule performance.” (p. 62)  “The functional checks on CAMs [continuous air monitors] were often delayed to allow waste-handling activities to continue.” (p. 64)  “[D]ue consideration for prioritization of maintenance of equipment is not given unless there is an immediate impact on the waste emplacement processes.” (p. ES-17)  These observations evidence an imbalance between the goals of production and prevention (against accidents and incidents) and, following the logic of the introductory statement, a weak SC.

Corrective Action

The corrective action program has problems.  “The [Jan. 2013] SCWE Self-Assessment . . . identified weaknesses in teamwork and mutual respect . . . Other than completing the [SCWE] National Training Center course, . . . no other effective corrective actions have been implemented. . . . [The Self-Assessment also ]“identified weaknesses in effective resolution of reported problems.” (p. 63)  For problems that were reported, “The Board noted several instances of reported deficiencies that were either not issued, or for which corrective action plans were not developed or acted on for months.” (p. 65)

Compensation

Here is the complete text of Conclusion 14, which was excerpted above: “DOE has exacerbated the safety culture problem by referring to numbers of ORPS [incident and problem] reports and other deficiency reporting documents, rather than the significance of the events, as a measure of performance by Source Evaluation Boards during contract bid evaluations, and poor scoring on award fee determinations.  Directly tying performance to the number of occurrence reports drives the contractor to non-disclosure of events in order to avoid the poor score. [emphasis added]  This practice is contrary to the Department’s goals of the development and implementation of a strong safety culture across our projects.” (p. ES-15)  ‘Nuff said. 

Backlogs

Maintenance was deferred if it interfered with production.  Equipment and systems were  allowed to degrade (pp. ES-7, ES-17, C-7)  There is no indication that maintenance backlogs were a problem; the work simply wasn’t done.

Other SC Issues

In addition to our Big Four and the issues cited from the Executive Summary, the report mentions the following concerns.  (A listing of all SC deficiencies is presented on p. D-3.)

• Delay in recognizing and responding to events,
• Bias for negative conclusions on Unreviewed Safety Question Determinations, and
• Infrequent presence of NWP management in the underground and surface.

Our Perspective

For starters, the Board appears to have a limited view of what SC is.  They see it as a cause for many of WIPP's problems but it can be fixed if it is “nurtured by strong leadership” and the report's recommendations are implemented.  The recommendations are familiar and can be summed up as “Row harder!”***  In reality, SC is both cause (it creates the context for decision making) and consequence (it is influenced by the observed actions of all organization members, not just senior management).  SC is an organizational property that cannot be managed directly.  

The report is a textbook example of linear, deterministic thinking, especially Appendix E (46 pgs.) on events and causal factors related to the incident.  The report is strong on what happened but weak on why things happened.  Going through Appendix E, SC is a top-level blanket cause of nuclear safety program and radiological event shortcomings (and, to a lesser degree, ventilation, CAMs and ground control problems) but there is no insight into how SC interacts with other organizational variables or with WIPP’s external (political, regulatory, DOE policy) environment. 

Here’s an example of what we’re talking about, viz., how one might gain some greater insight into a problem by casting a wider net and applying a bit of systems thinking.  The report faults DOE HQ for ineffective oversight, providing inadequate resources and not holding CBFO accountable for performance.  The recommended fix is for DOE HQ “to better define and execute their roles and responsibilities” for oversight and other functions. (p. ES-21)  That’s all what and no why.  Is there some basic flaw in the control loop involving DOE HQ, CBFO and NWP?  DOE HQ probably believes it transmits unambiguous orders and expectations through its official documents—why weren’t they being implemented in the field and why didn’t DOE know it?  Is the information flow from DOE to CBFO to NWP clear and adequate (policies, goals); how about the flow in the opposite direction (performance feedback, problems)?  Is something being lost in the translation from one entity to another?  Does this control problem exist between DOE HQ and other sites, i.e., is it a systemic problem?  Who knows.****

Are there other unexamined factors that make WIPP's problems more likely?  For example, has WIPP escaped the scrutiny and centralized controls that DOE applies to other entities?  As a consequence, has WIPP had too much autonomy to adjust its behavior to match its perception of the task environment?  Are DOE’s and WIPP’s mental models of the task environment similar or even adequate?  Perhaps WIPP (and possibly DOE) see the task environment as simpler than it actually is, and therefore the strategies for handling the environment lack requisite variety.  Was there an assumption that NWP would continue the apparently satisfactory performance of the previous contractor?  It's obvious these questions do not specifically address SC but they seek to ascertain how the organizations involved are actually functioning, and SC is an important variable in the overall system.

Contrast with Other DOE SC Investigations 

This report presents a sharp contrast to the foot-dragging that takes place elsewhere in DOE.  Why can’t DOE bring a similar sense of urgency to the SC investigations it is supposed to be conducting at its other facilities?  Was the WIPP incident that big a deal (because it involved a radioactive release) or is it merely something that DOE can wrap its head around?  (After all, WIPP is basically an underground warehouse.)  In any event, something rang DOE’s bell because they quickly assembled a 5 member board with 16 advisor/consultants and produced a 300 page report in less than two months.*****

Bottom line: You don't need to pore over this report but it provides some perspective on how DOE views SC and demonstrates that a giant agency can get moving if it's motivated to do so.


*  DOE Office of Environmental Management, “Accident Investigation Report: Radiological Release Event at the Waste Isolation Pilot Plant on February 14, 2014, Phase 1” (April 2014).  Retrieved April 30, 2014.  Our thanks to Mark Lyons who posted this report on the LinkedIn Nuclear Safety group discussion board.

**  NWP LLC was formed by URS Energy and Construction, Inc. and Babcock & Wilcox Technical Services Group, Inc.  Their major subcontractor is AREVA Federal Services, LLC.  All three firms perform work at other, i.e., non-WIPP, DOE facilities.  NWP assumed management of WIPP on Oct. 1, 2012.  From NWP website.  Retrieved May 2, 2014.

***  To the Board's credit, they did not go looking for individual scapegoats to blame for WIPP's difficulties.

****  In fairness, the report has at least one example of a feedback loop in the CBFO-NWP sub-system: CBFO's use of the condition reports as an input to NWP’s compensation review and NWP's predictable reaction of creating fewer condition reports.

*****  The Accident Investigation Board was appointed on Feb. 27, 2014 and completed its Phase 1 investigation on March 28, 2014.  The Phase 1 report was released to the public on April 22, 2014.

Assessing Safety Culture Using Cultural Attributes

Two weeks ago we posted on NUREG-2165, a document that formalizes a “common language” for describing nuclear safety culture (SC).  The NUREG contains a set of SC traits, attributes that define each trait and examples that would evidence each attribute.  We expressed concern about how traits and attributes could and would be applied in practice to assess SC.

Well, we didn’t have to wait very long.  This post reviews a recent International Nuclear Safety Journal article* that describes the SC oversight process developed by the Romanian nuclear regulatory agency (CNCAN).  The CNCAN process uses the International Atomic Energy Agency (IAEA) SC definition and attributes and illustrates how attributes can be used to evaluate SC.  Note that CNCAN is not attempting to directly regulate SC but they are taking comprehensive steps to evaluate and influence the licensee’s SC.

CNCAN started with the 37 IAEA attributes and decided that 20 were accessible via the normal review and inspection activities.  Some of the 20 could be assessed using licensee and related documentation, others through interviews with licensee and contractor personnel, and others by direct observation of relevant activities. 

CNCAN recognizes there are limitations to using this process, e.g., findings that reflect a reviewer’s subjective opinion, the quality of match (relevance) between an attribute and a specific technical or functional area, the quality of the information gathered and used, and over-reliance on one specific finding.  Time is also an issue.  “[A] large number of review and inspection activities are required, over a relatively long period of time, to gather sufficient data in order to make a judgement on the safety culture of an organisation as a whole.” (p. 4)

However, they are optimistic about longer-term effectiveness.  “. . . evidence of certain attributes not being met for several functional areas and processes would provide a clear indication of a problem that would warrant increased regulatory surveillance.”  In addition, “[t]he implementation of the [oversight process] proved that all the routine regulatory reviews and inspections reveal aspects that are of certain relevance to safety culture.  Interaction with plant staff during the various inspection activities and meetings, as well as the daily observation by the resident inspectors, provide all the necessary elements for having an overall picture of the safety culture of the licensee.” (ibid., emphasis added)

Our Perspective

We reviewed a draft of the CNCAN SC oversight process on March 23, 2012.  We found the treatment of issues we consider important to be generally good.  For example, in the area of decision making, goal conflict is explicitly addressed, from production vs. safety to differing personal opinions.  Corrective action (CA) gets appropriate attention, including CA prioritization based on safety significance and verification that fixes are implemented and effective.  Backlogs in many areas, including maintenance and corrective actions, are addressed.  In general, the treatment is more thorough than the examples included in the NUREG.

However, the treatment of management incentives is weak.  We favor a detailed evaluation of the senior managers’ compensation scheme focusing on how much of their compensation is tied to achieving safety (vs. production or other) goals.

So, do we feel better about the qualms we expressed over the NUREG, viz., that it is a step on the road to the bureaucratization of SC evaluation, a rigid checklist approach that ultimately creates an incomplete and possibly inaccurate picture of a plant’s SC?  Not really.  Our concerns are described below.

Over-simplification

For starters, CNCAN decided to focus on 20 attributes because they believed it was possible to gather relevant information on them.  What about the other 17?  Are they unrelated to SC simply because it might be hard to access them?

A second simplification is limiting the information search to artifacts: documents, interviews and observations.  One does not have to hold some esoteric belief, e.g., that SC is an emergent organizational property that results from the functioning of a socio-technical system, to see that focusing on the artifacts may be similar to the shadows in Plato’s cave.  Early on, the article refers to this problem by quoting from a 1999 NEA report: “the regulator can evaluate the outward operational manifestations of safety culture as well as the quality of work processes, and not the safety culture itself.” (p. 2)

Limited applicability

Romania has a single nuclear plant and what is, at heart, a one-size-fits-all approach is much more practical when “all” equals one.  This type of approach might even work in, say, France, where there are multiple plants but a single operator.  On the other hand, the U.S. currently has 32 operators reporting to 81 owners.**  Developing SC assessment techniques that are comprehensive, consistent and perceived as fair by such a large group is not a simple task.  The U.S. approach will continue to subsume SC evaluation under the ROP, which arguably ties SC evaluation to “objective” safety-related performance but unfortunately leads to de facto regulation of SC, less transparency and incomprehensible results in specific cases.***

(It could be worse.  For an example, just look at DOE where the recent “guidance” on conducting SC self-assessments led to unreliable self-assessment results that can’t be compared with each other.  For more on DOE, see our March 31, 2014 post or click on the DOE label at the bottom of this post.)

Bottom line

Ultimately the article can be summarized as follows: It’s hard, maybe impossible to directly evaluate SC but here’s what we (CNCAN) are doing and we think it works.  We say a CNCAN-style approach may be helpful but one should remain alert to important SC factors that may be overlooked.


*  M. Tronea, “Trends and Challenges in Regulatory Assessment of Nuclear Safety Culture,” International Nuclear Safety Journal, vol. 3 no. 1 (2014), pp. 1-5.  Retrieved April 14, 2014.  Dr. Tronea works for the Romanian nuclear authority (CNCAN) and is the founder/moderator of the LinkedIn Nuclear Safety group.

**  NEI website, retrieved April 15, 2014.

***  For an example, see our Jan. 30, 2013 post on Palisades.