Vit Plant Safety Culture Update

Hanford Waste Treatment Plant

DOE released a June 2014 follow-up assessment* on safety culture (SC) at the Hanford Waste Treatment Plant (WTP or the Vit Plant).  This post provides our perspective on the assessment.  We will not review every facet of the report but will focus on aspects that we think are important to understanding the current state of SC at the WTP project.

Overview

Back in 2011, the Defense Nuclear Facilities Safety Board (DNFSB) called the WTP safety culture (SC) “flawed.”  Following the DNFSB report, DOE conducted an assessment of the WTP SC and concluded “that a significant number of staff within ORP [DOE’s Office of River Protection] and BNI [Bechtel National Inc.] expressed reluctance to raise safety or quality concerns for various reasons.” (p. 1) 

Like DOE’s 2011 report, the current one is based on multiple data sources: structured interviews, focus groups, observations and a culture survey.  The report identifies many SC-related interventions that have been initiated, and lists positive and negative findings from the data collected.  Detailed assessment results are reported separately for ORP and BNI.**
 
WTP’s Safety Conscious Work Environment (SCWE)—Voicing Concerns, Challenging Decisions, Fear of Retaliation

The 2011 DNFSB critique focused on the treatment of project personnel who raised technical issues.  Some of these personnel complained about retaliation for bringing up such issues.  These issues can be raised in the Employee Concerns Program (ECP), the Differing Professional Opinions (DPO) process and challenging management decisions.  In what is arguably the report’s most significant finding, perceptions of conditions in these areas are worse than they were in 2011 for ORP and unchanged for BNI. (pp. 4-5)

Supporting Details

Although ORP senior management pointed to recent reductions in ECP concerns, “[s]ome interviewees indicated that they perceived a chilled environment at ORP and they did not believe that ECP concerns and DPO issues were always addressed or resolved in a timely manner.  Additionally, some interviewees described being told by supervision not to write a DPO because it would be a career limiting decision.” (p. B-21)  Interviewees from both ORP and BNI said they lacked trust in the ECP. (pp. B-22/23)

Most ORP employees believe that constructive criticism is not encouraged. (p. 4)  Within ORP, only 30% of all ORP survey respondents (and 65% of managers) feel that they can openly challenge decisions made by management. (p. B-21)  In BNI/URS, the numbers are 45% of all respondents and 75% of managers. (p. B-22)

“The statement that management does not tolerate retaliation of any kind for raising concerns was agreed to by approximately 80% of the ORP, . . . 72% of the BNI and 80% of the URS survey respondents.” (pp. B-21-22)  In addition, “Anonymous PIERs [Project Issues Evaluation Reports] are used a lot because of fear of retaliation.” (p. B-23)  

All in all, hardly a ringing endorsement of the WTP SCWE.

Decision Making, Corrective Action and Compensation

Safetymatters readers know of our long-standing interest in how SC is reflected in these key artifacts.

Decision Making

“Interviewees provided some examples of where decision making was not perceived to reflect the highest commitment to safety”  Examples included downgrading or elimination of assessment findings, the margin of safety in corrective action plans and the acceptable level of risk for the project.  (p. B-9)  Looks like there's some room for improvement in this area.

Corrective Action

Within BNI, there are positive comments about the corrective action process but the assessment team “observed a lack of accountability for a backlog of corrective actions at a PIRB [Performance Improvement Review Board] meeting.  There is a perceived lack of accountability for corrective actions in timeliness, ownership, and quality, ...” (p. B-16)

“[O]nly about 18% of all ORP interviewee respondents believed that employees are encouraged to notify management of problems they observe and that there is a system that evaluates the problem and makes a determination regarding future action.” (p. 13)

Plenty of room for improvement here.

Compensation

In the area of compensation, it appears some rewards for Bechtel are based on SC behaviors. (p. B-8)  We consider that a very positive development.

Concerns Over ORP Working More Collaboratively with Bechtel

On the surface this looks like a positive change: two entities working together to achieve a common goal.  However, this has led to at least two concerns.  First, as described in the report, some ORP personnel believe ORP is abdicating or compromising its responsibility to perform oversight of Bechtel, in other words, ORP is more of a teammate and less of an umpire. (pp. B-4, -9, -14)  Second, and this reflects our perspective, changing the relationship between the entities can result in revised system dynamics, with old performance-oversight feedback loops replaced by new ones.  The rules of engagement have changed and while safety may still be the number one priority, the cultural milieu in which safety is achieved has also changed.

Another complication is caused by the role of Bechtel Corporate.  The report says corporate’s values and goals may not be well-aligned with BNI’s need to prioritize SC attributes and behavior.  This can lead to a lack of transparency in BNI decisions. (pp. 5, 7)  That may be a bit of weasel wording in the report; in more direct terms, corporate’s number one priority is for the money train to keep running.

Bottom Line on SC at WTP 

A strong SC is, in some ways, about respect for the individual.  The concerns that WTP personnel  express about using the ECP or DPO process, or challenging management decisions suggest that the WTP project has a ways to go to inculcate an adequate level of such respect.  More importantly, it doesn’t appear they have made any significant process toward that goal in the last few years.

Morale is an aspect of the overall culture and at the WTP, morale is arguably low because of lack of progress and missed schedules. (pp. 6, B-10)  There is undoubtedly plenty of work to do but on a day to day basis, we wouldn’t be surprised if some people feel their work is not meaningful.


*  DOE Office of Environment, Safety and Health Assessments (now the Office of Independent Enterprise Assessments), “Independent Oversight Follow-up Assessment of Safety Culture at the Waste Treatment and Immobilization Plant” (June, 2014).

**  ORP has line management responsibility for the WTP, BNI is the primary contractor and URS Corporation (URS) is a major subcontractor.

Catching Up on DOE’s SCWE Extent of Condition Review

Hanford Waste Treatment Plant

On May 29, 2014 DOE submitted its partial response to the Defense Nuclear Facilities Safety Board (DNFSB) Recommendation 2011-1 in a report* on DOE’s Safety Conscious Work Environment (SCWE) extent of condition review and recommended actions for ongoing safety culture (SC) improvement at DOE facilities.

(Quick history: The June 9, 2011 DNFSB report on DOE’s Hanford Waste Treatment Plant (WTP or the Vit Plant) said the WTP SC was “flawed.”  The report’s recommendations included that DOE should conduct an extent of condition review to determine whether WTP SC weaknesses existed at other DOE facilities.  DOE agreed to perform the review but focused on SCWE because, in DOE’s view, the issues at WTP were primarily SCWE related.)

This post summarizes the report’s findings then parses the details and provides our perspective.

 Report Findings and Recommendations

The report was based on data from eleven independent SC assessments and thirty-one SCWE self-assessments conducted by individual organizations.  The DOE review team processed the data through their analyzer and homogenizer to identify four primary SC attributes** to focus on for continuous improvement in DOE:

• Demonstrated safety leadership
• Open communication and fostering an environment free from retribution
• Teamwork and mutual respect
• Credibility, trust and reporting errors and problems

Further processing through the bureaucratizer yielded three recommended actions to improve the SC attributes:

• Form a DOE SC Improvement Panel to ensure leadership and focus on DOE's SC initiatives
• Incorporate SC and SCWE concepts and practices into DOE training
• Evaluate contract language to incorporate clear references to SC  (pp. 3-4)

Our Perspective

We reviewed the DOE independent assessments on January 25, 2013 and the self-assessments on March 31, 2014.  From the former we concluded that issues similar to those found at the WTP existed at other DOE facilities, but to a lesser degree than WTP.  The self-assessments were of such varying quality and credibility that we basically couldn’t infer anything.***

The Analyzer

The DOE team reviewed all the assessments to identify specific issues (problems).  The team binned issues under the SC attributes in DOE's Integrated Safety Management System Guide and then counted the number of issues under each attribute; a higher count meant a more serious problem.  They performed a similar exercise to identify positive organizational trends (strengths) mentioned in the assessments. 

We could be picky and ask if all the issues (or strengths) were of essentially the same importance or magnitude but the team had a lot of data to review so we’ll let that slide.  Concurrently identifying strengths was a good idea; it harkens back to Peter Drucker who advised managers to build on strengths.****

The Homogenizer

Here’s where we begin to have problems.  The team focused on identifying SC attributes and developing recommendations that applied to or affected the entire Department, essentially boiling their results down to a one-size-fits-all approach.  However, their own data belies that approach.  For example, the Leadership attribute “Open communication and fostering an environment free from retribution” was identified as both an issue AND a strength. 

In plain English, some organizations don’t exhibit the desired communication attribute and others do.  One proper fix is to identify who is doing it right, define what exactly they’re doing, and develop a method for transferring that approach to the problem organizations.  The report even says this attribute “is an area in which management can learn lessons from those sites where it was deemed a strength so that best practices can become commonplace” but this statement is buried at the end of the report. (p. 22)

The DOE entities exhibit a wide variety of scale, scope, mission and organizational and technical complexity.  The Department’s goal should be to recognize that reality, develop it as an overall strength and then build on it to create site- or organization-specific interventions.

The Bureaucratizer

The proposed fixes would basically create a top-level coordinating and oversight group, enhance SC and SCWE training and modify contracts.  The recommendations reflect a concerted search for the lowest common denominator and a minimum amount of real change.

If the SC Improvement Panel is established, it should focus on setting or refining SC policy and ensuring those policies are implemented by line management, especially field management.  They should also be involved in evaluating major SC issues.  If things aren’t going well, this group should be the first to ask the hard “Why?” questions.  But most of the panel’s proposed tasks, viz., maintaining SC visibility, providing a forum for evaluating SC status and overseeing training improvements, are low-value make work.

“[S]afety culture training for all personnel, up and down the management chain, will be updated and/ or developed to ensure that roles and responsibilities are understood and personnel have the capabilities needed to play their part in continuously improving DOE's safety culture; . . .” (p. 24)  This is a standard fix for almost any perceived organizational problem.  It doesn’t require managers to do anything different.

Modifying contracts to incorporate clear references to SC is only a beginning.  What are the carrots and the sticks to incent the contractors to actually develop, measure and maintain an effective SCWE and strong SC?

Bottom Line

This report comports completely with an organization that resembles a fifty foot sponge.  You can kick it as hard as you like, your foot goes in deep and you think you’ve had an effect, but when you withdraw your foot, the organization fills in the hole like your kick never happened.  I thought I heard a loud pop on May 29.  I now realize it was likely DNFSB Chairman Winokur’s head exploding when he read this report.

But Wait, There’s More

Remember the question the DNFSB initially asked in 2011: Do WTP SC issues exist elsewhere in DOE?  Well, the answer is: “Review of assessment results from both [independent and self assessments] indicated there is a SCWE extent of condition that requires additional and ongoing actions to improve performance.” (p. 28, the penultimate page of the report)


*  J. Hutton (DOE) to P.S. Winokur (DNFSB) May 29, 2014 letter transmitting DOE Consolidated Report for DNFSB Recommendation 2011-1, Actions 2-8 and 2-9 (May 2014).

** DOE considers SC in three focus areas: Leadership, Employee Engagement and Organizational Learning.  Each focus area has a set of associated attributes that describe what a positive SC should look like. (pp. 4-5)

***  The report puts the self-assessments in the best possible light by describing them as learning experiences for the organizations involved. (p.9)

****  Drucker was referring to individuals but, in this case, we’ll stretch the blanket to cover organizations.  For individuals, weaknesses should not be ignored but the energy required to fix them, if it’s even possible, is often too great.  However, one should keep an eye on such weaknesses and not allow them to lead to performance failure.

Regulatory Oversight of Safety Culture in Belgium

The latest International Nuclear Safety Journal has an article* by Benoit Bernard describing a new safety culture (SC) regulatory oversight process now in use in Belgium.  It is based on observations of SC during interactions with a licensee.  This post describes the process and the rationale for it, followed by our perspective.

Bernard starts with a brief history of SC in the nuclear industry then describes two types of regulation currently used, compliance-based and performance-based, and highlights the shortcomings of each.  “Compliance-based” regulation is focused on a licensee’s control of isolated technical components.  This traditional approach can lead to an “adversarial legalism” between the regulator and the licensee, discourage open communication and fail to promote continuous improvement.  In contrast, “performance-based” regulation is based on specific outcomes the licensee is expected to achieve.  The regulator focuses on monitoring outcomes.  This is a reactive approach that can tend to concentrate on well-known risks or familiar issues, and ignore emergent new issues.  Both approaches are inadequate to deal with human factors issues.

The New Process

The author notes “Safety culture cannot be directly regulated but it can be observed . . . [The new Belgian approach] is based on field observations provided by inspectors or safety analysts during any contact with a licensee (inspections, meetings, phone calls…).” (p. 3)  It is expected to be more proactive and systemic than the earlier regulatory approaches.

The process has both short-term and longer-term applications.  In the short term, the purpose is to identify findings that require more-or-less immediate licensee attention or action.  In the longer term, SC observations are input to the overall oversight process. (p. 4)

Observations focus on both facts (what happened) and context (the circumstances surrounding an event).  The approach leads to “Why?” questions rather than degree of compliance with defined SC attributes.  For example, if someone doesn’t follow a rule, is it because of bad behavior or a bad rule?  Was there inadequate training or task-specific knowledge, an inadequate procedure, poor documentation or lack of management commitment to SC?  “The important point is to . . . shed light on the underlying reasons as to why the rules were ignored. . . . [L]inking an observation to an attribute must not be considered as an end but as a starting point to further questions.” (p. 7)

Bernard goes on to describe three aspects of SC that an overall assessment must address: Integration, Differentiation and Fragmentation.  “Integration” refers to the “level of consensus concerning a set of values unifying people and reflected in practices and management systems.” (p. 8)  Prior to the annual SC review with a licensee, SC observations are assessed through four key safety dimensions: Management, Organization, Workplace Practices and Behavior.  You probably can’t read the figure below but each dimension has two component factors, e.g., Management consists of “Management system” and “Leadership,” and each factor appears under two different dimensions, e.g., “Management system” appears under Management and Organization.  Each factor also has several attributes.  This is where the rubber meets the road so think about the training, teamwork, supervision and overall effort required to get regulatory observers (who are more likely to be technical experts than social scientists) to reliably associate specific observations with the correct dimension(s), factor(s) and attribute(s) and then integrate their findings into an overall SC assessment.



“Differentiation” refers to “the ability of [sub-]groups to share a common definition of problems” and “Fragmentation” refers to the “contrast of perceptions and contradictions [across an organization] about what is safe or dangerous.” (p. 9)

Comparison with Romanian Approach

If this topic sounds familiar, on April 21, 2014 we posted on an SC oversight process developed by the Romanian nuclear regulatory agency (CNCAN).  The CNCAN approach looks at artifacts (documents, interviews and observations) to develop an overall, longer-term perspective on SC.

CNCAN recognizes there are limitations to using their process including findings that reflect a reviewer’s subjective opinion and over-reliance on one specific finding.  The Belgian paper recognizes that training technically-oriented reviewers to become competent observers is a challenge.  But Bernard also appears to promote the possibility of “one specific finding” being an early warning, a leading indicator of problems.

Bernard explicitly states this is not a one-size-fits all approach.  The search for event context implies a type of customization of the process for each licensee.  The author says the result is “a regulation style responding to the reference framework of a particular licensee.” (p. 9)

Belgium has seven operating units at two sites, both sites owned and managed by Electrabel, a Belgium-based energy company.  A customized approach may work in Belgium.  But as we noted in our review of the CNCAN approach, “the U.S. currently has 32 operators reporting to 81 owners. Developing SC assessment techniques that are comprehensive, consistent and perceived as fair by such a large group is not a simple task.”

Our Perspective

This is a good paper for its comprehensive discussion of nuclear SC in general and its description of two existing regulatory world views. 

But we have some concerns with the SC observation process.  As noted above, training observers is a major challenge and we think it would be very difficult to adopt such a process in the relatively fragmented U.S. nuclear industry.

In addition, observations are a very soft artifact (compared to documents or even structured interviews) and thus open to to misunderstandings, observational errors and false positives.  It’s easy to imagine a licensee being sent off on a wild goose chase after a regulator misreads one (or more) interactions with licensee personnel.

Furthermore, as instant observations become used as leading indicators, the process could become more like a backseat driver commenting on every turn of the steering wheel.  Licensees might oversteer in their attempt to get back into this new type of compliance.  The risk is the observational process begins to intrude on day-to-day management.  And, at some point, ownership of plant SC could subtly shift from the licensee to the regulator.

Finally, although we constantly chide the industry for concentrating on the “what” and ignoring the “why” associated with incidents or infractions, it’s also clear that the pendulum could swing too far in the other direction.  In plain language, not every minor issue merits an in-depth “why” investigation; that can be a route to over-use of resources and organizational paralysis.

We’re not condemning this as a bad idea.  But a regulatory user (and licensees) should be alert to the possibility of unintended consequences.


**  B. Bernard, “Safety Culture as a Way of Responsive Regulation: Proposal for a Nuclear Safety Culture Oversight Model,” International Nuclear Safety Journal vol. 3 no. 2 (2014) pp. 1-11.  Thanks to Madalina Tronea for promoting this journal.  Dr. Tronea is the founder/moderator of the LinkedIn Nuclear Safety group.

NRC Non-Concurrence Process Assessment: Tempest in a Teapot?

On June 4, 2014 the NRC announced a revised agency-wide non-concurrence process (NCP) on their blog.*  A key objective of the NCP is “to ensure that a non-concurrence is heard, understood, and considered by employees included in the concurrence process so that the non-concurrence informs and supports the decisionmaking process.”**

The NRC performed an assessment*** of the prior NCP using multiple data sources, including the NRC’s 2012 Safety Culture and Climate Survey (SCCS) and an April 2013 survey targeted at employees who had been involved with the NCP as submitters or participants (employees who have responded to non-concurrences).

The assessment identified both strengths and weaknesses with the then-existing NCP.  In general, participants were aware of the NCP and were willing to use it.  However, “some users of the process felt they faced negative consequences, or that their views were not reflected in final decisions.” (blog post)  The assessment also included a bevy of planned actions to address NCP weaknesses.

For us, the interesting question is what does the assessment say or what can be inferred, if anything, with respect to the NRC’s safety culture (SC).  This post focuses on SC-related topics mentioned in the assessment that help us answer that question.

Leadership Commitment

Leadership commitment is an area of concern and planned actions. (p. 4)  “Data from several sources indicates that many of the responding employees are still uncertain about management’s support of the NCP. . . . management was just going through the motions. . . .[some employees] thought the process was biased . . .supervisors using the process indicated that they were concerned management would view it as a negative reflection on them [the supervisors].” (p. 11)  In the targeted survey, “more than half of submitters are concerned about management’s support of the NCP.” (p. 7)

Planned actions include “support managers in emphasizing their personal commitment to the welcoming of sharing differing views and the value of using the NCP in support of sound regulatory decisionmaking. . . . Management should demonstrate this [NCP is a positive] clearly and frequently through their actions and communications. . . . Staff will continue to support a variety of outreach activities and communication tools, such as EDO Updates, monthly senior management meetings, all-supervisor meetings, senior leadership meetings, Yellow Announcements, all-hands meetings, brown bag lunches, seminars, and articles in the NRC Reporter and office-level newsletters.” (p. 18)  Whew!

Potential Negative Consequences of Submitting a NCP

From the SCCS report the assessment highlights that “Forty-nine percent of employees believe that the NCP is effective (37 percent don’t have an opinion on the effectiveness of the NCP and 14 percent believe that the NCP is not effective).” (p. 6)  That 14 percent looks low but because there are only about a dozen NCP filings per year, it might actually reflect that a lot of people who use the process end up disappointed.  That view is supported by the targeted survey where “the majority of submitters believed that the rationale for the outcome was not clearly documented and that they experienced negative consequences as a result of submitting a non-concurrence.” (p. 7)

We reviewed the SCCS on April 6, 2013.  We noted that “The consultants' cover letter identified this [DPO/NCP] as an area for NRC management attention, saying the agency was “Losing significant ground on negative reactions when raising views different from senior management, supervisor, and peers.””

Planned actions include “proactively fostering an environment that encourages and supports differing views . . . evaluating the merits of infusing NCP key messages into existing training, including reinforcing that supervisors and managers will be held accountable for their actions. . . . consider training for all supervisors to address concerns of retaliation and chilling effect for engaging in the NCP. . . . hosting panel discussions including previous NCP submitters and participants . . . promote NCP success stories . . . evaluating the merits of establishing an anti-retaliation policy and procedures to address concerns of retaliation and chilling effect for engaging in the NCP. (p. 20)  Note these are all staff activities, management doesn’t have to do anything except go along with the program.

Goal Conflict

Goal conflict is another problem area.  The assessment notes “many responding employees commented they felt pressure to meet schedules at the expense of quality.” (p. 17)  That issue was also highlighted in the 2012 SCCS and could well be the source for the comment in the assessment.

Our Perspective

An effective NCP is important.  We believe NCP or some functionally equivalent practice should be more widely utilized in the world of formal organizations.

But it is easy to read too much into the NCP assessment.  The primary data input was the 2012 SCCS and that is relatively old news.  Another key input was the targeted survey.  However, the number of survey respondents was small because only a handful of people use the NCP.****  Based on the negative responses of the submitters, it appears that NRC needs to do a better job of administering the NCP, especially in the areas of (1) convincing submitters that their concerns were actually considered (even if ultimately rejected) and (2) ensuring there are no negative consequences associated with using the NCP.  These are real process implementation challenges but the NCP-related issues do not reflect some major, new problem in the agency’s SC.

On the other hand, perceptions of negative responses to rocking the boat in general or senior management’s lack of commitment to inclusive programs and “safety first” are SC signals to which attention must be paid.  If Staff trains their 10 gauge shotgun of interventions on these possibly systemic issues then some actual good could come out of this.


*  NRC blog “Improving NRC’s Internal Processes” (June 4, 2014).  Retrieved June 12, 2014.

**  NRC Non-Concurrence Process, Management Directive 10.158 (Mar. 14, 2014).  ADAMS ML13176A371.

“Non-concurrence” means an employee has a problem with a document the employee had a role in creating or reviewing.  For example, the employee might hold a different view on a technical matter or disagree with a proposed decision.

The NCP appears to be more formal and documented than the NRC Open Door policy and less restrictive than the Differing Professional Opinions (DPO) program which is reserved for concerns on established NRC positions.

***  NRC Office of Enforcement, “2014 Non-Concurrence Process Assessment”  (June 4, 2014).  ADAMS ML14056A294.

****  The survey was issued to 39 submitters (24 responded [62%]) and 62 participants (17 responded [27%]).

DOE Report on WIPP's Safety Culture

On Feb. 14, 2014, an incident at the Department of Energy (DOE) Waste Isolation Pilot Plant (WIPP) resulted in the release of radioactive americium and plutonium into the environment.  This post reviews DOE’s Phase 1 incident report*, with an emphasis on safety culture (SC) concerns.

From the Executive Summary

The Accident Investigation Board (the Board) concluded that a more thorough hazard analysis, coupled with a better filter system could have prevented the unfiltered above ground release. (p. ES-1)

The root cause of the incident was Nuclear Waste Partnership’s (NWP**, the site contractor) and the DOE Carlsbad Field Office’s (CBFO) failure to manage the radiological hazard. “The cumulative effect of inadequacies in ventilation system design and operability compounded by degradation of key safety management programs and safety culture [emphasis added] resulted in the release of radioactive material . . . and the delayed/ineffective recognition and response to the release.” (pp. ES 6-7)

The report presents eight contributing causes, most of which point to NWP deficiencies.  SC was included as a site-wide concern, specifically the SC does not fully implement DOE safety management policy, “[t]here is a lack of a questioning attitude, reluctance to bring up and document issues, and an acceptance and normalization of degraded equipment and conditions.”  A recent Safety Conscious Work Environment (SCWE) survey suggests a chilled work environment. (p. ES-8)

The report includes 31 conclusions, 4 related to SC.  “NWP and CBFO have allowed the safety culture at the WIPP project to deteriorate . . . Questioning attitudes are not welcomed by management . . . DOE has exacerbated the safety culture problem by referring to numbers of [problem] reports . . . as a measure of [contractor] performance . . . . [NWP and CBFO] failed to identify weaknesses in . . . safety culture.” (pp. ES 14-15, 19-20)

The report includes 47 recommendations (called Judgments of Need) with 4 related to SC.  They cover leadership (including the CBFO site manager) behavior, organizational learning, questioning attitude, more extensive use of existing processes to raise issues, engaging outside SC expertise and improving contractor SC-related processes. (ibid.)

Report Details

The body of the report presents the details behind the conclusions and recommendations.  Following are some of the more interesting SC items, starting with our hot button issues: decision making (esp. the handling of goal conflict), corrective action, compensation and backlogs. 

Decision Making

The introduction to section 5 on SC includes an interesting statement:  “In normal human behavior, production behaviors naturally take precedence over prevention behaviors unless there is a strong safety culture - nurtured by strong leadership.” (p. 61)

The report suggests nature has taken its course: WIPP values production first and most.  “Eighteen emergency management drills and exercises were cancelled in 2013 due to an impact on operations. . . .Management assessments conducted by the contractor have a primary focus on cost and schedule performance.” (p. 62)  “The functional checks on CAMs [continuous air monitors] were often delayed to allow waste-handling activities to continue.” (p. 64)  “[D]ue consideration for prioritization of maintenance of equipment is not given unless there is an immediate impact on the waste emplacement processes.” (p. ES-17)  These observations evidence an imbalance between the goals of production and prevention (against accidents and incidents) and, following the logic of the introductory statement, a weak SC.

Corrective Action

The corrective action program has problems.  “The [Jan. 2013] SCWE Self-Assessment . . . identified weaknesses in teamwork and mutual respect . . . Other than completing the [SCWE] National Training Center course, . . . no other effective corrective actions have been implemented. . . . [The Self-Assessment also ]“identified weaknesses in effective resolution of reported problems.” (p. 63)  For problems that were reported, “The Board noted several instances of reported deficiencies that were either not issued, or for which corrective action plans were not developed or acted on for months.” (p. 65)

Compensation

Here is the complete text of Conclusion 14, which was excerpted above: “DOE has exacerbated the safety culture problem by referring to numbers of ORPS [incident and problem] reports and other deficiency reporting documents, rather than the significance of the events, as a measure of performance by Source Evaluation Boards during contract bid evaluations, and poor scoring on award fee determinations.  Directly tying performance to the number of occurrence reports drives the contractor to non-disclosure of events in order to avoid the poor score. [emphasis added]  This practice is contrary to the Department’s goals of the development and implementation of a strong safety culture across our projects.” (p. ES-15)  ‘Nuff said. 

Backlogs

Maintenance was deferred if it interfered with production.  Equipment and systems were  allowed to degrade (pp. ES-7, ES-17, C-7)  There is no indication that maintenance backlogs were a problem; the work simply wasn’t done.

Other SC Issues

In addition to our Big Four and the issues cited from the Executive Summary, the report mentions the following concerns.  (A listing of all SC deficiencies is presented on p. D-3.)

• Delay in recognizing and responding to events,
• Bias for negative conclusions on Unreviewed Safety Question Determinations, and
• Infrequent presence of NWP management in the underground and surface.

Our Perspective

For starters, the Board appears to have a limited view of what SC is.  They see it as a cause for many of WIPP's problems but it can be fixed if it is “nurtured by strong leadership” and the report's recommendations are implemented.  The recommendations are familiar and can be summed up as “Row harder!”***  In reality, SC is both cause (it creates the context for decision making) and consequence (it is influenced by the observed actions of all organization members, not just senior management).  SC is an organizational property that cannot be managed directly.  

The report is a textbook example of linear, deterministic thinking, especially Appendix E (46 pgs.) on events and causal factors related to the incident.  The report is strong on what happened but weak on why things happened.  Going through Appendix E, SC is a top-level blanket cause of nuclear safety program and radiological event shortcomings (and, to a lesser degree, ventilation, CAMs and ground control problems) but there is no insight into how SC interacts with other organizational variables or with WIPP’s external (political, regulatory, DOE policy) environment. 

Here’s an example of what we’re talking about, viz., how one might gain some greater insight into a problem by casting a wider net and applying a bit of systems thinking.  The report faults DOE HQ for ineffective oversight, providing inadequate resources and not holding CBFO accountable for performance.  The recommended fix is for DOE HQ “to better define and execute their roles and responsibilities” for oversight and other functions. (p. ES-21)  That’s all what and no why.  Is there some basic flaw in the control loop involving DOE HQ, CBFO and NWP?  DOE HQ probably believes it transmits unambiguous orders and expectations through its official documents—why weren’t they being implemented in the field and why didn’t DOE know it?  Is the information flow from DOE to CBFO to NWP clear and adequate (policies, goals); how about the flow in the opposite direction (performance feedback, problems)?  Is something being lost in the translation from one entity to another?  Does this control problem exist between DOE HQ and other sites, i.e., is it a systemic problem?  Who knows.****

Are there other unexamined factors that make WIPP's problems more likely?  For example, has WIPP escaped the scrutiny and centralized controls that DOE applies to other entities?  As a consequence, has WIPP had too much autonomy to adjust its behavior to match its perception of the task environment?  Are DOE’s and WIPP’s mental models of the task environment similar or even adequate?  Perhaps WIPP (and possibly DOE) see the task environment as simpler than it actually is, and therefore the strategies for handling the environment lack requisite variety.  Was there an assumption that NWP would continue the apparently satisfactory performance of the previous contractor?  It's obvious these questions do not specifically address SC but they seek to ascertain how the organizations involved are actually functioning, and SC is an important variable in the overall system.

Contrast with Other DOE SC Investigations 

This report presents a sharp contrast to the foot-dragging that takes place elsewhere in DOE.  Why can’t DOE bring a similar sense of urgency to the SC investigations it is supposed to be conducting at its other facilities?  Was the WIPP incident that big a deal (because it involved a radioactive release) or is it merely something that DOE can wrap its head around?  (After all, WIPP is basically an underground warehouse.)  In any event, something rang DOE’s bell because they quickly assembled a 5 member board with 16 advisor/consultants and produced a 300 page report in less than two months.*****

Bottom line: You don't need to pore over this report but it provides some perspective on how DOE views SC and demonstrates that a giant agency can get moving if it's motivated to do so.


*  DOE Office of Environmental Management, “Accident Investigation Report: Radiological Release Event at the Waste Isolation Pilot Plant on February 14, 2014, Phase 1” (April 2014).  Retrieved April 30, 2014.  Our thanks to Mark Lyons who posted this report on the LinkedIn Nuclear Safety group discussion board.

**  NWP LLC was formed by URS Energy and Construction, Inc. and Babcock & Wilcox Technical Services Group, Inc.  Their major subcontractor is AREVA Federal Services, LLC.  All three firms perform work at other, i.e., non-WIPP, DOE facilities.  NWP assumed management of WIPP on Oct. 1, 2012.  From NWP website.  Retrieved May 2, 2014.

***  To the Board's credit, they did not go looking for individual scapegoats to blame for WIPP's difficulties.

****  In fairness, the report has at least one example of a feedback loop in the CBFO-NWP sub-system: CBFO's use of the condition reports as an input to NWP’s compensation review and NWP's predictable reaction of creating fewer condition reports.

*****  The Accident Investigation Board was appointed on Feb. 27, 2014 and completed its Phase 1 investigation on March 28, 2014.  The Phase 1 report was released to the public on April 22, 2014.

Assessing Safety Culture Using Cultural Attributes

Two weeks ago we posted on NUREG-2165, a document that formalizes a “common language” for describing nuclear safety culture (SC).  The NUREG contains a set of SC traits, attributes that define each trait and examples that would evidence each attribute.  We expressed concern about how traits and attributes could and would be applied in practice to assess SC.

Well, we didn’t have to wait very long.  This post reviews a recent International Nuclear Safety Journal article* that describes the SC oversight process developed by the Romanian nuclear regulatory agency (CNCAN).  The CNCAN process uses the International Atomic Energy Agency (IAEA) SC definition and attributes and illustrates how attributes can be used to evaluate SC.  Note that CNCAN is not attempting to directly regulate SC but they are taking comprehensive steps to evaluate and influence the licensee’s SC.

CNCAN started with the 37 IAEA attributes and decided that 20 were accessible via the normal review and inspection activities.  Some of the 20 could be assessed using licensee and related documentation, others through interviews with licensee and contractor personnel, and others by direct observation of relevant activities. 

CNCAN recognizes there are limitations to using this process, e.g., findings that reflect a reviewer’s subjective opinion, the quality of match (relevance) between an attribute and a specific technical or functional area, the quality of the information gathered and used, and over-reliance on one specific finding.  Time is also an issue.  “[A] large number of review and inspection activities are required, over a relatively long period of time, to gather sufficient data in order to make a judgement on the safety culture of an organisation as a whole.” (p. 4)

However, they are optimistic about longer-term effectiveness.  “. . . evidence of certain attributes not being met for several functional areas and processes would provide a clear indication of a problem that would warrant increased regulatory surveillance.”  In addition, “[t]he implementation of the [oversight process] proved that all the routine regulatory reviews and inspections reveal aspects that are of certain relevance to safety culture.  Interaction with plant staff during the various inspection activities and meetings, as well as the daily observation by the resident inspectors, provide all the necessary elements for having an overall picture of the safety culture of the licensee.” (ibid., emphasis added)

Our Perspective

We reviewed a draft of the CNCAN SC oversight process on March 23, 2012.  We found the treatment of issues we consider important to be generally good.  For example, in the area of decision making, goal conflict is explicitly addressed, from production vs. safety to differing personal opinions.  Corrective action (CA) gets appropriate attention, including CA prioritization based on safety significance and verification that fixes are implemented and effective.  Backlogs in many areas, including maintenance and corrective actions, are addressed.  In general, the treatment is more thorough than the examples included in the NUREG.

However, the treatment of management incentives is weak.  We favor a detailed evaluation of the senior managers’ compensation scheme focusing on how much of their compensation is tied to achieving safety (vs. production or other) goals.

So, do we feel better about the qualms we expressed over the NUREG, viz., that it is a step on the road to the bureaucratization of SC evaluation, a rigid checklist approach that ultimately creates an incomplete and possibly inaccurate picture of a plant’s SC?  Not really.  Our concerns are described below.

Over-simplification

For starters, CNCAN decided to focus on 20 attributes because they believed it was possible to gather relevant information on them.  What about the other 17?  Are they unrelated to SC simply because it might be hard to access them?

A second simplification is limiting the information search to artifacts: documents, interviews and observations.  One does not have to hold some esoteric belief, e.g., that SC is an emergent organizational property that results from the functioning of a socio-technical system, to see that focusing on the artifacts may be similar to the shadows in Plato’s cave.  Early on, the article refers to this problem by quoting from a 1999 NEA report: “the regulator can evaluate the outward operational manifestations of safety culture as well as the quality of work processes, and not the safety culture itself.” (p. 2)

Limited applicability

Romania has a single nuclear plant and what is, at heart, a one-size-fits-all approach is much more practical when “all” equals one.  This type of approach might even work in, say, France, where there are multiple plants but a single operator.  On the other hand, the U.S. currently has 32 operators reporting to 81 owners.**  Developing SC assessment techniques that are comprehensive, consistent and perceived as fair by such a large group is not a simple task.  The U.S. approach will continue to subsume SC evaluation under the ROP, which arguably ties SC evaluation to “objective” safety-related performance but unfortunately leads to de facto regulation of SC, less transparency and incomprehensible results in specific cases.***

(It could be worse.  For an example, just look at DOE where the recent “guidance” on conducting SC self-assessments led to unreliable self-assessment results that can’t be compared with each other.  For more on DOE, see our March 31, 2014 post or click on the DOE label at the bottom of this post.)

Bottom line

Ultimately the article can be summarized as follows: It’s hard, maybe impossible to directly evaluate SC but here’s what we (CNCAN) are doing and we think it works.  We say a CNCAN-style approach may be helpful but one should remain alert to important SC factors that may be overlooked.


*  M. Tronea, “Trends and Challenges in Regulatory Assessment of Nuclear Safety Culture,” International Nuclear Safety Journal, vol. 3 no. 1 (2014), pp. 1-5.  Retrieved April 14, 2014.  Dr. Tronea works for the Romanian nuclear authority (CNCAN) and is the founder/moderator of the LinkedIn Nuclear Safety group.

**  NEI website, retrieved April 15, 2014.

***  For an example, see our Jan. 30, 2013 post on Palisades. 

Our Gaze Returns to DOE and its Safety Culture

The Department of Energy (DOE) recently submitted a report* to the Defense Nuclear Facilities Safety Board (DNFSB) covering DOE’s evaluation of Safety Conscious Work Environment (SCWE) self-assessments at various DOE facilities.  This evaluation was included in the DOE’s Implementation Plan** (IP) developed in response to the DNFSB report, Safety Culture at the Waste Treatment and Immobilization Plant.*** (WTP, or the Vit Plant).  This post provides some background on how WTP safety culture (SC) problems led to a wider assessment of SC in DOE facilities and then reviews the current report.

Background

The DNFSB report on the WTP was issued June 9, 2011; it said the WTP SC was “flawed.”  Issues included discouraging technical dissent, goal conflicts between schedule/budget and safety, and intimidation of personnel.  We posted on the DNFSB report June 15, 2011.  The report’s recommendations included this one: that the Secretary of Energy “conduct an Extent of Condition Review to determine whether these safety culture weaknesses are limited to the WTP Project, . . .” (DNFSB, p. 6) 

After some back-and-forth between DOE and DNFSB, DOE published their IP in December 2011.  We reviewed the IP on Jan. 24, 2012.  Although the IP contained multiple action items, our overall impression was “that DOE believes there is no fundamental safety culture issue. . . . While endlessly citing all the initiatives previously taken or underway, never does the DOE reflect on why these initiatives have not been effective to date.”  So we were not exactly optimistic but DOE did say it would “conduct an Extent of Condition Review to find out whether similar safety culture weaknesses exist at other sites in addition to the WTP and whether there are barriers to strong safety culture at Headquarters and the Department as a whole (e.g., policies or implementation issues). The review will focus on the Safety Conscious Work Environment (SCWE) at each site examined.” (IP, p. 17)  In other words, SC was reduced to SCWE from the get-go.****

Part of the DOE review was to assess SCWE at a group of selected DOE facilities.  DOE submitted SC assessments covering five facilities to DNFSB on Dec. 12, 2012.  We reviewed the package in our post Jan. 25, 2013 and observed “The DOE submittal contained no meta-analysis of the five assessments, and no comparison to Vit Plant concerns.  As far as [we] can tell, the individual assessments made no attempt to focus on whether or not Vit Plant concerns existed at the reviewed facilities.”  We called the submittal “foot dragging” by DOE.

Report on SCWE Self-Assessments

A related DOE commitment was to perform SCWE self-assessments at numerous DOE facilities and then evaluate the results to determine if SCWE issues similar to WTP’s existed elsewhere.  It is important to understand that this latest report is really only the starting point for evaluating the self-assessments because it focuses on the processes used during the self-assessments and not the results obtained. 

The evaluation of the self-assessments was a large undertaking.  The evaluation team visited 22 DOE and contractor organizations and performed document reviews for 9 additional organizations, including the DOE Office of River Protection and Bechtel National, major players in the WTP drama. 

Problems abounded.  Self-assessment guidance was prepared but not distributed to all sites in a timely manner and there was no associated training.  Each self-assessment team had a “subject matter expert” but the qualifications for that role were not specified.  Data collection methods were not consistently applied and data analyses were of variable quality.  As a consequence, the self-assessment approaches used varied widely and the results obtained had variable reliability.

The self-assessment reports exhibited varying quality.  Some were satisfactory but “In many of the self-assessment reports, the overall conclusions did not accurately reflect the information in the data and analysis sections. In some cases, negative results were presented with a statement rationalizing or minimizing the issue, rather than indicating a need to find out more about the issue and resolve it.  In other cases, although data and/or analysis reflected potential problems, those problems were not mentioned in the conclusions or executive summaries, which senior management is most likely to read.” (p. 7)

The evaluation team summarized as follows: “The overall approach ultimately used to self-assess SCWE across the complex did not provide for consistent application of assessment methodologies and was not designed to ensure validity and credibility. . . . The wide variation in the quality of methodologies and analysis of results significantly reduces the confidence in the conclusions of many of the self-assessments.  Consequently, caution should be used in drawing firm conclusions about the state of SCWE or safety culture across the entire DOE complex based on a compilation of results from all the site self-assessments.” (p. iii)

“The Independent Oversight team concluded that DOE needs to take additional actions to ensure that future self-assessments provide a valid and accurate assessment of the status of the safety culture at DOE sites and organizations, . . .” (p. 8)  This is followed by a series of totally predictable recommendations for process improvements: “enhance guidance and communications,” increase management “involvement in, support for, and monitoring of site self-assessments,” and “DOE sites . . . should increase their capabilities to perform self-assessments . . .” (pp. 9-10)

Our Perspective

The steps taken to date do not inspire confidence in the DOE’s interest in determining if and what SCWE (much less more general SC) issues exist in the DOE complex.  For the facilities that were directly evaluated, we have some clues to the existence similar problems.  For the facilities that conducted self-assessments, so far we have—almost nothing.

There is one big step remaining: DOE also said it would “develop a consolidated report from the results of the self-assessments and HSS independent reviews.” (IP, p. 20)  We await that report with bated breath.

For our U.S. readers: This is your tax dollars at work. 


*  DOE Office of Enforcement and Oversight, “Independent Oversight Evaluation of Line Self-Assessments of Safety Conscious Work Environment” (Feb. 2014).

**  U.S. Dept. of Energy, “Implementation Plan for Defense Nuclear Facilities Safety Board Recommendation 2011-1, Safety Culture at the Waste Treatment and Immobilization Plant”  (Dec. 2011).

***  Defense Nuclear Facilities Safety Board, Recommendation 2011-1 to the Secretary of Energy "Safety Culture at the Waste Treatment and Immobilization Plant" (Jun 9, 2011).

****  DOE rationalized reducing the scope of investigation from SC to SCWE by saying “The safety culture issues identified at WTP are primarily SCWE issues. . .” (p. 17)  We posted a lecturette about SC being much more than SCWE here.

Deficient Safety Culture at Metro-North Railroad

A new Federal Railroad Administration (FRA) report* excoriates the safety performance of the Metro-North Commuter Railroad which serves New York, Connecticut and New Jersey.  The report highlights problems in the Metro-North safety culture (SC), calling it “poor”, “deficient” and “weak”.  Metro-North’s fundamental problem, which we have seen elsewhere, is putting production ahead of safety.  The report’s conclusion concisely describes the problem: “The findings of Operation Deep Dive demonstrate that Metro-North has emphasized on-time performance to the detriment of safe operations and adequate maintenance of its infrastructure. This led to a deficient safety culture that has manifested itself in increased risk and reduced safety on Metro-North.” (p. 4)

The proposed fixes are likewise familiar: “. . . senior leadership must prioritize safety above all else, and communicate and implement that priority throughout Metro-North. . . . submit to FRA a plan to improve the Safety Department’s mission and effectiveness. . . . [and] submit to FRA a plan to improve the training program. (p. 4)**

Our Perspective 

This report is typical.  It’s not bad, but it’s incomplete and a bit misguided.

The directive for senior management to establish safety as the highest priority and implement that priority is good but incomplete.  There is no discussion of how safety is or should be appropriately considered in decision-making throughout the agency, from its day-to-day operations to strategic considerations.  More importantly, Metro-North’s recognition, reward and compensation practices (keys to shaping behavior at all organizational levels) are not even mentioned.

The Safety Department discussion is also incomplete and may lead to incorrect inferences.  The report says “Currently, no single department or office, including the Safety Department, proactively advocates for safety, and there is no effort to look for, identify, or take ownership of safety issues across the operating departments. An effective Safety Department working in close communication and collaboration with both management and employees is critical to building and maintaining a good safety culture on any railroad.” (p. 13)  A competent Safety Department is certainly necessary to create a hub for safety-related problems but is not sufficient.  In a strong SC, the “effort to look for, identify, or take ownership of safety issues” is everyone’s responsibility.  In addition, the authors don’t appear to appreciate that SC is part of a loop—the deficiencies described in the report certainly influence SC, but SC provides the context for the decision-making that currently prioritizes on-time performance over safety.

Metro-North training is fragmented across many departments and the associated records system is problematic.  The proposed fix focuses on better organization of the training effort.  There is no mention of the need for training content to include any mention of safety or SC.

Not included in the report (but likely related to it) is that Metro-North’s president retired last January.  His replacement says Metro-North is implementing “aggressive actions to affirm that safety is the most important factor in railroad operations.”***

We have often griped about SC assessments where the recommended corrective actions are limited to more training, closer oversight and selective punishment.  How did the FRA do?   


*  Federal Railroad Administration, “Operation Deep Dive Metro-North Commuter Railroad Safety Assessment” (Mar. 2014).  Retrieved Mar. 14, 2014.  The FRA is an agency in the U.S. Department of Transportation.

**  The report also includes a laundry list of negative findings and required/recommended corrective actions in several specific areas.

***  M. Flegenheimer, “Report Finds Punctuality Trumps Safety at Metro-North,” New York Times (Mar. 14, 2014).  Retrieved Mar. 14, 2014)

Four Years of Safetymatters

Aztec Calendar

Over the four plus years we have been publishing this blog, regular readers will have noticed some recurring themes in our posts.  The purpose of this post is to summarize our perspective on these key themes.  We have attempted to build a body of work that is useful and insightful for you.

Systems View

We have consistently considered safety culture (SC) in the nuclear industry to be one component of a complicated socio-technical system.  A systems view provides a powerful mental model for analyzing and understanding organizational behavior. 

Our design and explicative efforts began with system dynamics as described by authors such as Peter Senge, focusing on characteristics such as feedback loops and time delays that can affect system behavior and lead to unexpected, non-linear changes in system performance.  Later, we expanded our discussion to incorporate the ways systems adapt and evolve over time in response to internal and external pressures.  Because they evolve, socio-technical organizations are learning organizations but continuous improvement is not guaranteed; in fact, evolution in response to pressure can lead to poorer performance.

The systems view, system dynamics and their application through computer simulation techniques are incorporated in the NuclearSafetySim management training tool.

Decision Making

A critical, defining activity of any organization is decision making.  Decision making determines what will (or will not) be done, by whom, and with what priority and resources.  Decision making is  directed and constrained by factors including laws, regulations, policies, goals, procedures and resource availability.  In addition, decision making is imbued with and reflective of the organization's values, mental models and aspirations, i.e., its culture, including safety culture.

Decision making is intimately related to an organization's financial compensation and incentive program.  We've commented on these programs in nuclear and non-nuclear organizations and identified the performance goals for which executives received the largest rewards; often, these were not safety goals.

Decision making is part of the behavior exhibited by senior managers.  We expect leaders to model desired behavior and are disappointed when they don't.  We have provided examples of good and bad decisions and leader behavior. 

Safety Culture Assessment

We have cited NRC Commissioner Apostolakis' observation that “we really care about what people do and maybe not why they do it . . .”  We sympathize with that view.  If organizations are making correct decisions and getting acceptable performance, the “why” is not immediately important.  However, in the longer run, trying to identify the why is essential, both to preserve organizational effectiveness and to provide a management (and mental) model that can be transported elsewhere in a fleet or industry.

What is not useful, and possibly even a disservice, is a feckless organizational SC “analysis” that focuses on a laundry list of attributes or limits remedial actions to retraining, closer oversight and selective punishment.  Such approaches ignore systemic factors and cannot provide long-term successful solutions.

We have always been skeptical of the value of SC surveys.  Over time, we saw that others shared our view.  Currently, broad-scope, in-depth interviews and focus groups are recognized as preferred ways to attempt to gauge an organization's SC and we generally support such approaches.

On a related topic, we were skeptical of the NRC's SC initiatives, which culminated in the SC Policy Statement.  As we have seen, this “policy” has led to back door de facto regulation of SC.

References and Examples

We've identified a library of references related to SC.  We review the work of leading organizational thinkers, social scientists and management writers, attempt to accurately summarize their work and add value by relating it to our views on SC.  We've reported on the contributions of Dekker, Dörner, Hollnagel, Kahneman, Perin, Perrow, Reason, Schein, Taleb, Vaughan, Weick and others.

We've also posted on the travails of organizations that dug themselves into holes that brought their SC into question.  Some of these were relatively small potatoes, e.g., Vermont Yankee and EdF, but others were actual disasters, e.g., Massey Energy and BP.  We've also covered DOE, especially the Hanford Waste Treatment and Immobilization Plant (aka the Vit plant).

Conclusion

We believe the nuclear industry is generally well-managed by well-intentioned personnel but can be affected by the natural organizational ailments of complacency, normalization of deviation, drift, hubris, incompetence and occasional criminality.  Our perspective has evolved as we have learned more about organizations in general and SC in particular.  Channeling John Maynard Keynes, we adapt our models when we become aware of new facts or better ways of looking at the data.  We hope you continue to follow Safetymatters.  

INPO: Traits of a Healthy Nuclear Safety Culture

The Institute of Nuclear Power Operations (INPO) has released a document* that aims at aligning their previous descriptions of safety culture (SC) with current NRC SC terminology.  The document describes the essential traits and attributes of a healthy** nuclear SC.  “[A] trait is defined as a pattern of thinking, feeling, and behaving such that safety is emphasized over competing priorities. . . . The attributes clarify the intent of the traits.” (p. 3)  While there is an effort to align with NRC, the document remains consistent with INPO policy, viz., SC is a primary leadership responsibility.  Leaders are expected to regularly reinforce SC, measure SC in their organization and communicate what constitutes a healthy SC.

There are ten traits organized into three categories.  Each trait has multiple attributes and each attribute has representative observable behaviors that are supposed to evidence the attribute's existence, scope and strength.  Many of the behaviors stress management's responsibilities.  The report has too much detail to summarize in this post so we'll concentrate on one of the key SC artifacts we have repeatedly emphasized on this blog: decision making.

Decision making (DM) is one of the ten traits.  DM has three attributes: a consistent process, conservative bias and single-point accountability.  Risk insights are incorporated as appropriate.  Observable behaviors include: the organization establishes a well-defined DM process; individuals demonstrate an understanding of the DM process; leaders seek inputs from different work groups or organizations; when previous decisions are called into question by new facts, leaders reevaluate these decisions; conservative assumptions are used when determining whether emergent or unscheduled work can be conducted safely; leaders take a conservative approach to DM, particularly when information is incomplete or conditions are unusual; managers take timely action to address degraded conditions; executives and senior managers reinforce the expectation that the reactor will be shut down when procedurally required, when the margin for safe operation has degraded unacceptably, or when the condition of the reactor is uncertain; individuals do not rationalize assumptions for the sake of completing a task; and the organization ensures that important nuclear safety decisions are made by the correct person at the lowest appropriate level. (pp. 19-20)  That's quite a mouthful but it's not all of the behaviors and some of the included ones have been shortened to fit.

In addition to the above, communicating, explaining, challenging and justifying individual decisions are mentioned throughout the document.  Finally, “Leaders demonstrate a commitment to safety in their decisions and behaviors.” (p. 15)

Our perspective

On the positive side, the INPO treatment of DM is much more comprehensive than what we've seen in the NRC Common Language Path Forward materials released to date.

But the DM example illustrates a major problem with this type of document: a lengthy laundry list of observable behaviors that can morph into de facto requirements.  Now INPO says “. . . this document is not intended to be used as a checklist. It is encouraged that this document be considered for inclusion and use in self-assessments, root cause analyses, and training content, as appropriate.” (p. 3)  But while the observable behaviors may be intended as representative or illustrative, in practice they are likely to become first expectations then requirements.  An overall tone of absolutism reinforces this possibility.

The same tone is evident in the discussion of DM's larger context.  For example, INPO asserts that SC is a board and corporate responsibility but explicit or implicit priorities from above can create constraints on plant management's DM flexibility.  INPO also says “Executives and senior managers ensure sufficient corporate resources are allocated to the nuclear organization for short- and long-term safe and reliable operation” (p. 15) but the top and bottom of the organization may not agree on what level of resources is “sufficient.”

Another problem is the lack of priorities or relative importance.  Are all the traits equally important?  How about the attributes?  And the observable behaviors?  Is it up to, say, a team of QA assessors to determine what they need to include or do they only look at what the boss says or do they try to evaluate everything even remotely related to the scope of their inquiry?

But our biggest difficulty is with this statement: “These traits and attributes, when embraced, will be reflected in the values, assumptions, behaviors, beliefs, and norms of an organization and its members.” (p. 3)  This is naïve absolutism at its worst.  While some members of an organization may incorporate new values, others may comply with the rules and exhibit the desired behavior based on other factors, e.g., fear, peer pressure, desire for recognition or power, or money.  And ultimately, who cares why they do it?  As Commissioner Apostolakis said during an NRC meeting when the proposed SC policy was being discussed: “[W]e really care about what people do and maybe not why they do it. . . .”  (See our Feb. 12, 2011 post.)

We could not say it better ourselves.


*  Institute of Nuclear Power Operations (INPO), “Traits of a Healthy Nuclear Safety Culture”  INPO 12-012, Rev. 1 (April 2013).  The report has two addenda.  One describes nuclear safety behaviors and actions that contribute to a healthy nuclear SC by organizational level and the other provides cross-references to other INPO documents, the NRC ROP cross-cutting area components and the IAEA SC characteristics.  Thanks to Madalina Tronea for making these documents available.

** INPO refers to SC “health” while the NRC refers to SC “strength.”

Safety Culture at the Pantex Plant

Pantex Plant

On January 25, 2013 we posted about DOE's report to the Defense Nuclear Facilities Safety Board (DNFSB) on the results of safety culture (SC) assessments at several DOE facilities, including the Pantex Plant.  Pantex was assessed because two Pantex employees had reported retaliation for raising a safety concern but the plant also had a history of potentially SC-significant issues.*

The Pantex SC assessment was performed in November 2012.  The report included several significant findings:

“Efforts to communicate and implement the principles of a High Reliability Organization (HRO) have been ongoing for several years. . . . [However,] The realization of the HRO principles has not yet been internalized by the Plant, . . .”

“The belief that the organization places a priority on safety is undermined by employee observations of poor facility conditions, lack of focus on meeting personal needs (work quality of life), and a sense of cronyism. . . . [This] has created the perception among many employees that the financial bottom line is the only focus that matters.

“There is a strong perception that retaliation exists for ‘rocking the boat.’ . . . The perception has created an environment where the raising of questions or identification of problems is not the consistently accepted way of doing business.

“The Pantex Plant has not been successful in understanding the organizational and programmatic behaviors that are necessary for a healthy safety culture. . . . organizational barriers have been created that will prevent successful implementation of the initiatives needed to enhance safe and reliable performance. . . . The barriers are also evident in the lack of respect, difficulty in effective communication, the non-alignment between the perceptions around the unions and management relationships and the notion of ‘need to know’ being extended to almost everything.”**

Sounds serious.  So what's happened since the report was published?  Well, DNFSB held a public meeting on March 14, 2013 to discuss SC problems at Pantex and management's approach to addressing them.  As one might expect, the DOE opening statement declared the SC problems were intolerable and DOE had taken immediate action—by firing off a letter to the contractor. 

“NNSA issued a letter to B&W Pantex directing immediate focus at all management and working levels to a safety conscious work environment (SCWE) for all on-going activities and operations. . . . Other immediate actions included the development of a single stop/pause work process which was provided to all employees and discussed at daily work planning meetings. The Employee Concerns Program and Equal Opportunity Office reporting chain was immediately elevated to a direct report to the General Manager’s Office, providing the highest level accesses for any employee concerns in these areas. Additionally, the differing professional opinion process was reinstated providing a formal mechanism for recognition and resolution of differing views on technical matters.

“Further, this letter directed B&W Pantex to prepare a comprehensive long-term corrective action plan mentioned previously, taking into account the Institute of Nuclear Power Operations (INPO) paper on the principles of a strong nuclear safety culture.”***

DOE listed actions being implemented, including an SC focus team, a seminar on HRO attributes, SCWE training, the issuance of a plant-specific SC policy and an alignment of performance incentives with SC objectives. 

Babcock & Wilcox (B&W)

B&W, the entity that actually has to do the work, did not file any written testimony for the public meeting so we have to go to the meeting video for their comments.****

B&W appears to be on board with the need for change.  The B&W plant general manager and his safety manager were appropriately deferential to the DNFSB members and seemingly well-informed about the plant's remedial actions and SC improvement initiatives.  The GM said he “fully accepts” the assessment findings (which were consistent with the plant's own SC survey conducted just prior to the DOE HSS assessment) and acknowledged that specific problems, e.g., communication issues with respect to safety vs production, existed.  Improving SC/SCWE is the GM's “top priority.”  B&W reiterated its commitment to building an HRO at Pantex, an initiative that overlaps with actions to strengthen SC and SCWE.  Perhaps the most significant change the GM described was that 30% of managers' performance evaluations would be based on their modeling of appropriate SC/SCWE traits.

Our perspective    

Problem solved?  Not yet and not for awhile.  Pantex had some serious vertical communication and organizational structure issues.  Their attempt to build an HRO has been ongoing for years.  Their SCWE has had some cold spots. 

In addition, the actions Pantex has initiated may be necessary but there is no guarantee they will be sufficient to achieve the plant's SC/SCWE/HRO goals.  For example, there is no real discussion of how decision making processes will be affected other than resolving Nuclear Explosive Safety issues and the usual commitment to conservative decision making.  There is no mention of a corrective action program (or some functional equivalent); an integrated process for identifying, evaluating and fixing problems is essential for ensuring safety, priorities and resource allocation are treated consistently throughout the plant.

We'll watch for progress (or lack thereof) and keep you posted.


*  Pantex is the sole US site that assembles and disassembles nuclear weapons.  Within DOE, the National Nuclear Security Administration (NNSA) has line management responsibility for Pantex.  Babcock & Wilcox is responsible for managing and operating the plant under contract with DOE.

The SC issues identified in the November 2012 assessment did not pop out of nowhere.  The DNFSB identified SC-related concerns at the plant during the previous year.  See statement of D.G. Ogg, Group Lead for Nuclear Weapons Programs, DNFSB, at the Pantex public meeting (March14, 2013).
   
**  DOE Office of Enforcement and Oversight, “Independent Oversight Assessment of Nuclear Safety Culture at the Pantex Plant” (Nov. 2012) p. 3.  The report is attached to the letter from G.S. Podansky (DOE) to P.S. Winokur (DNFSB) transmitting five independent safety culture assessments (Dec. 12, 2012).

***  Written testimony of Neile Miller, Acting Administrator, NNSA before the Defense Nuclear Facilities Safety Board Pantex Plant Public Meeting (March 14, 2013), p. 4. 

****  The meeting video is available on the DNFSB website.  The NNSA panel on Pantex SC runs from about 1:25 to 2:30, the B&W panel runs from about 2:35 to 3:20.