Vit Plant Safety Culture Update

Hanford Waste Treatment Plant

DOE released a June 2014 follow-up assessment* on safety culture (SC) at the Hanford Waste Treatment Plant (WTP or the Vit Plant).  This post provides our perspective on the assessment.  We will not review every facet of the report but will focus on aspects that we think are important to understanding the current state of SC at the WTP project.

Overview

Back in 2011, the Defense Nuclear Facilities Safety Board (DNFSB) called the WTP safety culture (SC) “flawed.”  Following the DNFSB report, DOE conducted an assessment of the WTP SC and concluded “that a significant number of staff within ORP [DOE’s Office of River Protection] and BNI [Bechtel National Inc.] expressed reluctance to raise safety or quality concerns for various reasons.” (p. 1) 

Like DOE’s 2011 report, the current one is based on multiple data sources: structured interviews, focus groups, observations and a culture survey.  The report identifies many SC-related interventions that have been initiated, and lists positive and negative findings from the data collected.  Detailed assessment results are reported separately for ORP and BNI.**
 
WTP’s Safety Conscious Work Environment (SCWE)—Voicing Concerns, Challenging Decisions, Fear of Retaliation

The 2011 DNFSB critique focused on the treatment of project personnel who raised technical issues.  Some of these personnel complained about retaliation for bringing up such issues.  These issues can be raised in the Employee Concerns Program (ECP), the Differing Professional Opinions (DPO) process and challenging management decisions.  In what is arguably the report’s most significant finding, perceptions of conditions in these areas are worse than they were in 2011 for ORP and unchanged for BNI. (pp. 4-5)

Supporting Details

Although ORP senior management pointed to recent reductions in ECP concerns, “[s]ome interviewees indicated that they perceived a chilled environment at ORP and they did not believe that ECP concerns and DPO issues were always addressed or resolved in a timely manner.  Additionally, some interviewees described being told by supervision not to write a DPO because it would be a career limiting decision.” (p. B-21)  Interviewees from both ORP and BNI said they lacked trust in the ECP. (pp. B-22/23)

Most ORP employees believe that constructive criticism is not encouraged. (p. 4)  Within ORP, only 30% of all ORP survey respondents (and 65% of managers) feel that they can openly challenge decisions made by management. (p. B-21)  In BNI/URS, the numbers are 45% of all respondents and 75% of managers. (p. B-22)

“The statement that management does not tolerate retaliation of any kind for raising concerns was agreed to by approximately 80% of the ORP, . . . 72% of the BNI and 80% of the URS survey respondents.” (pp. B-21-22)  In addition, “Anonymous PIERs [Project Issues Evaluation Reports] are used a lot because of fear of retaliation.” (p. B-23)  

All in all, hardly a ringing endorsement of the WTP SCWE.

Decision Making, Corrective Action and Compensation

Safetymatters readers know of our long-standing interest in how SC is reflected in these key artifacts.

Decision Making

“Interviewees provided some examples of where decision making was not perceived to reflect the highest commitment to safety”  Examples included downgrading or elimination of assessment findings, the margin of safety in corrective action plans and the acceptable level of risk for the project.  (p. B-9)  Looks like there's some room for improvement in this area.

Corrective Action

Within BNI, there are positive comments about the corrective action process but the assessment team “observed a lack of accountability for a backlog of corrective actions at a PIRB [Performance Improvement Review Board] meeting.  There is a perceived lack of accountability for corrective actions in timeliness, ownership, and quality, ...” (p. B-16)

“[O]nly about 18% of all ORP interviewee respondents believed that employees are encouraged to notify management of problems they observe and that there is a system that evaluates the problem and makes a determination regarding future action.” (p. 13)

Plenty of room for improvement here.

Compensation

In the area of compensation, it appears some rewards for Bechtel are based on SC behaviors. (p. B-8)  We consider that a very positive development.

Concerns Over ORP Working More Collaboratively with Bechtel

On the surface this looks like a positive change: two entities working together to achieve a common goal.  However, this has led to at least two concerns.  First, as described in the report, some ORP personnel believe ORP is abdicating or compromising its responsibility to perform oversight of Bechtel, in other words, ORP is more of a teammate and less of an umpire. (pp. B-4, -9, -14)  Second, and this reflects our perspective, changing the relationship between the entities can result in revised system dynamics, with old performance-oversight feedback loops replaced by new ones.  The rules of engagement have changed and while safety may still be the number one priority, the cultural milieu in which safety is achieved has also changed.

Another complication is caused by the role of Bechtel Corporate.  The report says corporate’s values and goals may not be well-aligned with BNI’s need to prioritize SC attributes and behavior.  This can lead to a lack of transparency in BNI decisions. (pp. 5, 7)  That may be a bit of weasel wording in the report; in more direct terms, corporate’s number one priority is for the money train to keep running.

Bottom Line on SC at WTP 

A strong SC is, in some ways, about respect for the individual.  The concerns that WTP personnel  express about using the ECP or DPO process, or challenging management decisions suggest that the WTP project has a ways to go to inculcate an adequate level of such respect.  More importantly, it doesn’t appear they have made any significant process toward that goal in the last few years.

Morale is an aspect of the overall culture and at the WTP, morale is arguably low because of lack of progress and missed schedules. (pp. 6, B-10)  There is undoubtedly plenty of work to do but on a day to day basis, we wouldn’t be surprised if some people feel their work is not meaningful.


*  DOE Office of Environment, Safety and Health Assessments (now the Office of Independent Enterprise Assessments), “Independent Oversight Follow-up Assessment of Safety Culture at the Waste Treatment and Immobilization Plant” (June, 2014).

**  ORP has line management responsibility for the WTP, BNI is the primary contractor and URS Corporation (URS) is a major subcontractor.

Catching Up on DOE’s SCWE Extent of Condition Review

Hanford Waste Treatment Plant

On May 29, 2014 DOE submitted its partial response to the Defense Nuclear Facilities Safety Board (DNFSB) Recommendation 2011-1 in a report* on DOE’s Safety Conscious Work Environment (SCWE) extent of condition review and recommended actions for ongoing safety culture (SC) improvement at DOE facilities.

(Quick history: The June 9, 2011 DNFSB report on DOE’s Hanford Waste Treatment Plant (WTP or the Vit Plant) said the WTP SC was “flawed.”  The report’s recommendations included that DOE should conduct an extent of condition review to determine whether WTP SC weaknesses existed at other DOE facilities.  DOE agreed to perform the review but focused on SCWE because, in DOE’s view, the issues at WTP were primarily SCWE related.)

This post summarizes the report’s findings then parses the details and provides our perspective.

 Report Findings and Recommendations

The report was based on data from eleven independent SC assessments and thirty-one SCWE self-assessments conducted by individual organizations.  The DOE review team processed the data through their analyzer and homogenizer to identify four primary SC attributes** to focus on for continuous improvement in DOE:

• Demonstrated safety leadership
• Open communication and fostering an environment free from retribution
• Teamwork and mutual respect
• Credibility, trust and reporting errors and problems

Further processing through the bureaucratizer yielded three recommended actions to improve the SC attributes:

• Form a DOE SC Improvement Panel to ensure leadership and focus on DOE's SC initiatives
• Incorporate SC and SCWE concepts and practices into DOE training
• Evaluate contract language to incorporate clear references to SC  (pp. 3-4)

Our Perspective

We reviewed the DOE independent assessments on January 25, 2013 and the self-assessments on March 31, 2014.  From the former we concluded that issues similar to those found at the WTP existed at other DOE facilities, but to a lesser degree than WTP.  The self-assessments were of such varying quality and credibility that we basically couldn’t infer anything.***

The Analyzer

The DOE team reviewed all the assessments to identify specific issues (problems).  The team binned issues under the SC attributes in DOE's Integrated Safety Management System Guide and then counted the number of issues under each attribute; a higher count meant a more serious problem.  They performed a similar exercise to identify positive organizational trends (strengths) mentioned in the assessments. 

We could be picky and ask if all the issues (or strengths) were of essentially the same importance or magnitude but the team had a lot of data to review so we’ll let that slide.  Concurrently identifying strengths was a good idea; it harkens back to Peter Drucker who advised managers to build on strengths.****

The Homogenizer

Here’s where we begin to have problems.  The team focused on identifying SC attributes and developing recommendations that applied to or affected the entire Department, essentially boiling their results down to a one-size-fits-all approach.  However, their own data belies that approach.  For example, the Leadership attribute “Open communication and fostering an environment free from retribution” was identified as both an issue AND a strength. 

In plain English, some organizations don’t exhibit the desired communication attribute and others do.  One proper fix is to identify who is doing it right, define what exactly they’re doing, and develop a method for transferring that approach to the problem organizations.  The report even says this attribute “is an area in which management can learn lessons from those sites where it was deemed a strength so that best practices can become commonplace” but this statement is buried at the end of the report. (p. 22)

The DOE entities exhibit a wide variety of scale, scope, mission and organizational and technical complexity.  The Department’s goal should be to recognize that reality, develop it as an overall strength and then build on it to create site- or organization-specific interventions.

The Bureaucratizer

The proposed fixes would basically create a top-level coordinating and oversight group, enhance SC and SCWE training and modify contracts.  The recommendations reflect a concerted search for the lowest common denominator and a minimum amount of real change.

If the SC Improvement Panel is established, it should focus on setting or refining SC policy and ensuring those policies are implemented by line management, especially field management.  They should also be involved in evaluating major SC issues.  If things aren’t going well, this group should be the first to ask the hard “Why?” questions.  But most of the panel’s proposed tasks, viz., maintaining SC visibility, providing a forum for evaluating SC status and overseeing training improvements, are low-value make work.

“[S]afety culture training for all personnel, up and down the management chain, will be updated and/ or developed to ensure that roles and responsibilities are understood and personnel have the capabilities needed to play their part in continuously improving DOE's safety culture; . . .” (p. 24)  This is a standard fix for almost any perceived organizational problem.  It doesn’t require managers to do anything different.

Modifying contracts to incorporate clear references to SC is only a beginning.  What are the carrots and the sticks to incent the contractors to actually develop, measure and maintain an effective SCWE and strong SC?

Bottom Line

This report comports completely with an organization that resembles a fifty foot sponge.  You can kick it as hard as you like, your foot goes in deep and you think you’ve had an effect, but when you withdraw your foot, the organization fills in the hole like your kick never happened.  I thought I heard a loud pop on May 29.  I now realize it was likely DNFSB Chairman Winokur’s head exploding when he read this report.

But Wait, There’s More

Remember the question the DNFSB initially asked in 2011: Do WTP SC issues exist elsewhere in DOE?  Well, the answer is: “Review of assessment results from both [independent and self assessments] indicated there is a SCWE extent of condition that requires additional and ongoing actions to improve performance.” (p. 28, the penultimate page of the report)


*  J. Hutton (DOE) to P.S. Winokur (DNFSB) May 29, 2014 letter transmitting DOE Consolidated Report for DNFSB Recommendation 2011-1, Actions 2-8 and 2-9 (May 2014).

** DOE considers SC in three focus areas: Leadership, Employee Engagement and Organizational Learning.  Each focus area has a set of associated attributes that describe what a positive SC should look like. (pp. 4-5)

***  The report puts the self-assessments in the best possible light by describing them as learning experiences for the organizations involved. (p.9)

****  Drucker was referring to individuals but, in this case, we’ll stretch the blanket to cover organizations.  For individuals, weaknesses should not be ignored but the energy required to fix them, if it’s even possible, is often too great.  However, one should keep an eye on such weaknesses and not allow them to lead to performance failure.

Reaction to the Valukas Report on GM Ignition Switch Problems

CEO Mary Barra and Anton Valukas

General Motors released the report* by its outside attorney, Anton Valukas, investigating the hows and whys of the failure to recall Chevy Cobalts due to faulty ignition switches.  We blogged on these issues and the choice of Mr. Valukas on May 19, 2014 and May 22, 2014 indicating our concern that his law firm had prior and ongoing ties to GM.  The report is big, 314 pages, and for some reason is marked as “Confidential, Attorney-Client Privileged”.  This is curious for a report always intended to be public and tends to highlight that Valukas and GM are in a proprietary relationship - perhaps not the level of independence one might expect for this type of assessment.

Our take, in brief, is that the Valukas report documents the "hows" but not the "whys" of what happened.  In fact it appears to be a classic legal analysis of facts based on numerous interviews of “witnesses” and reviews of documentation.  It is heavy with citations and detail but it lacks any significant analysis of the events or insight as to why people did or did not do things.  “Culture” is the designated common mode failure.  But there is no exploration of extent of condition or even consideration of why GM’s safety processes failed in the case of the Cobalt but have been effective in many other situations.  Its recommendations for corrective actions by GM are bland, programmatic and process intensive, and lack any demonstrable linkage to being effective in addressing the underlying issues.  On its part GM has accepted the findings, fired 15 low level engineers and promised a new culture.

The response to the report has reflected the inherent limitations and weaknesses of the assessment.  There have been many articles written about the report that provide useful perspectives.  An example is a column in the Wall Street Journal by Holman Jenkins titled “GM’s Cobalt Report Explains Nothing."**  In a nutshell that sums it up pretty well.  It is well worth reading in its entirety.

Congressional response has also been quite skeptical.  On June 18, 2014 the House Committee on Energy and Commerce, Subcommittee on Oversight and Investigations, held a hearing with GM CEO Barra and Valukas testifying.  A C-SPAN video of the proceeding is available and is of some interest.***  Questioning by subcommittee members focused on the systemic nature of the problems at GM, how GM hoped to change an entrenched culture, and the credibility of the findings that malfeasance did not extend higher into the organization.

The Center for Auto Safety, perhaps predictably, was not impressed with the report, stating: “The Valukas Report is clearly flawed in accepting GM’s explanation that its engineers and senior managers did not know stalling was safety related.”****

Why doesn’t the Valukas report explain more?  There are several possibilities.  Mr. Valukas is an attorney.  Nowhere in the report is there a delineation of the team assembled by Mr. Valukas or their credentials. It is not clear if the team included expertise on complex organizations, safety management or culture.  We suspect not.  The Center for Auto Safety asserts that the report is a shield for GM against potential criminal liability.  Impossible for us to say.  Congressional skepticism seemed to reflect a suspicion that the limited scope of the investigation was designed to protect senior GM executives.  Again hard to know but the truncated focus of the report is a significant flaw.

What is clear from these reactions to the report is that, at a minimum, it is ineffective in establishing that a full and expert analysis of GM’s management performance has been achieved.  Assigning fault to the GM culture is at once too vague and ultimately too convenient in avoiding more specific accountability.  It also suggests that internally GM has not come to grips with the fundamental problems in its management system and decision making.  If so, it is hard to believe that the corrective actions being taken will be effective in changing that system or assuring better safety performance going forward.


*  A.R. Valukas, "Report to Board of Directors of General Motors Company Regarding Ignition Switch Recalls" (May 29, 2014).

**  H.W. Jenkins, Jr., "GM's Cobalt Report Explains Nothing," Wall Street Journal (June 6, 2014).

***  C-SPAN, "GM Recall Testimony" (June 18, 2014).  Retrieved June 26, 2014.

****  C.Ditlow (Center for Auto Safety), letter to A.R. Valukas (June 17, 2014), p. 3.  Retrieved June 26, 2014.

Regulatory Oversight of Safety Culture in Belgium

The latest International Nuclear Safety Journal has an article* by Benoit Bernard describing a new safety culture (SC) regulatory oversight process now in use in Belgium.  It is based on observations of SC during interactions with a licensee.  This post describes the process and the rationale for it, followed by our perspective.

Bernard starts with a brief history of SC in the nuclear industry then describes two types of regulation currently used, compliance-based and performance-based, and highlights the shortcomings of each.  “Compliance-based” regulation is focused on a licensee’s control of isolated technical components.  This traditional approach can lead to an “adversarial legalism” between the regulator and the licensee, discourage open communication and fail to promote continuous improvement.  In contrast, “performance-based” regulation is based on specific outcomes the licensee is expected to achieve.  The regulator focuses on monitoring outcomes.  This is a reactive approach that can tend to concentrate on well-known risks or familiar issues, and ignore emergent new issues.  Both approaches are inadequate to deal with human factors issues.

The New Process

The author notes “Safety culture cannot be directly regulated but it can be observed . . . [The new Belgian approach] is based on field observations provided by inspectors or safety analysts during any contact with a licensee (inspections, meetings, phone calls…).” (p. 3)  It is expected to be more proactive and systemic than the earlier regulatory approaches.

The process has both short-term and longer-term applications.  In the short term, the purpose is to identify findings that require more-or-less immediate licensee attention or action.  In the longer term, SC observations are input to the overall oversight process. (p. 4)

Observations focus on both facts (what happened) and context (the circumstances surrounding an event).  The approach leads to “Why?” questions rather than degree of compliance with defined SC attributes.  For example, if someone doesn’t follow a rule, is it because of bad behavior or a bad rule?  Was there inadequate training or task-specific knowledge, an inadequate procedure, poor documentation or lack of management commitment to SC?  “The important point is to . . . shed light on the underlying reasons as to why the rules were ignored. . . . [L]inking an observation to an attribute must not be considered as an end but as a starting point to further questions.” (p. 7)

Bernard goes on to describe three aspects of SC that an overall assessment must address: Integration, Differentiation and Fragmentation.  “Integration” refers to the “level of consensus concerning a set of values unifying people and reflected in practices and management systems.” (p. 8)  Prior to the annual SC review with a licensee, SC observations are assessed through four key safety dimensions: Management, Organization, Workplace Practices and Behavior.  You probably can’t read the figure below but each dimension has two component factors, e.g., Management consists of “Management system” and “Leadership,” and each factor appears under two different dimensions, e.g., “Management system” appears under Management and Organization.  Each factor also has several attributes.  This is where the rubber meets the road so think about the training, teamwork, supervision and overall effort required to get regulatory observers (who are more likely to be technical experts than social scientists) to reliably associate specific observations with the correct dimension(s), factor(s) and attribute(s) and then integrate their findings into an overall SC assessment.



“Differentiation” refers to “the ability of [sub-]groups to share a common definition of problems” and “Fragmentation” refers to the “contrast of perceptions and contradictions [across an organization] about what is safe or dangerous.” (p. 9)

Comparison with Romanian Approach

If this topic sounds familiar, on April 21, 2014 we posted on an SC oversight process developed by the Romanian nuclear regulatory agency (CNCAN).  The CNCAN approach looks at artifacts (documents, interviews and observations) to develop an overall, longer-term perspective on SC.

CNCAN recognizes there are limitations to using their process including findings that reflect a reviewer’s subjective opinion and over-reliance on one specific finding.  The Belgian paper recognizes that training technically-oriented reviewers to become competent observers is a challenge.  But Bernard also appears to promote the possibility of “one specific finding” being an early warning, a leading indicator of problems.

Bernard explicitly states this is not a one-size-fits all approach.  The search for event context implies a type of customization of the process for each licensee.  The author says the result is “a regulation style responding to the reference framework of a particular licensee.” (p. 9)

Belgium has seven operating units at two sites, both sites owned and managed by Electrabel, a Belgium-based energy company.  A customized approach may work in Belgium.  But as we noted in our review of the CNCAN approach, “the U.S. currently has 32 operators reporting to 81 owners. Developing SC assessment techniques that are comprehensive, consistent and perceived as fair by such a large group is not a simple task.”

Our Perspective

This is a good paper for its comprehensive discussion of nuclear SC in general and its description of two existing regulatory world views. 

But we have some concerns with the SC observation process.  As noted above, training observers is a major challenge and we think it would be very difficult to adopt such a process in the relatively fragmented U.S. nuclear industry.

In addition, observations are a very soft artifact (compared to documents or even structured interviews) and thus open to to misunderstandings, observational errors and false positives.  It’s easy to imagine a licensee being sent off on a wild goose chase after a regulator misreads one (or more) interactions with licensee personnel.

Furthermore, as instant observations become used as leading indicators, the process could become more like a backseat driver commenting on every turn of the steering wheel.  Licensees might oversteer in their attempt to get back into this new type of compliance.  The risk is the observational process begins to intrude on day-to-day management.  And, at some point, ownership of plant SC could subtly shift from the licensee to the regulator.

Finally, although we constantly chide the industry for concentrating on the “what” and ignoring the “why” associated with incidents or infractions, it’s also clear that the pendulum could swing too far in the other direction.  In plain language, not every minor issue merits an in-depth “why” investigation; that can be a route to over-use of resources and organizational paralysis.

We’re not condemning this as a bad idea.  But a regulatory user (and licensees) should be alert to the possibility of unintended consequences.


**  B. Bernard, “Safety Culture as a Way of Responsive Regulation: Proposal for a Nuclear Safety Culture Oversight Model,” International Nuclear Safety Journal vol. 3 no. 2 (2014) pp. 1-11.  Thanks to Madalina Tronea for promoting this journal.  Dr. Tronea is the founder/moderator of the LinkedIn Nuclear Safety group.

NRC Non-Concurrence Process Assessment: Tempest in a Teapot?

On June 4, 2014 the NRC announced a revised agency-wide non-concurrence process (NCP) on their blog.*  A key objective of the NCP is “to ensure that a non-concurrence is heard, understood, and considered by employees included in the concurrence process so that the non-concurrence informs and supports the decisionmaking process.”**

The NRC performed an assessment*** of the prior NCP using multiple data sources, including the NRC’s 2012 Safety Culture and Climate Survey (SCCS) and an April 2013 survey targeted at employees who had been involved with the NCP as submitters or participants (employees who have responded to non-concurrences).

The assessment identified both strengths and weaknesses with the then-existing NCP.  In general, participants were aware of the NCP and were willing to use it.  However, “some users of the process felt they faced negative consequences, or that their views were not reflected in final decisions.” (blog post)  The assessment also included a bevy of planned actions to address NCP weaknesses.

For us, the interesting question is what does the assessment say or what can be inferred, if anything, with respect to the NRC’s safety culture (SC).  This post focuses on SC-related topics mentioned in the assessment that help us answer that question.

Leadership Commitment

Leadership commitment is an area of concern and planned actions. (p. 4)  “Data from several sources indicates that many of the responding employees are still uncertain about management’s support of the NCP. . . . management was just going through the motions. . . .[some employees] thought the process was biased . . .supervisors using the process indicated that they were concerned management would view it as a negative reflection on them [the supervisors].” (p. 11)  In the targeted survey, “more than half of submitters are concerned about management’s support of the NCP.” (p. 7)

Planned actions include “support managers in emphasizing their personal commitment to the welcoming of sharing differing views and the value of using the NCP in support of sound regulatory decisionmaking. . . . Management should demonstrate this [NCP is a positive] clearly and frequently through their actions and communications. . . . Staff will continue to support a variety of outreach activities and communication tools, such as EDO Updates, monthly senior management meetings, all-supervisor meetings, senior leadership meetings, Yellow Announcements, all-hands meetings, brown bag lunches, seminars, and articles in the NRC Reporter and office-level newsletters.” (p. 18)  Whew!

Potential Negative Consequences of Submitting a NCP

From the SCCS report the assessment highlights that “Forty-nine percent of employees believe that the NCP is effective (37 percent don’t have an opinion on the effectiveness of the NCP and 14 percent believe that the NCP is not effective).” (p. 6)  That 14 percent looks low but because there are only about a dozen NCP filings per year, it might actually reflect that a lot of people who use the process end up disappointed.  That view is supported by the targeted survey where “the majority of submitters believed that the rationale for the outcome was not clearly documented and that they experienced negative consequences as a result of submitting a non-concurrence.” (p. 7)

We reviewed the SCCS on April 6, 2013.  We noted that “The consultants' cover letter identified this [DPO/NCP] as an area for NRC management attention, saying the agency was “Losing significant ground on negative reactions when raising views different from senior management, supervisor, and peers.””

Planned actions include “proactively fostering an environment that encourages and supports differing views . . . evaluating the merits of infusing NCP key messages into existing training, including reinforcing that supervisors and managers will be held accountable for their actions. . . . consider training for all supervisors to address concerns of retaliation and chilling effect for engaging in the NCP. . . . hosting panel discussions including previous NCP submitters and participants . . . promote NCP success stories . . . evaluating the merits of establishing an anti-retaliation policy and procedures to address concerns of retaliation and chilling effect for engaging in the NCP. (p. 20)  Note these are all staff activities, management doesn’t have to do anything except go along with the program.

Goal Conflict

Goal conflict is another problem area.  The assessment notes “many responding employees commented they felt pressure to meet schedules at the expense of quality.” (p. 17)  That issue was also highlighted in the 2012 SCCS and could well be the source for the comment in the assessment.

Our Perspective

An effective NCP is important.  We believe NCP or some functionally equivalent practice should be more widely utilized in the world of formal organizations.

But it is easy to read too much into the NCP assessment.  The primary data input was the 2012 SCCS and that is relatively old news.  Another key input was the targeted survey.  However, the number of survey respondents was small because only a handful of people use the NCP.****  Based on the negative responses of the submitters, it appears that NRC needs to do a better job of administering the NCP, especially in the areas of (1) convincing submitters that their concerns were actually considered (even if ultimately rejected) and (2) ensuring there are no negative consequences associated with using the NCP.  These are real process implementation challenges but the NCP-related issues do not reflect some major, new problem in the agency’s SC.

On the other hand, perceptions of negative responses to rocking the boat in general or senior management’s lack of commitment to inclusive programs and “safety first” are SC signals to which attention must be paid.  If Staff trains their 10 gauge shotgun of interventions on these possibly systemic issues then some actual good could come out of this.


*  NRC blog “Improving NRC’s Internal Processes” (June 4, 2014).  Retrieved June 12, 2014.

**  NRC Non-Concurrence Process, Management Directive 10.158 (Mar. 14, 2014).  ADAMS ML13176A371.

“Non-concurrence” means an employee has a problem with a document the employee had a role in creating or reviewing.  For example, the employee might hold a different view on a technical matter or disagree with a proposed decision.

The NCP appears to be more formal and documented than the NRC Open Door policy and less restrictive than the Differing Professional Opinions (DPO) program which is reserved for concerns on established NRC positions.

***  NRC Office of Enforcement, “2014 Non-Concurrence Process Assessment”  (June 4, 2014).  ADAMS ML14056A294.

****  The survey was issued to 39 submitters (24 responded [62%]) and 62 participants (17 responded [27%]).

DNFSB Observations on Safety Culture

DNFSB Headquarters

The Defense Nuclear Facilities Safety Board (DNFSB) has been busy in the safety culture (SC) space.  First, their Chairman’s May 7, 2014 presentation on preventing major accidents provides a window into how the DNFSB views safety management and SC in the DOE complex.  Second, the DNFSB’s meeting on May 28, 2014 heard presentations on SC concepts from industry and government experts.  This post reviews and provides our perspective on both events. 

Chairman’s Presentation

This presentation was made at a DOE workshop.*  Chairman Winokur opened with some examples of production losses that followed incidents at DOE facilities and concluded the cost of safety is small compared to the cost of an accident.  He went on to discuss organizational factors that can set the stage for accidents or promote improved safety performance.  Some of these factors are tied to SC and will be familiar to Safetymatters readers.  They include the following:

Leadership

The presentation quotes Schein: “The only thing of real importance that leaders do is to create and manage culture.” (p. 13)  This quote is used by many in the nuclear industry to support a direct and complete connection between leadership and an organization’s culture.   While effective leadership is certainly necessary, we have long argued for a more nuanced view, viz., that leaders influence but do not unilaterally define culture.  In fact, on the same page in Organizational Culture, Schein says “Culture is the result of a complex group learning process that is only partially influenced by leader behavior.” **

Budget and production pressures and
Rewards that favor mission over safety 

As Winokur pointed out, it is unfortunately true that poor safety performance (accidents and incidents) can attract resources while good safety performance can lead to resources being redirected.  Good safety performance becomes taken for granted and is largely invisible.  “Always focus on balancing mission and safety.  There will always be trade-offs, but safety should not get penalized for success.” (p. 19) 

On our part, we feel like we’ve been talking about goal conflicts forever.  The first step in addressing goal conflicts is to admit they exist, always have and probably always will.  The key to resolving them is not by issuing a safety policy, it is to assure that an entity’s decision making process and its reward and compensation system treat safety with the priority it warrants. 

Decision making

Winokur says “Understand the nature of low-probability, high-consequence accidents driven by inadequate control of uncertainty, not cause-effect relationships . . .” (p. 14) and “Risk-informed decision making can be deceptive; focus on consequences, as well as probabilities.” (p. 16)  These observations are directly compatible with Nicholas Taleb: “This idea that in order to make a decision you need to focus on the consequences (which you can know) rather than the probability (which you can’t know) is the central idea of uncertainty.”***  See our June 18, 2013 post for a discussion of decisions that led to high-consequence (i.e., really bad) outcomes at Crystal River, Kewaunee and San Onofre.

There is no additional material in the presentation for a few important factors, so we will repeat earlier Safetymatters commentary on these topics.    

Complacency and
Accumulated residual risks that erode the safety margin

We have pointed out how organizations, especially high reliability organizations, strive to maintain mindfulness and combat complacency.  Complacency leads to hubris (“It can’t happen here”) and opens the door for the drift toward failure that occurs with normalization of deviance, constant environmental adaptations, “normal” system performance excursions, group think and an irreducible tendency for SC to decay over time.

Lack of oversight

This refers to everyone who has the responsibility to provide competent, timely, incisive assessment of an entity’s activities but fails to do so.  Their inaction or incompetence neither reinforces a strong SC nor prods a weak SC to improve. 

DNFSB Hearing with SC Expert Presentations

This was "the first of two hearings the Board will convene to address safety culture at Department of Energy defense nuclear facilities and the Board’s Recommendation 2011–1, Safety Culture at the Waste Treatment and Immobilization Plant."****  This hearing focused on presentations by SC experts: Sonya Haber (an SC consultant to DOE), NRC and NASA.  The experts’ slide presentations and a video of the hearing are available here.

Haber hit the right buttons in her presentation but neither she nor anyone else mentioned her DOE client's failure to date to integrate the SC assessments and self-assessments DOE initiated at various facilities in response to Recommendation 2011-1.  We still don’t know whether WTP SC problems exist elsewhere in the DOE complex.  We commented on the DOE’s response to 2011-1 on January 25, 2013 and March 31, 2014.

Winokur asked Haber about the NRC's "safety first" view vs. the DOE's "mission/safety balance."  The question suggests he may be thinking the "balance" perspective gives the DOE entities too much wiggle room to short change safety in the name of mission.

The NRC presenter was Stephanie Morrow.  Her slides recited the familiar story of the evolution of the SC Policy Statement and its integration into the Reactor Oversight Process.  She showed a new figure that summarized NRC’s SC interests in different columns of the ROP action matrix.  Chairman Winokur asked multiple questions about how much direction the NRC gives the licensees in how to perform SC assessments.  The answer was clear: In the NRC’s world, SC is the licensee's responsibility; the NRC looks for adequacy in the consideration of SC factors in problem resolution and SC assessments.  Morrow basically said if DNFSB is too prescriptive, it risks ending up "owning" the facility SC instead of the DOE and facility contractor.

Our Perspective

The Chairman’s presentation addressed SC in a general sense.  However, the reality of the DOE complex is a formidable array of entities that vary widely in scope, scale and missions.  A strong SC is important across the complex but one-size-fits-all approaches probably won’t work.  On the other hand, the custom fit approach, where each entity has flexibility to build its SC on a common DOE policy foundation doesn’t appear to lead to uniformly good results either.  The formal hearing to receive presentations from SC industry experts evidences that the DNFSB is gathering information on what works in other fields.  

Bottom line: The DNFSB is still trying to figure out the correct balance between prescription and flexibility in its effort to bring DOE to heel on the SC issue.  SC is an vital part of the puzzle of how to increase DOE line management effectiveness in ensuring adequate safety performance at DOE facilities.


*  P.S. Winokur, “A User’s Guide to Preventing Major Accidents,” presentation at the 2014 Nuclear Facility Safety Programs Annual Workshop (May 7, 2014).  The workshop was sponsored by the DOE Office of Environment, Health, Safety, and Security.  Thanks to Bill Mullins for bring this presentation to our attention.

**  E. Schein, Organizational Culture and Leadership (San Francisco, CA: Jossey-Bass, 2004), p. 11.

***  N. Taleb, The Black Swan (New York: Random House, 2007), p. 211.

****  DNFSB May 28, 2014 Public Hearing on Safety Culture and Board Recommendation 2011-1.

A Systems View of Two Industries: Nuclear and Air Transport

We have long promoted a systems view of nuclear facilities and the overall industry.  One consequence of that view is an openness to possible systemic problems as the root causes of incidents in addition to searching for malfunctioning components, both physical and human.

One system where we see this openness is the air transport industry—the air carriers and the Federal Aviation Administration (FAA).  The FAA has two programs for self-reporting of incidents and problems: the Voluntary Disclosure Reporting Program (VDRP) and the Aviation Safety Action Program (ASAP).  These programs are discussed in a recent report* by the FAA’s Office of Inspector General (OIG) and are at least superficially similar to the NRC’s Licensee Event Reporting and Employee Concerns Program.

What’s interesting is that VDRP is receptive to the reporting of both individual and systemic issues.  The OIG report says the difference between individual and systemic is “important because if the issue is systemic, the carrier will have to develop a detailed fix to address the system as a whole—whereas if the issue is more isolated or individual, the fix will be focused more at the employee level, such as providing counseling or training.” (p. 7)  In addition, it appears both FAA programs  are imbued with the concept of a “just culture,” another topic we have posted about on several occasions and which is often associated with a systems view.  A just culture is one where people are encouraged to provide essential safety-related information, the blame game is aggressively avoided, and a clear line exists between acceptable and unacceptable behavior.

Now the implementation of the FAA programs is far from perfect.  As the OIG points out, the FAA doesn't ensure root causes are identified or corrective actions are sufficient and long-lived, and safety data is not analyzed to identify trends that represent risks.  Systemic issues may not always be reported by the carriers or recognized by the FAA.  But overall, there appears to be an effort at open, comprehensive communication between the regulator and the regulated.

So why does the FAA encourage a just culture while the nuclear industry seems fixated on a culture of blame?  One factor might be the NRC’s focus on hardware-centric performance measures.  If these are improving over time, one might infer that any incidents are more likely caused by non-hardware, i.e., humans. 

But perhaps we can gain greater insight into why one industry is more accepting of systemic issues by looking at system-level factors, specifically the operational (or actual) coupling among industry participants versus their coupling as perceived by external observers.**

As a practical matter, the nuclear industry is loosely coupled, i.e., each plant operates more or less independently of the others (even though plants with a common owner are subject to the same policies as other members of the fleet).  There is seldom any direct competition between plants.  However, the industry is viewed by many external observers, especially anti-nukes, as a singular whole, i.e, tightly coupled.  Insiders reinforce this view when they say things like “an accident at one plant is an accident for all.”  And, in fact, one incident (e.g., Davis-Besse) can have industry-wide implications although the physical risk may be entirely local.  In such a socio-political environment, there is implicit pressure to limit or encapsulate the causes of any incidents or irregularities to purely local sources and avoid the mention of possible systemic issues.  The leads to a search for the faulty component, the bad employee, a failure to update a specific procedure or some other local problem that can be fixed by improved leadership and oversight, clearer expectations, more attention to detail, training etc.  The result of this approach (plus other industry-wide factors, e.g., the lack of transparency in certain oversight practices*** and the “special and unique” mantra) is basically a closed system whose client, i.e., the beneficiary of system efforts, is itself.

In contrast, the FAA’s world has two parts, the set of air carriers whose relationship with each another is loosely coupled, similar to the nuclear industry, and the air traffic control (ATC) sub-system, which is more tightly coupled because all the carriers share the same airspace and ATC.  Because of loose coupling, a systemic problem at a single carrier affects only that carrier and does not infect the rest of the industry.  What is most interesting is that a single airline accident (in the tightly coupled portion of the system) does not lead to calls to shut down the industry.  Air transport has no organized opposition to its existence.  Air travel is such an integral part of so many people’s lives that pressure exists to keep the system running even in the face of possible hazards.  As a consequence, the FAA has to occasionally reassert its interest in keeping safety risks from creeping into the system.  Overall, we can say the air transport industry is relatively open, able to admit the existence of problems, even systemic ones, without taking an inadvertent existential risk. 

The foregoing is not intended to be a comprehensive comparison of the two industries.  Rather it is meant to illustrate how one can apply a simple systems concept to gain some insights into why participants in different industries behave differently.  While both the FAA and NRC are responsible for identifying systemic issues in their respective industries, it appears FAA has an easier time of it.  This is not likely to change given the top-level factors described above. 


*  FAA Office of Inspector General, “Further Actions are Needed to Improve FAA’s Oversight of the Voluntary Disclosure Reporting Program” Report No. AV-2014-036 (April 10, 2014).  Thanks to Bill Mullins for pointing out this report to us.

“VDRP provides air carriers the opportunity to voluntarily report and correct areas of non-compliance without civil penalty. The program also provides FAA important safety information that might not otherwise come to its attention.“ (p. 1)  ASAP “allows individual aviation employees to disclose possible safety violations to air carriers and FAA without fear that the information will be used to take enforcement or disciplinary action against them.” (p. 2)

**  “Coupling” refers to the amount of slack, buffer or give between two items in a system.

***  For example, INPO’s board of directors is comprised of nuclear industry CEOs, INPO evaluation reports are delivered in confidence to its members and INPO has basically unfettered access to the NRC.  This is not exactly a recipe for gaining public trust.  See J.O. Ellis Jr. (INPO CEO), Testimony before the National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling (Aug. 25, 2010).  Retrieved from NEI website May 27, 2014.