On April 9th, the nominee for Secretary of Energy, Dr. Ernest Moniz, appeared before the Senate Committee on Energy and Natural Resources. Most of the three hour hearing was in a Q&A format, with the committee chairman showing special interest in the major problems at Hanford, viz., leaking waste storage tanks and explosive hydrogen accumulation in same, the Waste Treatment Plant (aka Vit Plant) project and the site safety culture (SC).*
With respect to the SC issue, the nominee said it was “unacceptable” for SC to not be where it needs to be. In response to a question from the committee chairman, Dr. Moniz said he was willing to meet with Vit Plant whistleblowers. Depending on the outcome of such a meeting, if it occurs, the new Secretary could send a powerful signal to the Hanford site and beyond about his views on SC, Differing Professional Opinion (and related) practices, a Safety Conscious Work Environment and retaliation against employees who question organizational decisions.
* The meeting video is available here, Hanford is discussed from about 3:05 to 3:20.
As we all know the NRC issued a policy statement on safety culture almost two years ago, presumably to set expectations for the industry to maintain strong safety cultures. The policy statement is long on fuzzy things like traits and values but unfortunately short on specific “what do we do’s”. The biggest hint may be in the invocation to accord safety issues the priority warranted by their significance. Sounds right in theory but how does this actually get operationalized?
We are going to suggest a specific approach to apply safety culture policy in day-to-day nuclear management decision making. As we’ve argued many times, focusing on specific decisions moves safety from the realm of theory to practice. In general we think there may be significant advantages for organizations to do more to highlight, document and measure decisions involving safety considerations. The benefit will be insight and emphasis on the “how and why” decisions are made and whether they achieve the safety balance envisioned by the NRC - and more importantly by the organization’s leadership.
We start with a numeric scale for what we call “decision quality”. In our minds quality means how well a decision balances the priority accorded safety in light of the significance of the issue being addressed; in other words how well the decision does what the policy statement asks. Conceptually this implies that an optimal decision achieves just the right balance for safety and that other decisions could under or over-shoot the optimal balance. Can there be too much priority for safety? Sure - remember the goal is to perform the nuclear mission safely, not to just pursue safety itself. Here is the scale:
The scale sets a 0 value as the measure of an optimal balance of safety - meaning that it meets the expectation of the policy statement to give the priority warranted by safety significance. Increasing positive values are associated with decisions that accord extra weight to safety; increasing negative values accord too little. Use of a quantitative scale is the first step in being able to grade, track and provide feedback on decisions on a consistent basis. When coupled with discussion of how significance was assessed and what the appropriate safety response needs to be, it provides many opportunities for a check and adjust process and organizational learning.
This leads to the next question which is: how should significance be determined? There is of course NRC guidance via the significance determination process (SDP), including the red, yellow, white and green rainbow of significance levels, and this is our starting point. The SDP include both qualitative (e.g. significant reduction in safety margin) and quantitative criteria (e.g. values of delta CDF and delta LERF). While qualitative criteria may seem to some as lacking specificity, we’re fine with their use and in any event they are endemic in safety regulation. We’re actually not that fond of the quantitative criteria since they are inherently hardware centric and do not encompass the complexity of the overall “system” that ultimately determines safety. To provide quantification our approach is to again create a scale that correlates numeric values with the qualitative criteria. Such “anchored scales” are a common and effective tool in decision analysis. In addition we feel that the significance determinations need to be supplemented with an assessment of their uncertainty.
It doesn’t take reviewing many event reports to see that judgments about safety significance are not always clear cut or unambiguous. This variability in the adjudged significance can be the enabling mechanism for safety to not receive the appropriate priority - not because the priority doesn’t match the significance but because the significance has been discounted to justify a lower priority. The catalysts can be as simple as overly optimistic thinking, normalization of deviation, complacency, or failing to ensure that the burden is on showing that something is safe versus showing that it is unsafe. Our approach is to explicitly address the uncertainty of safety significance by introducing a second quantitative scale for this purpose. When used together a judgment regarding significance would include both a nominal value (per SDM) and an uncertainty value. These scales are illustrated below:
For decision making purposes the three scales would operate together to help arrive at appropriate decisions. The significance scale would provide a nominal risk value. If there was a little uncertainty in the assessed significance then the objective would be to make a decision that scores approximately “0” on the balance scale. If there was greater uncertainty in the assessed significance the objective would be to select a decision option that scored higher on the balance scale; essentially giving safety higher priority to accommodate the potentially greater significance. Decision options that rated negative balance values would avoided.
We see much of the value in this approach to be the focus on making the decision formation process more explicit, transparent and measurable. Over time this structure provides greater opportunities for the organization to understand decisions and learn from the process not just the outcomes. We also believe it may provide the basis for inferring and trending the safety culture within an organization.
In an upcoming post we’ll apply these decision scales to a specific plant situation to see how they might work in practice.
Originally published 4-4-13. This version updated with data from the associated staff briefing slide presentation.
The 2012 NRC Safety Culture and Climate Survey results are available in an Inspector General report* and a consultants' slide briefing.** The top-level findings are (1) the 2012 results are not as favorable as the previous 2009 survey results and (2) the NRC compares favorably with national norms but lags when compared to a group of high performing companies (with strong financial results and high employee survey scores). Let's look at some of the details. All page references are to the report except where noted otherwise.
The survey's 132 items were aimed at evaluating employee perceptions in 20 categories. Many of these categories primarily addressed personnel practices—communication, supervision, diversity, training, development and the like. However, it should come as no surprise to our regular readers that the categories of interest to us address, at least in part, the key business processes of decision making, priority setting and conflict resolution, i.e., areas where the goal of safety often competes with other goals. Four categories appear to satisfy our criteria:
DPO/Non-Concurrence (DPO): “. . . employee awareness and perceived effectiveness of the Differing Professional Opinions program and the Non-concurrence process.” This is one type of conflict resolution. This category had the lowest number of favorable responses (although still over 50%) in the survey. (p. 13) Three specific DPO items were among those that showed the most slippage, i.e., had fewer favorable responses, in 2012 compared to 2009. (Slides, p. 16) Region IV had significantly*** fewer favorable 2012 scores on DPO compared to 2009. (p. 26)
The consultants' cover letter identified this as an area for NRC management attention, saying the agency was “Losing significant ground on negative reactions when raising views different from senior management, supervisor, and peers.”
NRC Mission and Strategic Plan: “. . . whether employees believe management decisions are consistent with the mission and strategic plan. . . .” Compared with the high performing companies, the NRC scored 1 point lower on NRC Mission and Strategic Plan. (p. 17) The Office of New Reactors and Region IV had significantly fewer favorable 2012 scores on NRC Mission and Strategic Plan compared to 2009. (pp. 25-26)
The NRC Mission and Strategic Plan was identified as one of three key drivers of employee Engagement, also a survey category**** but treated as a dependent variable in a supporting multiple regression analysis. In responding to specific questions, employees said they believed they were “sufficiently informed about NRC's performance of its mission” and that “management decisions are consistent with the mission” but both items scored significantly lower than in 2009 and compared to the high performing companies. (Slides, p. 33)
Quality focus: “. . . employee views on . . . the sacrifice of quality work due to the need to meet a deadline or the need to satisfy a personal or political agenda.” This category had the third lowest number of favorable responses in the survey. (p. 13) This category was also mentioned in the consultants' cover letter: “Reinforcing a key point raised in the focus groups [but one that did not stand out in the survey results], there is a clear opportunity to impact the perception that people sacrifice quality in order to meet metrics.”
Senior Management: “. . . confidence in management’s decisions.” Compared with the high performing companies, the NRC scored 7 points lower on Senior Management, in a 3-way tie for second lowest. (p. 17) This result may have been affected by this item: Only 41 percent of the respondents “. . . feel significant actions have been taken as a result of the previous Safety Culture and Climate survey.” (Slides, p. 23) This issue was included in the list of conclusions to the consultants' report. On the other hand, at least 75% favorable responses were recorded for senior management providing a clear sense of direction and employee confidence in senior management decisions. (Slides, p. 23) That may look good but both items scored significantly lower than in 2009 and compared to the high performing companies.
The Office of New Reactors and Region IV had significantly less favorable 2012 scores on Senior Management compared to 2009. (pp. 25-26) Region IV also had a significantly less favorable 2012 score on Senior Management than the overall NRC score. (p. 23)
Our Perspective
The report consists of mostly charts and graphs, with a lot of superficial data slicing and dicing and some authoritative-sounding conclusions. The slide presentation shows additional data to illustrate some problem areas. Both documents reinforce our belief in the limited usefulness of surveys and the problems associated with over-reliance on outside experts. My “analysis” above is obviously limited but it's difficult to dig deeply because only a few of the 132 specific items are detailed in the report and slides.
But the available data suggest that raising views inconsistent with the party line can lead to negative reactions. NRC employees have some confidence the agency makes decisions consistent with its mission but less confidence in their senior management to take action on survey results.
NRC senior management has a much more favorable view of the agency's situation than the overall organization. Senior managers' survey responses were significantly more favorable than the overall NRC response in ALL 20 categories and an average of 18 percent more favorable in the 4 categories included in this post. (Slides, p. 37) This suggests a possible disconnect between the bosses and everyone else.
And speaking of disconnections, it appears neither the group responsible for the Nuclear Renaissance nor Region IV is fully on the same page as the rest of the agency.
Finally, the documents' omission of safety as a goal or priority is notable. “Nuclear safety” as a goal is only mentioned in the definition of SC. Safety is mentioned as “safety concepts” in the Training category and the “NRC’s commitment to public safety” in the Continuous Improvement Commitment category. One might expect safety to be more front and center in the SC survey.
* NRC Office of the Inspector General, “2012 NRC Safety Culture and Climate Survey,” OIG-13-A-15 (March 28, 2013) ADAMS ML13087A326. Although this was mostly a survey, the consultants (Towers Watson) did conduct some individual interviews and focus groups to help shape the survey content. Interestingly, the definition of safety culture used in the 2012 survey was not the same as the definition in the current NRC policy statement. Instead, an earlier definition was used to permit comparisons between current survey results and prior years.
*** “Significant” means statistically significant.
**** Engagement “Probes employees’ willingness to recommend the NRC as a good place to work, whether they feel they are a part of the agency, their pride in working for the NRC and their belief in NRC goals, objectives, and values.” (p. 10)
On January 25, 2013 we posted about DOE's report to the Defense Nuclear Facilities Safety Board (DNFSB) on the results of safety culture (SC) assessments at several DOE facilities, including the Pantex Plant. Pantex was assessed because two Pantex employees had reported retaliation for raising a safety concern but the plant also had a history of potentially SC-significant issues.*
The Pantex SC assessment was performed in November 2012. The report included several significant findings:
“Efforts to communicate and implement the principles of a High Reliability Organization (HRO) have been ongoing for several years. . . . [However,] The realization of the HRO principles has not yet been internalized by the Plant, . . .”
“The belief that the organization places a priority on safety is undermined by employee observations of poor facility conditions, lack of focus on meeting personal needs (work quality of life), and a sense of cronyism. . . . [This] has created the perception among many employees that the financial bottom line is the only focus that matters.
“There is a strong perception that retaliation exists for ‘rocking the boat.’ . . . The perception has created an environment where the raising of questions or identification of problems is not the consistently accepted way of doing business.
“The Pantex Plant has not been successful in understanding the organizational and programmatic behaviors that are necessary for a healthy safety culture. . . . organizational barriers have been created that will prevent successful implementation of the initiatives needed to enhance safe and reliable performance. . . . The barriers are also evident in the lack of respect, difficulty in effective communication, the non-alignment between the perceptions around the unions and management relationships and the notion of ‘need to know’ being extended to almost everything.”**
Sounds serious. So what's happened since the report was published? Well, DNFSB held a public meeting on March 14, 2013 to discuss SC problems at Pantex and management's approach to addressing them. As one might expect, the DOE opening statement declared the SC problems were intolerable and DOE had taken immediate action—by firing off a letter to the contractor.
“NNSA issued a letter to B&W Pantex directing immediate focus at all management and working levels to a safety conscious work environment (SCWE) for all on-going activities and operations. . . . Other immediate actions included the development of a single stop/pause work process which was provided to all employees and discussed at daily work planning meetings. The Employee Concerns Program and Equal Opportunity Office reporting chain was immediately elevated to a direct report to the General Manager’s Office, providing the highest level accesses for any employee concerns in these areas. Additionally, the differing professional opinion process was reinstated providing a formal mechanism for recognition and resolution of differing views on technical matters.
“Further, this letter directed B&W Pantex to prepare a comprehensive long-term corrective action plan mentioned previously, taking into account the Institute of Nuclear Power Operations (INPO) paper on the principles of a strong nuclear safety culture.”***
DOE listed actions being implemented, including an SC focus team, a seminar on HRO attributes, SCWE training, the issuance of a plant-specific SC policy and an alignment of performance incentives with SC objectives.
Babcock & Wilcox (B&W) B&W, the entity that actually has to do the work, did not file any written testimony for the public meeting so we have to go to the meeting video for their comments.****
B&W appears to be on board with the need for change. The B&W plant general manager and his safety manager were appropriately deferential to the DNFSB members and seemingly well-informed about the plant's remedial actions and SC improvement initiatives. The GM said he “fully accepts” the assessment findings (which were consistent with the plant's own SC survey conducted just prior to the DOE HSS assessment) and acknowledged that specific problems, e.g., communication issues with respect to safety vs production, existed. Improving SC/SCWE is the GM's “top priority.” B&W reiterated its commitment to building an HRO at Pantex, an initiative that overlaps with actions to strengthen SC and SCWE. Perhaps the most significant change the GM described was that 30% of managers' performance evaluations would be based on their modeling of appropriate SC/SCWE traits. Our perspective
Problem solved? Not yet and not for awhile. Pantex had some serious vertical communication and organizational structure issues. Their attempt to build an HRO has been ongoing for years. Their SCWE has had some cold spots.
In addition, the actions Pantex has initiated may be necessary but there is no guarantee they will be sufficient to achieve the plant's SC/SCWE/HRO goals. For example, there is no real discussion of how decision making processes will be affected other than resolving Nuclear Explosive Safety issues and the usual commitment to conservative decision making. There is no mention of a corrective action program (or some functional equivalent); an integrated process for identifying, evaluating and fixing problems is essential for ensuring safety, priorities and resource allocation are treated consistently throughout the plant.
We'll watch for progress (or lack thereof) and keep you posted.
* Pantex is the sole US site that assembles and disassembles nuclear weapons. Within DOE, the National Nuclear Security Administration (NNSA) has line management responsibility for Pantex. Babcock & Wilcox is responsible for managing and operating the plant under contract with DOE.
The SC issues identified in the November 2012 assessment did not pop out of nowhere. The DNFSB identified SC-related concerns at the plant during the previous year. See statement of D.G. Ogg, Group Lead for Nuclear Weapons Programs, DNFSB, at the Pantex public meeting (March14, 2013).
** DOE Office of Enforcement and Oversight, “Independent Oversight Assessment of Nuclear Safety Culture at the Pantex Plant” (Nov. 2012) p. 3. The report is attached to the letter from G.S. Podansky (DOE) to P.S. Winokur (DNFSB) transmitting five independent safety culture assessments (Dec. 12, 2012).
*** Written testimony of Neile Miller, Acting Administrator, NNSA before the Defense Nuclear Facilities Safety Board Pantex Plant Public Meeting (March 14, 2013), p. 4.
**** The meeting video is available on the DNFSB website. The NNSA panel on Pantex SC runs from about 1:25 to 2:30, the B&W panel runs from about 2:35 to 3:20.
Last week we previewed the safety culture (SC) content of the then-upcoming NRC RIC. The Idaho National Lab speaker's slides were not available at that time but they are now and his presentation is reviewed below. The focus is on the Advanced Test Reactor Programs but I think it's fair to infer that the thinking is representative of a wider swath of the DOE complex.
The presentation opens with five lengthy quotes from Admiral Rickover's November 1983 assessment of GPU and its competence to operate TMI-1. The apparent intent is to illustrate that the principles for safe nuclear operations have been known (or at least available) for a long time. Coincidentally, we posted on the Rickover assessment two months ago, and focused on one of the same quotes. If you aren't acquainted with Rickover's seven principles, you really should read the introduction to the assessment, which is available from the Dickinson College library.
The presentation describes components of the new DOE Cross Cutting Performance Areas for category 1, 2 and 3 nuclear facilities: Evaluating the effectiveness of operations, maintenance, engineering and training programs; developing, monitoring and evaluating SC; and evaluating issue identification and resolution activities, including the significance determination process and the evaluation and resolution process for high significance issues.
The presentation concluded with a list of areas being emphasized at the Idaho lab: What is the right (as opposed to allowable) thing to do, educating leaders, communications, and decision making that reflects a learning organization and doesn't result in safety drift.
The presentation hit most of the right notes, a major exception being no mention of management or contractor financial incentive plans. However, the unmistakable tone is there is really nothing new required of the lab, just a refinement of past and current practices. Perhaps that's true for them but I have limited confidence in DOE entities' ability to self-evaluate. We're pretty sure SC issues exist or have existed at other DOE facilities, especially the Vit Plant (click the label in the top right-hand column to pull up our posts).
The RIC is this week, March 12-14. The teaser on the NRC blog says the technical sessions will include safety culture (SC) policies. Let's look at the program agenda and see what's in store for SC.
There will be 36 technical sessions. I reviewed all the titles and drilled down into sessions that might make some mention of SC, e.g., T4 - Construction Inspection Experience–The First Year and T7 - Human Impacts. However, I could find no mention of SC in any of the currently available slide presentations.
The introductory remarks summarize the development of the SC policy statement and its implementation. There is no news here.
The SC common language presentation reviews the history of this initiative (which we have previously reviewed here and here). The presentation has one quotable statement: “NRR will work to incorporate language into the ROP guidance documents and inspection procedures, as appropriate.” Does that sound like back door regulation of SC to you?
A presentation on domestic and international cooperation reviews the relationship between NRC and INPO, NRC and IAEA, and others. As an example of cooperation, the authors summarize the INPO SC survey data that were collected from operating plants and then analyzed by INPO (and later NRC) to show “statistically significant relationships between safety culture survey results and measures of plant performance.” We commented on this work when it first appeared in 2010, congratulating INPO for making the effort and agreeing with some of the findings but finally concluding that the analysis was incomplete and potentially misleading.
Anindustry presentation by Nuclear Fuel Services Inc. (NFS) describing their SC improvement program is worth a look. It lists almost two dozen program components, none of which is a trivial undertaking, which suggest how much work is involved in changing an existing SC. (I have no idea if NFS is actually pursuing the listed activities or how well they're doing.)
All in all, it's probably not worth traveling to Bethesda if you're seeking enlightenment about SC.
In November 2003 Professor Edgar Schein gave a speech at the INPO CEO conference.* It was not a lengthy academic lecture but his focus on managing culture, as opposed to changing or creating it, was interesting. At the time Schein was doing some work for ConEd and had a notion of nuclear plant culture, which he divided into four sub-cultures: engineering, hourly, operator and executive, each with its own underlying assumptions and values.
The engineering culture emphasizes elegant, possibly expensive designs that minimize the role of error-prone humans. Engineers want and value respect from other engineers, including those outside the plant (an external orientation).
The hourly culture (which I think means maintenance) values teamwork and has an experience-based perspective on safety. They want job security, fair wages, good equipment, adequate training and respect from their peers and supervisors.
The operator culture values teamwork and open communications. They see the invaluable contributions they make to keeping the plant running safely and efficiently. They want the best equipment, training and to be recognized for their contributions.
The executive culture is about money. They want productivity, cost control, safety and good relations with their boards of directors (another external orientation).
These sub-cultures are in conflict because they all can't have everything they want. The executive needs to acknowledge that cultural differences exist and each sub-culture brings certain strengths to the table. The executive's role is to create a climate of mutual respect and to work toward aligning the sub-cultures to achieve common goals, e.g., safety. The executive should not be trying to impose the values of a single sub-culture on everyone else. In other words, the executive should be a culture manager, not a culture changer.
This was a brief speech and I don't want to read too much into it. There are dysfunctional or no longer appropriate cultures and they have to be reworked, i.e., changed. But if many things are working OK, then build on the existing strengths.**
This was not a speech about cultural interventions. At the beginning, Schein briefly described his tri-level cultural model and noted if the observed artifacts match the espoused values, then there's no need to analyze the underlying assumptions. This is reminiscent of Commissioner Apostolakis' comment that “. . . we really care about what people do and maybe not why they do it . . . .”
** Focusing on strengths (and not wasting resources trying to shore up weaknesses unless they constitute a strategic threat) is a management prescription first promoted by Peter Drucker.
Another domino has fallen in the ongoing determination of culpability at Massey Energy in the Big Branch mine disaster. The February 28, 2013 Wall Street Journal* reports that the former head of a Massey subsidiary, Green Valley Coal, warned miners when federal inspectors were on their way into mines and to conceal safety hazards. The former executive specifically stated that the order to do this came from Massey’s CEO.
Thus it appears prosecutors are following the trail of bread crumbs in an inexorable climb to the CEO level. So often situations like this are simply attributed to weaknesses in the organization’s safety culture, particularly at the working levels. It is assumed that senior management’s policies and direction to make safety the first priority aren’t permeating the organization. More training, more indoctrination in safety priorities is required to get workers aligned with their corporate leadership. But what is becoming very apparent in the case of Massey, it is the intentional decisions by senior management prioritizing production over safety that drove the behavior of subordinates - and it was those working levels that suffered the immediate consequences. Now perhaps the consequences are being more fairly distributed.
