Thursday, April 25, 2013

Inhibiting Excessive Risk Taking by Executives

The Federal Reserve Board has been doing some arm twisting with U.S. financial services companies to adjust their executive compensation plans - and those plans are in fact being modified to cap bonuses associated with achieving performance goals.  These actions have their genesis in the financial crisis where it appeared that incentives could encourage excessive risk taking by management.  A Wall Street Journal article* notes “regulators are still looking at ways to lower risk in the banking system, even if it means interfering with private pay practices.”  This follows a similar trend in Europe and where some firms are considering increasing salaries to make up for less bonus potential.

These actions merit some thoughtful consideration within the U.S. nuclear industry.  While the Fed’s concern is excessive business risk, the analog in nuclear operations is safety risk.  Both go to ensuring that the “system” (banking or nuclear production) remain within controlled limits.  As we have noted in prior blog posts (July 6 and July 9, 2010), there have been trends for nuclear executive compensation to both escalate and include significant performance based components.  The increased salaries probably reflect competition for the best qualified executives and are indicative of the great responsibilities of nuclear management.  However the trend to include large short-term bonuses (comprising up to 60-70% of total compensation) may be indicative of the evolution of “nuclear generation as a business” and the large profit potential available at high capacity factors.  Whatever the nominal amount of pressure on nuclear executives to achieve operating goals, the presence of very large monetary incentives can only increase that pressure.  In a strong safety culture environment where perception of management’s priorities is central, incentive based compensation plans can easily create presumptions regarding the motivation for management decisions.  At least one nuclear utility has concluded that incentives were not appropriate and taken action to adjust their compensation plans.  We have advocated dialing back incentives in favor of more direct compensation.

It is also rather interesting that the Fed decided to step into the province of private compensation practices.  A similar initiative by the NRC seems unlikely given its reluctance to impinge on management performance in any manner.  As noted in our February 28, 2013 post the NRC has included some nominal but poorly focused language on incentives in its Safety Culture Common Language Path Forward.  This seems to indicate that the NRC believes incentives are or could be relevant.  The best approach may be for the NRC to become more intrusive - to determine if compensation plans have the potential to lead to excessive risk taking.  This would require the NRC to obtain compensation plan information from its licensees, characterize the extent and magnitude of performance based incentives, and consider the effect of such incentives in assessing specific operational issues that arise in its normal regulatory oversight activities.  Only if some relationship appeared would the NRC need to consider whether to take action similar to the Fed or other means to ameliorate risk taking.



 

*  A.Lucchetti and J. Steinberg, "Regulators Get Banks to Rein In Bonus Pay," Wall Street Journal (April 23, 2013).

Monday, April 22, 2013

IAEA on Safety Culture in New Plant Design and Construction

The International Atomic Energy Agency (IAEA) has a 2012 publication* that provides guidance on establishing a strong safety culture (SC) during the design and construction of new nuclear power plants.  The report's premise, with which we agree, is a weak SC during plant design and construction can lead to later problems during plant operations.   

Major issues can arise during plant design and construction.  For example, the numerous organizations involved may have limited direct experience and/or insufficient knowledge of nuclear safety requirements, or projects may be located in countries with no existing nuclear industry or countries may have a nuclear industry but no recent construction experience.

The report attempts to cover the different needs, challenges and circumstances that may face project participants (governments, regulators, owners, designers, builders, manufacturers, etc.) anywhere in the world.  Most of the content addresses generic issues, e.g., understanding SC, the role of leadership, appropriate management systems, or communication and organizational learning.  Each issue is discussed in terms of specific challenges, goals, and recommended approaches and methods.  However, in their effort to attain maximum coverage (scope) IAEA sacrifices depth.  For example, the discussion of leadership covers five pages of the report but scarcely mentions the two most important activities of leaders: decision making and modeling safety-focused behavior.

If we look at the report's specific advice and recommendations, we see uneven coverage of the observable artifacts we consider essential for a strong SC: a decision making process that appropriately values safety, an effective corrective action program and financial incentives that reward safety performance. 

Decision making process

 
One overall challenge facing new projects is “Conflicts between schedule, cost and safety objectives can adversely affect conservative decision making and the maintenance of a questioning attitude, or impair the ability to perceive links between short term actions and their long term consequences.” (p. 2)

That's a good starting point but what are the characteristics of an appropriate decision making process?  It seems that decision making should be “conservative” (pp. 32, 34, 39), “broad” (p. 43) and “risk informed” (pp. 50, 51) but the terms are not defined. 
More specificity on how the decision making process should handle competing goals, set priorities and assign resources would be useful.
 
What about the decision makers?  Leaders should be able to “Explain the relationships between time periods/horizons and decision making to help resolve competing priorities.” (p. 41)  That's OK but the need goes beyond time periods.  The manager must be able to explain the rationale for significant decisions related to safety.  What were the considerations, assumptions, priorities, alternatives, decision factors and their relative weights, and the applicable laws, rules and regulations?  How should leaders
treat devil’s advocates who raise concerns about possible unfavorable outcomes?  Do leaders get the most qualified people involved in key decisions?
 
In addition, leaders should “Simulate decision making in a fast paced, complex environment to help leaders identify risks in their own approaches.”  (p. 41)  This is an excellent approach and we wholeheartedly support it. 

Corrective action

“Ineffective problem identification, inadequate reporting and inadequate corrective actions” (p. 9) were identified in a 2006 investigation as causal factors of construction problems at a nuclear plant site.  But there is no follow-up to describe the characteristics of an effective corrective action program.  There should be more about the CAP's ability to recognize and diagnose problems, formulate and implement solutions that consistently and appropriately consider safety, and monitor the effects of corrective actions. The importance of robust cause analysis, i.e., analysis that finds the real causes of problems so they do not recur, should be mentioned.  This would not be an unreasonable level of detail for this general report.

Financial incentives

The report correctly notes that “In construction environments, cultural attributes such as schedule awareness, cost focus and urgency of problem resolution are reinforced because they are rewarded by immediate measures of success.” (p. 8)  This becomes a specific challenge when “Contractor incentives are often driven by cost and schedule rather than by safety culture performance.” (p. 26).  A recommended fix is to “Establish a reward and incentive programme [sic] for the overall project, with objectives for safety performance and rewards that are either monetary or in the form of future contracts as a long term partner.” (p. 27).  This will probably result in a focus on industrial safety performance rather than the overall SC but it may be the best practical solution.  Periodic assessment of key contractors' SC should be used to identify any general SC issues. 

Our perspective

In prior posts, we have taken the IAEA to task for their overly bureaucratic approach.  So we're pleased to report this document actually provides some useful, sensible guidance (albeit often in an unprioritized, laundry list style) applicable to both countries initially embarking on the nuclear road and more experienced countries experiencing a nuclear renaissance.

The report makes a few important points.  For example, IAEA proposes a systems approach to thinking about all the project participants and the varied work they must accomplish.  “In the case of NPP projects, the ‘system’ involves human–social systems, work processes, complex technologies and multiple organizations in a global economic, energy, environmental and regulatory context.” (p. 11)  This is a viewpoint we have repeatedly advocated in this blog.
 

Overall this report is satisfactory and it does refer the reader to other IAEA publications for additional information on specific subjects.  But in trying to provide relevant material to a plethora of stakeholders, the report gives shorter shrift to factors we consider vital to establishing and maintaining a strong SC. 


*  M. Haage (IAEA), “Safety culture in pre-operational phases of nuclear power plant projects” (Vienna : International Atomic Energy Agency,  2012).

Tuesday, April 16, 2013

Warning Shot for Chevron

White vapor and black smoke.  From CSB report.
On August 6, 2012 a leaking pipe at the Chevron refinery in Richmond, CA led to a fire that shut down a crude oil distillation unit and caused over 15,000 people to report to local hospitals seeking treatment for respiratory and other health issues.  This was not a Texas City.  About 20 of the 15,000 people were admitted to local hospitals and there were some minor injuries to employees in the area of the fire but no fatalities.  However, it should be a wake-up call for Chevron. 

The proximate cause of the leak was a pipe ("4-sidecut") that had corroded because of the fluids that flowed through it.  But the Chevron and Chemical Safety Board (CSB) investigations showed there was a ten-year trail of missed possibilities to identify and correct the problem, including the following: In 2002, an employee inspector had expressed concern about sulfidation corrosion in the 4-sidecut and recommended upgrading it but his recommendation was never implemented.  In the same year, an incident at another Chevron refinery led the company to recommend 100% inspection of pipes for corrosion but this was not implemented at Richmond.  In 2009 and 2010 Chevron promulgated new warnings about sulfur corrosion and reiterated the recommendation for 100% inspection but Richmond did not implement any remedial actions on the 4-sidecut.*  In 2011, after a fire in another pipe, Richmond employees complained to Cal/OSHA about the company ignoring corrosion dangers but Chevron rationalized their way out of the issue.**

Chevron's incident investigation, including a root cause analysis, resulted in multiple corrective actions that will ring familiar to our readers.  Summarized, they are: look harder for corrosion; upgrade the hardware reliability program and supporting procedures; increase oversight and training; implement new rules for evaluating leaks; and emphasize the importance of process safety in decision making.  In even fewer words, tweak the system and retrain.

There is no mention of safety culture (SC) but the odor of a weak or compromised SC wafts from the report.  In a strong SC, the 2002 inspector would have identified the potential problem, documented it in the corrective action program and monitored progress until the issue was resolved.  The corrective action program would have evaluated, prioritized and resourced the problem's resolution consistent with its safety significance.  Outside experience and directives (from other Chevron entities or elsewhere) would have been regularly integrated into local operating practices, including inspection, maintenance and process procedures.

We are not alone in recognizing the importance of SC.  The local county supervisor, who also chairs the Bay Area Air Quality Management District, said “We need to do a thorough review of the safety culture at the refinery.”***  The CSB's managing director said the company had a “tolerance for allowing piping to run toward failure” and “I think it points to a certain cultural issue.”****  The CSB's interim report says “After reviewing evidence and decisions . . . the CSB has determined that issues relating to safety culture are relevant to this incident. The CSB will examine the Chevron Richmond Refinery’s approach to safety, its safety culture and any organizational deficiencies, to determine how to best prevent future incidents.” (p. 61)

We'll see if Chevron gets the hint.


*  CUSA Richmond Investigation Team, “Richmond Refinery 4 Crude Unit Incident August 6, 2012” (April 12, 2013).  Attachment to letter from S. Wildman (Chevron) to R.L. Sawyer (Contra Costa County Health Services), “Seventh Update to the 30-Day Report for the CWS Level 3 Event of August 6, 2012” (April 12, 2013).  


U.S. Chemical Safety and Hazard Investigation Board, “Interim Investigation Report Chevron Richmond Refinery Fire” (April 15, 2013).  In addition to Chevron, the CSB also criticizes regulatory and other government agencies, particularly Cal/OSHA, for shortcomings in their oversight of refinery activities.

**  J. Van Derbeken, “Chevron ignored risk in '11, workers say” sfgate.com (Oct. 13, 2012).

***  J. Van Derbeken, “Chevron fire report shows troubled history” sfgate.com (April 13, 2013).

****  J. Van Derbeken, “Chevron fire sign of weak oversight” sfgate.com (April 15, 2013).

Friday, April 12, 2013

A New Sheriff Coming to DOE?

On April 9th, the nominee for Secretary of Energy, Dr. Ernest Moniz, appeared before the Senate Committee on Energy and Natural Resources.  Most of the three hour hearing was in a Q&A format, with the committee chairman showing special interest in the major problems at Hanford, viz., leaking waste storage tanks and explosive hydrogen accumulation in same, the Waste Treatment Plant (aka Vit Plant) project and the site safety culture (SC).*

With respect to the SC issue, the nominee said it was “unacceptable” for SC to not be where it needs to be.  In response to a question from the committee chairman, Dr. Moniz said he was willing to meet with Vit Plant whistleblowers.  Depending on the outcome of such a meeting, if it occurs, the new Secretary could send a powerful signal to the Hanford site and beyond about his views on SC, Differing Professional Opinion (and related) practices, a Safety Conscious Work Environment and retaliation against employees who question organizational decisions.


*  The meeting video is available here, Hanford is discussed from about 3:05 to 3:20. 

A letter from the DNFSB chairman provides a good summary of the key issues at Hanford.  See P.S. Winokur (DNFSB) to R.L. Wyden (chairman of the Senate Committee on Energy and Natural Resources), letter providing the DNFSB's perspective on the state of nuclear safety at the Hanford Site (April 1, 2013).

Tuesday, April 9, 2013

How Do You Actually Implement the NRC’s Safety Culture Policy Statement?

As we all know the NRC issued a policy statement on safety culture almost two years ago, presumably to set expectations for the industry to maintain strong safety cultures.  The policy statement is long on fuzzy things like traits and values but unfortunately short on specific “what do we do’s”.  The biggest hint may be in the invocation to accord safety issues the priority warranted by their significance.  Sounds right in theory but how does this actually get operationalized?

We are going to suggest a specific approach to apply safety culture policy in day-to-day nuclear management decision making.  As we’ve argued many times, focusing on specific decisions moves safety from the realm of theory to practice.  In general we think there may be significant advantages for organizations to do more to highlight, document and measure decisions involving safety considerations.  The benefit will be insight and emphasis on the “how and why” decisions are made and whether they achieve the safety balance envisioned by the NRC - and more importantly by the organization’s leadership.

We start with a numeric scale for what we call “decision quality”.  In our minds quality means how well a decision balances the priority accorded safety in light of the significance of the issue being addressed; in other words how well the decision does what the policy statement asks.  Conceptually this implies that an optimal decision achieves just the right balance for safety and that other decisions could under or over-shoot the optimal balance.  Can there be too much priority for safety?  Sure - remember the goal is to perform the nuclear mission safely, not to just pursue safety itself.  Here is the scale:



The scale sets a 0 value as the measure of an optimal balance of safety - meaning that it meets the expectation of the policy statement to give the priority warranted by safety significance.  Increasing positive values are associated with decisions that accord extra weight to safety; increasing negative values accord too little.  Use of a quantitative scale is the first step in being able to grade, track and provide feedback on decisions on a consistent basis.  When coupled with discussion of how significance was assessed and what the appropriate safety response needs to be, it provides many opportunities for a check and adjust process and organizational learning.

This leads to the next question which is: how should significance be determined?  There is of course NRC guidance via the significance determination process (SDP), including the red, yellow, white and green rainbow of significance levels, and this is our starting point.  The SDP include both qualitative (e.g. significant reduction in safety margin) and quantitative criteria (e.g. values of delta CDF and delta LERF).  While qualitative criteria may seem to some as lacking specificity, we’re fine with their use and in any event they are endemic in safety regulation.  We’re actually not that fond of the quantitative criteria since they are inherently hardware centric and do not encompass the complexity of the overall “system” that ultimately determines safety.  To provide quantification our approach is to again create a scale that correlates numeric values with the qualitative criteria.  Such “anchored scales” are a common and effective tool in decision analysis.  In addition we feel that the significance determinations need to be supplemented with an assessment of their uncertainty. 

It doesn’t take reviewing many event reports to see that judgments about safety significance are not always clear cut or unambiguous.  This variability in the adjudged significance can be the enabling mechanism for safety to not receive the appropriate priority - not because the priority doesn’t match the significance but because the significance has been discounted to justify a lower priority.  The catalysts can be as simple as overly optimistic thinking, normalization of deviation, complacency, or failing to ensure that the burden is on showing that something is safe versus showing that it is unsafe.  Our approach is to explicitly address the uncertainty of safety significance by introducing a second quantitative scale for this purpose.  When used together a judgment regarding significance would include both a nominal value (per SDM) and an uncertainty value.  These scales are illustrated below:







For decision making purposes the three scales would operate together to help arrive at appropriate decisions.  The significance scale would provide a nominal risk value.  If there was a little uncertainty in the assessed significance then the objective would be to make a decision that scores approximately “0” on the balance scale.  If there was greater uncertainty in the assessed significance the objective would be to select a decision option that scored higher on the balance scale; essentially giving safety higher priority to accommodate the potentially greater significance.  Decision options that rated negative balance values would avoided.

We see much of the value in this approach to be the focus on making the decision formation process more explicit, transparent and measurable.  Over time this structure provides greater opportunities for the organization to understand decisions and learn from the process not just the outcomes.  We also believe it may provide the basis for inferring and trending the safety culture within an organization.

In an upcoming post we’ll apply these decision scales to a specific plant situation to see how they might work in practice.

Saturday, April 6, 2013

2012 NRC Safety Culture Survey Results

ADAMS ML13087A326
Originally published 4-4-13.  This version updated with data from the associated staff briefing slide presentation.

The 2012 NRC Safety Culture and Climate Survey results are available in an Inspector General report* and a consultants' slide briefing.**  The top-level findings are (1) the 2012 results are not as favorable as the previous 2009 survey results and (2) the NRC compares favorably with national norms but lags when compared to a group of high performing companies (with strong financial results and high employee survey scores).  Let's look at some of the details.  All page references are to the report except where noted otherwise. 

The survey's 132 items were aimed at evaluating employee perceptions in 20 categories.  Many of these categories primarily addressed personnel practices—communication, supervision, diversity, training, development and the like.  However, it should come as no surprise to our regular readers that the categories of interest to us address, at least in part, the key business processes of decision making, priority setting and conflict resolution, i.e., areas where the goal of safety often competes with other goals.


Four categories appear to satisfy our criteria:

DPO/Non-Concurrence (DPO): “. . . employee awareness and perceived effectiveness of the Differing Professional Opinions program and the Non-concurrence process.”  This is one type of conflict resolution.  This category had the lowest number of favorable responses (although still over 50%) in the survey. (p. 13)  Three specific DPO items were among those that showed the most slippage, i.e., had fewer favorable responses, in 2012 compared to 2009. (Slides, p. 16)  Region IV had significantly*** fewer favorable 2012 scores on DPO compared to 2009. (p. 26) 

The consultants' cover letter identified this as an area for NRC management attention, saying the agency was “Losing significant ground on negative reactions when raising views different from senior management, supervisor, and peers.” 

NRC Mission and Strategic Plan: “. . . whether employees believe management decisions are consistent with the mission and strategic plan. . . .”    Compared with the high performing companies, the NRC scored 1 point lower on NRC Mission and Strategic Plan. (p. 17)  The Office of New Reactors and Region IV had significantly fewer favorable 2012 scores on NRC Mission and Strategic Plan compared to 2009. (pp. 25-26) 

The NRC Mission and Strategic Plan was identified as one of three key drivers of employee Engagement, also a survey category**** but treated as a dependent variable in a supporting multiple regression analysis.  In responding to specific questions, employees said they believed they were “sufficiently informed about NRC's performance of its mission” and that “management decisions are consistent with the mission” but both items scored significantly lower than in 2009 and compared to the high performing companies. (Slides, p. 33)

Quality focus: “. . . employee views on . . . the sacrifice of quality work due to the need to meet a deadline or the need to satisfy a personal or political agenda.”  This category had the third lowest number of favorable responses in the survey. (p. 13)  This category was also mentioned in the consultants' cover letter: “Reinforcing a key point raised in the focus groups [but one that did not stand out in the survey results], there is a clear opportunity to impact the perception that people sacrifice quality in order to meet metrics.”

Senior Management: “. . . confidence in management’s decisions.”  Compared with the high performing companies, the NRC scored 7 points lower on Senior Management, in a 3-way tie for second lowest. (p. 17)  This result may have been affected by this item: Only 41 percent of the respondents “. . . feel significant actions have been taken as a result of the previous Safety Culture and Climate survey.” (Slides, p. 23)  This issue was included in the list of conclusions to the consultants' report.  On the other hand, at least 75% favorable responses were recorded for senior management providing a clear sense of direction and employee confidence in senior management decisions. (Slides, p. 23)  That may look good but both items scored significantly lower than in 2009 and compared to the high performing companies.

The Office of New Reactors and Region IV had significantly less favorable 2012 scores on Senior Management compared to 2009. (pp. 25-26)  Region IV also had a significantly less favorable 2012 score on Senior Management than the overall NRC score. (p. 23)

Our Perspective

The report consists of mostly charts and graphs, with a lot of superficial data slicing and dicing and some authoritative-sounding conclusions.  The slide presentation shows additional data to illustrate some problem areas.  Both documents reinforce our belief in the limited usefulness of surveys and the problems associated with over-reliance on outside experts.  My “analysis” above is obviously limited but it's difficult to dig deeply because only a few of the 132 specific items are detailed in the report and slides. 

But the available data suggest that raising views inconsistent with the party line can lead to negative reactions.  NRC employees have some confidence the agency makes decisions consistent with its mission but less confidence in their senior management to take action on survey results.   

NRC senior management has a much more favorable view of the agency's situation than the overall organization.  Senior managers' survey responses were significantly more favorable than the overall NRC response in ALL 20 categories and an average of 18 percent more favorable in the 4 categories included in this post. (Slides, p. 37)  This suggests a possible disconnect between the bosses and everyone else.

And speaking of disconnections, it appears neither the group responsible for the Nuclear Renaissance nor Region IV is fully on the same page as the rest of the agency.

Finally, the documents' omission of safety as a goal or priority is notable.  “Nuclear safety” as a goal is only mentioned in the definition of SC.  Safety is mentioned as “safety concepts” in the Training category and the “NRC’s commitment to public safety” in the Continuous Improvement Commitment category.  One might expect safety to be more front and center in the SC survey. 


*  NRC Office of the Inspector General, “2012 NRC Safety Culture and Climate Survey,” OIG-13-A-15 (March 28, 2013)  ADAMS ML13087A326.  Although this was mostly a survey, the consultants (Towers Watson) did conduct some individual interviews and focus groups to help shape the survey content. Interestingly, the definition of safety culture used in the 2012 survey was not the same as the definition in the current NRC policy statement.  Instead, an earlier definition was used to permit comparisons between current survey results and prior years.

**  Towers Watson, “Nuclear Regulatory Commission 2012 Safety Culture and Climate Survey Briefing for NRC Staff” (Nov. 8, 2012).

***  “Significant” means statistically significant.

****  Engagement “Probes employees’ willingness to recommend the NRC as a good place to work, whether they feel they are a part of the agency, their pride in working for the NRC and their belief in NRC goals, objectives, and values.” (p. 10)