Going Beyond SCART: A More Useful Guidebook for Evaluating Safety Culture

Our March 11 post reviewed the IAEA SCART guidelines.  We found its safety culture characteristics and attributes comprehensive but its “guiding questions” for evaluators were thin gruel, especially in the areas we consider critical for safety culture: decision making, corrective action, work backlogs and management incentives.

This post reviews another document that combines the SCART guidelines, other IAEA documents and the author’s insights to yield a much more robust guidebook for evaluating a facility’s safety culture.  It’s called “Guidelines for Regulatory Assessment of Safety Culture in Licensees’ Organisations.”*  It starts with the SCART characteristics and attributes but gives more guidance to an evaluator: recommendations for documents to review, what to look for during the evaluation, additional (and more critical) guiding questions, and warning signs that can indicate safety culture weaknesses or problems.

Specific guidance in the areas we consider critical is generally more complete.  For example, in the area of decision making, evaluators are told to look for a documented process applicable to all matters that affect safety, attend meetings to observe the decision-making process, note the formalization of the decision making process and how/if long-term consequences of decisions are considered.  Goal conflict is explicitly addressed, including how differing opinions, conflict based on different experiences, and questioning attitudes are dealt with, and the evidence of fair and impartial methods to resolve conflicts.  Interestingly, example conflicts are not limited to the usual safety vs. cost or production but include safety vs. safety, e.g., a proposed change that would increase plant safety but cause additional personnel rad exposure to implement.  Evidence of unresolved conflicts is a definite warning flag for the evaluator. 

Corrective action (CA) also gets more attention, with questions and flags covering CA prioritization based on safety significance, the timely implementation of fixes, lack of CA after procedure violations or regulatory findings, verification that fixes are implemented and effective, and overall support or lack thereof for the CAP. 

Additional questions and flags cover backlogs in maintenance, corrective actions, procedure changes, unanalyzed physical or procedural problems, and training.

However, the treatment of management incentives is still weak, basically the same as the SCART guidelines.  We recommend a more detailed evaluation of the senior managers’ compensation scheme or, in more direct language, how much do they get paid for production, and how much for safety?

The intended audience for this document is a regulator charged with assessing a licensee’s safety culture.  As we have previously discussed, some regulatory agencies are evaluating this approach.  For now, that’s a no-go in the U.S.  In any case, these guidelines provide a good checklist for self-assessors, internal auditors and external consultants.


*  M. Tronea, “Guidelines for Regulatory Oversight of Safety Culture in Licensees’ Organisations” Draft, rev. 8 (Bucharest, Romania:  National Commission for Nuclear Activities Control [CNCAN], April 2011).  In addition to being on the staff of CNCAN, the nuclear regulatory authority of Romania, Dr. Tronea is the founder/manager of the LinkedIn Nuclear Safety group.  

More BP

We have posted numerous times on the travails of BP following the Deepwater Horizon disaster and the contribution of safety culture to these performance results.  BP is back in the news since the trial date for a variety of suits and countersuits is coming up shortly.  We thought we would take the opportunity for a quick update.

The good news is the absence of any more significant events at BP facilities.  In its presentation to investors on 4Q11 and 2012 Strategy, BP highlighted its 10 point moving forward plan, including at the top of the list, “relentless focus on safety and managing risk”.* 

It is impossible for us to assess how substantive and effective this focus has been or will be, but we’ve now heard from BP’s Board member Frank Bowman.  Bowman is head of the Board’s Safety, Ethics and Environment Assurance Committee.  He served on the panel that investigated BP’s US refineries after the Texas City explosion in 2005 and then became a member of BP’s US advisory council; and in November 2010, he joined the main board as a non-executive director.  Basically Bowman’s mission is to help transfer his U.S. nuclear navy safety philosophy to BP’s energy business.

Bowman reports that he has been impressed by the way the safety and operational risk and upstream organizations have taken decisions to suspend operations when necessary. “We’ve recently walked away from several jobs where our standards were not being met by our partners or a contractor. That sends a message heard around the world, and we should continue to do that.”**

Looking for more specifics in the 4Q11 investor presentation, we came across the following “safety performance record”. (BP 4Q11, p. 12)

The charts plot “loss of containment” issues (these are basically releases of hydrocarbons) and personnel injury frequency.  The presentation notes that “Aside from the exceptional activities of the Deepwater Horizon response, steady progress has been made over the last decade.”  Perhaps but we are skeptical that these data are useful for measuring progress in the area of safety culture and management.  For one they both show positive trends over a time period where BP had two major disasters - the Texas City oil refinery fire in 2005 and Deepwater Horizon in 2010.  At a minimum these charts confirm that the tracked parameters do nothing to proactively predict safety health.  As Mr. Bowman notes, “Culture is set by the collective behaviour of an organisation’s leaders… The collective behaviour of BP’s leaders must consistently endorse safety as central to our very being.” (BP Magazine, p. 10)

On the subject of management behavior, the investigations and analyses of Deepwater Horizon consistently noted the contribution of business pressures and competing priorities that lead to poor decisions.  In our September 30, 2010 blog post we included a quote from the then-new BP CEO:

“Mr. Dudley said he also plans a review of how BP creates incentives for business performance, to find out how it can encourage staff to improve safety and risk management.”

The 4Q11 presentation and Mr. Bowman’s interview are noticeably silent on this subject.  The best we could come up with was the following rather cryptic statement in the 4Q11: “We’ve also evolved our approach to performance management and reward, requiring employees to set personal priorities for safety and risk management, focus more on the long term and working as one team.” (BP 4Q11, p. 15)  We’re not sure how “personal priorities” relate to the compensation incentives which were the real focus of the concerns expressed in the accident investigations.

Looking a bit further we uncovered the following in a statement by the chairwoman of BP’s Board Remuneration Committee: “For 2011 the overall policy for executive directors [compensation] will remain largely unchanged…”***  If you guessed that incentives would be based only on meeting business results, you would be right.

In closing we leave with one other comment from Mr. Bowman, one that we think has great salience in the instant situation of BP and for other high risk industries including nuclear generation: “In any business dealing with an unforgiving environment, complacency is your worst enemy. You have to be very careful about what conclusion to draw from the absence of an accident.” (BP Magazine, p. 9) [emphasis added]

*  BP 4Q11 & 2012 Strategy presentation, p. 8.

**  BP Magazine, Issue 4 2011, p. 9.

***  Letter from the chairman of the remuneration committee (Mar. 2, 2011).

Ultimate Bonuses

Just when you think there is a lack of humor in the exposition of dry, but critical issues, such as risk management, our old friend Nicholas Taleb comes to the rescue.*  His op-ed piece in the New York Times** earlier this week has a subdued title, “End Bonuses for Bankers”, but includes some real eye-openers.  For example Taleb cites (with hardly concealed admiration) the ancient Hammurabi code which protected home owners by calling for the death of the home builder if the home collapsed and killed the owner.  Wait, I thought we were talking about bonuses, not capital punishment.

What Taleb is concerned about is that bonus systems in entities that pose systemic risks almost universally encourage behaviors that may not be consistent with the public good much less the long term health of the business entity.  In short he believes that bonuses provide an incentive to take risks.***  He states, “The asymmetric nature of the bonus (an incentive for success without a corresponding disincentive for failure) causes hidden risks to accumulate in the financial system and become a catalyst for disaster.”  Now just substitute “nuclear operations” for “the financial system”. 

Central to Taleb’s thesis is his belief that management has a large informational advantage over outside regulators and will always know more about risks being taken within their operation.  It affords management the opportunity to both take on additional risk (say to meet an incentive plan goal) and to camouflage the latent risk from regulators.

In our prior posts [here, here and here] on management incentives within the nuclear industry, we also pointed to the asymmetry of bonus metrics - the focus on operating availability and costs, the lack of metrics for safety performance, and the lack of downside incentive for failure to meet safety goals.  The concern was amplified due to the increasing magnitude of nuclear executive bonuses, both in real terms and as a percentage of total compensation. 

So what to do?  Taleb’s answer for financial institutions too big to fail is “bonuses and bailouts should never mix”; in other words, “end bonuses for bankers”.  Our answer is, “bonuses and nuclear safety culture should never mix”; “end bonuses for nuclear executives”.  Instead, gross up the compensation of nuclear executives to include the nominal level of expected bonuses.  Then let them manage nuclear operations using their best judgment to assure safety, unencumbered by conflicting incentives.


*  Taleb is best known for The Black Swan, a book focusing on the need to develop strategies, esp. financial strategies, that are robust in the face of rare and hard-to-predict events.

**  N. Taleb, “End Bonuses for Bankers,” New York Times website (Nov. 7, 2011).

*** It is widely held that the 2008 financial crisis was exacerbated, if not caused, by executives making more risky decisions than shareholders would have thought appropriate. Alan Greenspan commented: “I made a mistake in presuming that the self-interests of organizations, specifically banks and others, were such that they were best capable of protecting their own shareholders” (Testimony to Congress, quoted in A. Clark and J. Treanor, “Greenspan - I was wrong about the economy. Sort of,” The Guardian, Oct. 23, 2008). The cause is widely thought to be the use of bonuses for performance combined with limited liability.  See also J.M. Malcomson, “Do Managers with Limited Liability Take More Risky Decisions? An Information Acquisition Model”, Journal of Economics & Management Strategy, Vol. 20, Issue 1 (Spring 2011), pp. 83–120.

Incredible

“...notwithstanding the tragic loss of life in the Gulf of Mexico, we [Transocean] achieved an exemplary statistical safety record as measured by our total recordable incident rate (‘‘TRIR’’) and total potential severity rate (‘‘TPSR’’).  As measured by these standards, we recorded the best year in safety performance in our Company’s history, which is a reflection on our commitment to achieving an incident free environment, all the time, everywhere.”*

Good grief.  Did Transocean really say this?  Eleven people including nine Transocean employees died in the Deepwater Horizon oil rig explosion.  The quote is from Transocean’s 2010 Annual Report and Proxy recently filed with the SEC.  It provides another illuminating example where the structure and award of management incentives speak much greater volumes than corporate safety rubrics.  (For our report on compensation structures within nuclear power companies and the extent to which such compensation included incentives other than safety, look here and here.)  Or as the saying goes, “Follow the money”.

To fully comprehend how Transocean’s incentive program purports to encourage safety performance we are providing the following additional quotes from its Annual Report.

“Safety Performance.  Our business involves numerous operating hazards and we remain committed to protecting our employees, our property and the environment. Our ultimate goal is expressed in our Safety Vision of ‘‘an incident-free workplace—all the time, everywhere…..

"The [Compensation] Committee measures our safety performance through a combination of our total recordable incident rate (‘‘TRIR’’) and total potential severity rate (‘‘TPSR’’).

•    "TRIR is an industry standard measure of safety performance that is used to measure the frequency of a company’s recordable incidents and comprised 50% of the overall safety metric. TRIR is measured in number of recordable incidents per 200,000 employee hours worked.

•    "TPSR is a proprietary safety measure that we use to monitor the total potential severity of incidents and comprised 50% of the overall safety metric. Each incident is reviewed and assigned a number based on the impact that such incident could have had on our employees and contractors, and the total is then combined to determine the TPSR.

"The occurrence of a fatality may override the safety performance measure.

"….Based on the foregoing safety performance measures, the actual TRIR was 0.74 and the TPSR was 35.4 for 2010. These outcomes together resulted in a calculated payout percentage of 115% for the safety performance measure for 2010. However, due to the fatalities that occurred in 2010, the Committee exercised its discretionary authority to modify the TRIR payout component to zero, which resulted in a modified payout percentage of 67.4% for the safety performance measure." (p. 45)

The treatment of bonuses for Transocean execs was picked up in various media outlets and met with, shall we say, skepticism.  Transocean responded to the blowback with the following:

“We acknowledge that some of the wording in our 2010 proxy statement may have been insensitive in light of the incident that claimed the lives of eleven exceptional men last year and we deeply regret any pain that it may have caused...” **

Note that the apology is directed at the “wording” of the proxy, not to the actual award of bonus compensation for safety performance.  We are tempted here to make some reference to “density” but it is self-evident.

Perhaps realizing that something more would be appropriate, Transocean announced yesterday that members of the senior management team would be donating their bonuses to the Deepwater Horizon Memorial Fund.*** 

Oops, actually they will be donating just the “safety portion” of their bonuses to the fund.  All other bonus amounts and incentive awards are not affected and the Transocean incentive structure for safety performance remains unchanged for 2011.

*  Transocean Proxy Statement and 2010 Annual Report, p. 35.

**  J. McMahon, "Transocean Execs Get Bonuses after ‘Best Year in Safety,’ Despite Gulf Oil Disaster," Forbes.com (Apr 2, 2011 2:23am).

***  Announcement by Transocean Ltd. Senior Management Team, Zug, Switzerland (Apr 5, 2011 MARKETWIRE via COMTEX).

Survival of the Safest

One of our goals with SafetyMatters is bringing thought provoking materials to our readers, particularly materials they might not otherwise come across.  This post is an example from the greater business world and the current state of the U.S. economy.  Once again it is based on some interesting research from professors at Yale University* and described in an article in the New York Times.**

“Corporate managers struggling to preserve their companies and protect their core employees have inadvertently contributed to a vicious cycle of rising unemployment and plummeting national morale. If we are to break out of this downward spiral, we first need to understand the problem…professional managers throughout the business world see it as their job to keep work-force morale high. But, paradoxically, the actions they take for their own workplaces often make the overall crisis more severe.”

These issues have been the subject of research by Yale economics professor Truman Bewley.  While his specific focus is on labor markets and how wages respond (or don’t respond) to periods of reduced demand, some of the insights channel directly into the current issues of safety culture at nuclear plants. 

Bewley’s approach was to interview hundreds of corporate managers at length about the driving forces for their actions.  The article goes on to describe how corporate managers respond to recessions by protecting their most important staff, but paradoxically these actions tend to produce unforeseen and often counter-productive results. 

The description of how actions result in unintended consequences is emblematic of the complexity of business systems, where dynamics and interdependencies are not always seen or understood by the managers tasked with achieving results.  Nuclear safety culture exists in such a complex socio-technical system and requires more than just “leadership” to assure long term sustainability. 

This brings us to the first part of Dr. Bewley’s approach - his focus on identifying and understanding the driving forces for managers’ actions.  We see this as precisely the right prescription for improving our understanding of nuclear safety culture dynamics, particularly in cases where safety culture weaknesses have been observed.  A careful and penetrating look at why people don’t act in accordance with safety culture principles would do much to identify the types of factors, such as performance incentives, cost and schedule pressures, etc. that may be at work in an organization.  Driving forces are not necessarily different from root causes - a term more familiar in the nuclear industry - but I tend to prefer it because it explicitly reminds us that safety culture is dynamic, and results from the interaction of many moving parts.  Currently the focus of the industry, and the NRC for that matter, is on safety culture “traits”.  Traits are really the results or manifestations of safety culture and thus build out the picture of what is desired.  But they do not get at what factors actually produce strong safety culture in the first place.

As an example we refer you to a comment we posted on a Nuclear Safety Culture group thread on LinkedIn.com.  Dr. Bill Corcoran initiated a thread asking for proposals of safety culture traits that were at least as important as those in the NRC strawman.  Our response proposed:

 “The compensation structure in the corporation is aligned with its safety priorities and does not create real or perceived conflicts in decisions affecting nuclear safety.” ***

While this was proposed as a “trait” in response to Bill’s request, it is clearly a driving force that will enable and support strong safety culture behaviors and decisions.

* To read about other interesting work at Yale, check out our August 30, 2010 post.

** Robert J. Shiller, "The Survival of the Safest," New York Times (Oct 2, 2010).

*** The link to the thread (including Bob's comment) is here.  This may be difficult for readers who are not LinkedIn members to access.  We are not promoting LinkedIn but the Nuclear Safety Culture group has some interesting commentary.

NRC Decision on FPL (Part 2)

NRR Director’s decision addresses FPL Retention Bonus Agreement and ECP.

As described in our previous post, a decision by the NRC’s Director of NRR (DD-10-01) was released last week addressing certain contentions raised in a petition to the NRC Chairman by a former FPL nuclear employee.*  In this post we address several aspects of the second issue raised by petitioner regarding the efficacy of the Employee Concerns Program (ECP).

In brief the petitioner “raised issues related to weaknesses in the ECP as a means of getting issues entered into the CAP and "chilling effects" that exist at Turkey Point and are spreading to St. Lucie where employees are dissuaded from freely raising nuclear safety concerns to the NRC or within FPL for fear of retaliation by FPL management.”  [DD-10-01]  As described in the Director’s Decision, the petitions overlap a fairly prolonged history of weaknesses in the CAP at FPL and regulatory and licensee actions to improve the program.  For example, the NRC held a public meeting on October 20, 2009 (ADAMS Accession No. ML093090274), to discuss FPL's processes for addressing employee concerns and planned, fleet-wide corrective actions for addressing FPL-identified weaknesses. The licensee indicated that it planned to implement 86 corrective actions to address the weaknesses.

In this post we are taking a selective focus on a few important findings emanating from the comprehensive actions undertaken by FPL.  The first relates to a root cause analysis performed by FPL in 2009 and reported at the October 2009 meeting with the NRC.  Two root causes were identified, the second being:

Root Cause 2 (RC 2):

“Certain management actions have negatively impacted employee trust and resulted in a perception that production often takes precedence over safety”  [Slide 18 of FPL presentation]

Associated with RC 2 were a number of Contributing Causes including:  

Contributing Cause 2.2 (CC 2.2):     

“Many operational and other management decisions have been perceived to place emphasis on production at the expense of safety” [Slide 19]  

Contributing Cause 2.4 (CC 2.4):     

“The reward and recognition system is perceived to be heavily weighted toward production over safety” [Slide 21]  

We found the results of the root cause analysis to be quite interesting, remarkable even.  The RC went right to the issue of competing priorities, production taking precedence over safety, which we have argued many times may be the most important threat to safety culture.  The root cause also correctly connects management actions, and the perception of those actions, as the linchpin of employee trust.  Finally, CC 2.4 also explicitly identifies the reward and recognition system (read: compensation and promotion) as a potential conflicting influence on management decision making.  We applaud this effort and the results it produced.  

Corrective actions were identified for each contributing cause.  For CC 2.2 the approach was to implement a decision-making process that identifies risk criteria and communication requirements.  It also establishes a DPO process.  In our view this is an important and potentially highly useful approach, if implemented rigorously.  We have argued that many of the decisions nuclear managers make are very nuanced and fuzzy in terms of safety implications.  Reaching the appropriate decisions in an environment where other business priorities are also present creates the opportunity for the misperception of those decisions or decisions that do not adequately reflect safety priorities.  Providing for a defined and open process, including explanation of how safety risks are considered, should improve decision making as well as the perception of those decisions by the organization.

With regard to CC 2.4, the corrective action was to evaluate the current reward system and “ensure measures of NSC [nuclear safety culture] are considered”. [Slide 21]  This action is hard to assess based only on the available documentation.  As we have described in recent posts on nuclear compensation, we see potentially significant conflicts in current compensation structures given the amounts of compensation at risk for performance and the metrics used to trigger those incentives.  

FPL reported on progress in addressing its 86 corrective actions in a meeting with the NRC on April 20, 2010 [ADAMS Accession No. ML101110727].  Based on the FPL slides it is difficult to see exactly how some of the new processes are designed and whether they are effective.  With regard to the reward system, it is simply stated, “Revised current reward system to ensure consistent health of NSC” [Slide 28].   

We would have hoped that FPL would provide their analysis of specifically what aspects of the compensation system were found to be problematic with regard to safety culture and how the system was re-structured to mitigate the problems.  It is more than likely that similar issues could exist in the compensation systems at other nuclear operators and the approach taken by FPL could be highly useful.  We would welcome a post from FPL on this blog to provide such clarification.

*ADAMS Accession Number ML101790315

Nuclear Management Compensation (Part 2)

In Part 1 we provided the results of our survey of nuclear executive compensation levels, structure and performance metrics.  In this post we try to ascertain what impact, if any, these factors could have on nuclear safety culture and safety management.

To us there were several fairly striking observations from the data.  One, the amounts of compensation are very significant for the top nuclear executives.  Two, the compensation is heavily dependent on each year’s performance.  And three, business performance measured by EPS is the key to compensation, safety performance is a minor contributor.  A corollary to the third point might be that in no cases that we could identify was safety performance a condition precedent or qualification for earning the business-based incentives.

Some specifics.  Generally the compensation programs state that they are designed, in part, to reward safety and other stewardship goals.  Sometimes the term “operational excellence” is used and can include “safety” but it is difficult to know what this encompasses as SEC filings include various levels of specificity.   In other cases the term “culture” is used but in general the context is not nuclear safety culture.*   Some incentive plans state they are designed to prevent “promoting excessive risk” or unnecessary risks.  These plans did not discuss any specific penalties or reductions in incentive linked to risk so it is difficult to judge how they are intended to accomplish the goal.  It may be that using company stock for significant parts of the incentive payout is intended to ensure that executives have an interest in protecting their ownership stake.  By far the most common metric in the incentive programs associated with safety is industrial safety accident rate and/or fatalities.  This is understandable in that across the enterprise of a power generation company, industrial safety is a common goal and can be measured on a common basis.  Generally, fatal accidents result in a safety penalty in the incentive program, and the penalties could reduce the total incentive by about 5-10%.  The largest safety impact on incentive that I came across was a 10% factor for OSHA safety and 10% for nuclear safety culture.

One caveat is that the incentives and executives described in SEC filings necessarily represent the very top level of the enterprise.  The specifics of incentive programs for lower tier executives, including those within the nuclear operating organization, are not available.  However, it seems a safe assumption that the design of the incentive program is carried down the organization to at least the VP level, including significant performance incentives, but with metrics weighted more to business unit performance. 

What is the significance of these compensation programs to nuclear safety management, if any?  First, with regard to total compensation, the high levels recently achieved could be viewed as a positive in assuring that the highest quality management is attracted and retained in nuclear operations.  The responsibilities of nuclear executives are unusually large, complex and unremitting.  While the totals are eye-popping, viewed in the context of the compensation for other executives in the enterprise, there is only parity. 

With regard to the structure of compensation - the amounts that are based on performance and the metrics used to define goals - there may cause for greater concern.  As shown in Part 1, with 60-70% of total compensation at risk, executives can see their compensation, and that of the entire management team, impacted by as much as several million dollars in a year.  And almost all of that incentive is based on business goals such as EPS, cash flow, budgets, and plant performance.

As we have commented many times in this blog, we view achievement of nuclear safety culture as a process of successfully balancing safety goals with other competing business priorities.  That context is unavoidable.  Simplistic statements that “safety is our highest priority” do not reflect the reality of day-to-day decision making, or insulate decision makers.  The balance is complicated and difficult since decisions are rarely black and white.  And we have seen time and again, bad outcomes can evolve from a series of decisions and judgments that tilt the wrong way, apparent only in retrospective.  When executive incentive programs weight business goals 90% and some aspect of safety 10%, what is the message?  How does it not amplify the pressure on the executives and possibly skew the balancing act?  How does it “prevent promoting excessive risk” as some compensation plans state? 

In Part 3 we will discuss whether executive compensation should be an element of nuclear safety culture and what might be done to ensure that it reinforces the commitment to safety.

*  In one case “People, Culture, Reputation” as a goal is expanded to mean: employee performance appraisals, executive-level employee diversity, ethics and compliance training, corporate values, and regulatory compliance.  It counts for 10% of the total incentive.

Nuclear Management Compensation (Part 1)

We have posted previously on the general topic of competing priorities (cost, schedule, political, personal career interest, etc.) being a potential issue for nuclear managers to balance in making safety decisions.  We have also recently posted regarding the possible influence that cost and schedule pressures may have had on BP personnel in the events leading up to the spill and in several prior BP accidents.  Lastly we highlighted some research results that suggest that compensation incentives may have perverse impacts on desired results.  In this post we turn directly to the issue of compensation for executives with corporate responsibilities for nuclear plants and their operations.  In subsequent posts we will discuss the potential implications of compensation on nuclear safety management.

We researched compensation data that was available in SEC filings of public corporations owning nuclear facilities.  As a practical matter this is the only publicly available information on this subject.  In their annual prospectus, corporations disclose data for the top five compensated executives - the CEO and four other “NEOs” (Named Executive Officers).  In some cases the executive who is the Chief Nuclear Officer is an NEO.  In other cases a higher level executive, typically head of the generation business unit or other operating officer, is the NEO with direct, attributable responsibility for the nuclear facilities.  To obtain an overview of compensation practices we looked at CNOs when available and, if not, the responsible NEOs .  We compiled data for compensation levels, the structure of compensation, and the performance metrics being applied by the corporation. 

Total Compensation

For NEOs with direct nuclear responsibilities, total average annual compensation is now $2.5-3 million (the range for our sample was approximately $1.25 to $5 million).  We counted salary, annual bonus, non-equity incentive awards and equity incentive awards.  We did not include changes in retirement benefit accruals (which can be quite large as well) or other miscellaneous fringe benefits. 

In virtually all cases the level of compensation has shown a significant increase in the last 3-4 years.  This may reflect increased competition for nuclear executive talent rippling through the industry and/or the increased business value associated with nuclear generation assets.  The figure below illustrates how compensation has changed over recent years in two nuclear organizations, one operating a single unit and the other a small fleet.

Compensation Structure

We also examined the structure of executive compensation with an emphasis on the amounts of compensation that are fixed on an annual basis (typically salary and benefits) and those amounts that are “at risk”, meaning they are performance-based.  We found that compensation is heavily incentive based where the amount of total compensation that is earned based on performance now averages greater than 70% (the range for our sample was 60% - 80%).  As with total compensation, the percent of compensation at risk has increased significantly in the last 3-4 years.  The next figure shows the trend lines for the same two corporations as in the first figure.

These results are not an accident as the SEC filings for these corporations set out compensation policies that are heavily weighted toward performance.  For the most part incentives are based on the current year performance though for some specific incentives such as stock options, rolling three averages may be used.  The principal objective appears to be to align executive pay with shareholder interests.

Performance Metrics

Typically the compensation structure uses a combination of corporate level performance and business unit performance to determine incentive payouts.  At the highest level of the organization, the CEO, metrics may be exclusively corporate level and almost always focused on earnings per share (EPS).  Lower tier executives generally have a significant portion (60-70%) of their incentive determined by corporate performance (almost always EPS) with the remainder determined by business unit or personal goals.  The metrics for nuclear business units are typically capacity factor and budget based, occasionally using an index of 8-10 metrics.  This performance will generally be associated with about 20% of the total incentive.  The remaining 10% of incentive is usually associated with some type of safety performance.  OSHA industrial safety measures are most common though we identified one corporation that used a “nuclear safety culture” metric.

In Part 2 we will try to ascertain what impact these factors could have on nuclear safety culture and safety management.

Safety, Cost and Bonuses

An article in the June 29, 2010 Wall Street Journal, “As CEO Hayward Remade BP, Safety, Cost Drives Clashed”, fills in some real world examples of the dynamics of conflicting goals within BP and presumably contributing to the current spill disaster.  The presence of compensation incentives may have played a role in how safety vs. cost decisions were being made.  We will be focusing more on this issue in upcoming posts.

Can Safety Culture Be Regulated? (Part 2)

Part 1 of this topic covered the factors important to safety culture and amenable to measurement or assessment, the “known knowns.”   In this Part 2 we’ll review other factors we believe are important to safety culture but cannot be assessed very well, if at all, the “known unknowns” and the potential for factors or relationships important to safety culture that we don’t know about, the “unknown unknowns.”

Known Unknowns

These are factors that are probably important to regulating safety culture but cannot be assessed or cannot be assessed very well.  The hazard they pose is that deficient or declining performance may, over time, damage and degrade a previously adequate safety culture.

Measuring Safety Culture

This is the largest issue facing a regulator.  There is no meter or method that can be applied to an organization to obtain the value of some safety culture metric.  It’s challenging (impossible?) to robustly and validly assess, much less regulate, a variable that cannot be measured.  For a more complete discussion of this issue, please see our June 15, 2010 post. 

Trust

If the plant staff does not trust management to do the right thing, even when it costs significant resources, then safety culture will be negatively affected.  How does one measure trust, with a survey?  I don’t think surveys offer more than an instantaneous estimate of any trust metric’s value.

Complacency

Organizations that accept things as they are, or always have been, and see no opportunity or need for improvement are guilty of complacency or worse, hubris.  Lack of organizational reinforcement for a questioning attitude, especially when the questions may result in lost production or financial costs, is a de facto endorsement of complacency.  Complacency is often easy to see a posteriori, hard to detect as it occurs.  

Management competence

Does management implement and maintain consistent and effective management policies and processes?  Is the potential for goal conflict recognized and dealt with (i.e., are priorities set) in a transparent and widely accepted manner?  Organizations may get opinions on their managers’ competence, but not from the regulator.

The NRC does not evaluate plant or owner management competence.  They used to, or at least appeared to be trying to.  Remember the NRC senior management meetings, trending letters, and the Watch List?  While all the “problem” plants had material or work process issues, I believe a contributing factor was the regulator had lost confidence in the competence of plant management.  This system led to the epidemic of shutdown plants in the 1990s.*   In reaction, politicians became concerned over the financial losses to plant owners and employees, and the Commission become concerned that the staff’s explicit/implicit management evaluation process was neither robust and nor valid.

So the NRC replaced a data-informed subjective process with the Reactor Oversight Program (ROP) which looks at a set of “objective” performance indicators and a more subjective inference of cross-cutting issues: human performance, finding and fixing problems (CAP, a known), and management attention to safety and workers' ability to raise safety issues (SCWE, part known and part unknown).  I don’t believe that anyone, especially an outsider like a regulator, can get a reasonable picture of a plant’s safety culture from the “Rope.”  There most certainly are no leading or predictive safety performance indicators in this system.

External influences

These factors include changes in plant ownership, financial health of the owner, environmental regulations, employee perceptions about management’s “real” priorities, third-party assessments, local socio-political pressures and the like.  Any change in these factors could have some effect on safety culture.

Unknown Unknowns

These are the factors that affect safety culture but we don’t know about.  While a lot of smart people have invested significant time and effort in identifying factors that influence safety culture, new possibilities can still emerge.

For example, a new factor has just appeared on our radar screen: executive compensation.  Bob Cudlin has been researching the compensation packages for senior nuclear executives and some of the numbers are eye-popping, especially in comparison to historical utility norms.  Bob will soon post on his findings, including where safety figures into the compensation schemes, an important consideration since much executive compensation is incentive-based.

In addition, it could well be that there are interactions (feedback loops and the like), perhaps varying in structure and intensity over time, between and among the known and unknown factors, that have varying impacts on the evolutionary arc of an organization’s safety culture.  Because of such factors, our hope that safety culture is essentially stable, with a relatively long decay time, may be false; safety culture may be susceptible to sudden drop-offs. 

The Bottom Line

Can safety culture be regulated?  At the current state of knowledge, with some “known knowns” but no standard approach to measuring safety culture and no leading safety performance indicators, we’d have to say “Yes, but only to some degree.”  The regulator may claim to have a handle on an organization’s safety culture through SCWE observations and indirect evidence, but we don’t think the regulator is in a good position to predict or even anticipate the next issue or incident related to safety culture in the nuclear industry. 

* In the U.S. in 1997, one couldn’t swing a dead cat without hitting a shutdown nuclear power plant.  17 units were shutdown during all or part of that year, out of a total population of 108 units. 

When Money Motivates

A website that has caught our attention is called “Nudge” whose focus is on improving decisions and “choice architecture”. A June 1, 2010 post titled “When Money Motivates Employees and When It Doesn’t” includes some material that bears on issues of safety culture and safety management. The post is actually a video animated presentation and runs about 11 minutes.

As is common in the business world, and increasingly in the nuclear generation business, incentives are part of many employees’ compensation packages, and in the case of senior management can be quite substantial. Incentives can be cash bonuses, stock participation awards, or similar. Incentives are tied to the achievement of certain specified performance objectives that can target broad corporate level metrics as well as more specific ones associated with a manager’s responsibilities, e.g., plant capacity factor and budget. Safety performance metrics may or may not be explicitly part of the incentive program; safety may be viewed as an absolute performance requirement and sometimes as a condition precedent to access other performance incentives.

So, as the Nudge post asks, do money incentives provide an appropriate motivation for employees? Their short answer is only under certain limited circumstances. If an employee’s responsibilities are relatively simple and straightforward, involving mechanical skills or rudimentary cognitive skills, more reward leads to more performance. On the other hand if complex or sophisticated cognitive skills or creative thinking is required, then not only is the direct connection lost, the reverse may even be true - the incentive results in poorer performance.

Now this is social “science” theory put forward by a team of economists and sociologists and appears to be a “neat” formulation. But it is a little hard to judge the validity of the findings as only minimal information regarding the studies is provided. The authors claim that many, many similar studies come to similar conclusions, so the body of research may in fact be persuasive. I would have to say that my own experience is not necessarily consistent with these findings. Incentives can be a powerful driver of results - albeit often limited to the specific results targeted by the incentives - leading to unintended consequences in other performance areas that are not targeted or that may get sacrificed in the pursuit of the targeted goals.

This leads to what I found to be the more interesting observation from the “Money Motivates” post: the researchers believe the best approach is to pay people enough to effectively “take money off the table”, allowing them to balance all relevant job priorities. The researchers concluded that people are basically “purpose maximizers” meaning that we are motivated to achieve the overarching goals and purpose of our jobs. Problems arise when purpose and profit motives are not aligned or profit is paramount. Where safety is a vital component of purpose, it is possible to see where incentives can lead to compromises.

What is interesting to us is the intersection of incentives and the related issue of competing priorities and pressures on nuclear managers when balancing safety and other business objectives. Incentives are really just a different form of pressure, individualized to personal success and gain. Are there implications for nuclear safety management? How common are incentives for nuclear managers and what specific performance goals are targeted? How does safety performance factor in and is there the potential for safety and incentives not to be aligned? Has there been any assessment of the impact of incentives in cases where safety culture problems have occurred?