A Factory for Producing Decisions

The subject of this post is the compelling insights of Daniel Kahneman into issues of behavioral economics and how we think and make decisions.  Kahneman is one of the most influential thinkers of our time and a Nobel laureate.  Two links are provided for our readers who would like additional information.  One is via the McKinsey Quarterly, a video interview* done several years ago.  It runs about 17 minutes.  The second is a current review in The Atlantic** of Kahneman’s just released book, Thinking Fast and Slow.

Kahneman begins the McKinsey interview by suggesting that we think of organizations as “factories for producing decisions” and therefore, think of decisions as a product.  This seems to make a lot of sense when applied to nuclear operating organizations - they are the veritable “River Rouge” of decision factories.  What may be unusual for nuclear organizations is the large percentage of decisions that directly or indirectly include safety dimensions, dimensions that can be uncertain and/or significantly judgmental, and which often conflict with other business goals.  So nuclear organizations have to deliver two products: competitively priced megawatts and decisions that preserve adequate safety.

To Kahneman decisions as product logically raises the issue of quality control as a means to ensure the quality of decisions.  At one level quality control might focus on mistakes and ensuring that decisions avoid recurrence of mistakes.  But Kahneman sees the quality function going further into the psychology of the decision process to ensure, e.g., that the best information is available to decision makers, that the talents of the group surrounding the ultimate decision maker are being used effectively, and the presence of an unbiased decision-making environment.

He notes that there is an enormous amount of resistance within organizations to improving decision processes. People naturally feel threatened if their decisions are questioned or second guessed.  So it may be very difficult or even impossible to improve the quality of decisions if the leadership is threatened too much.  But, are there ways to avoid this?  Kahneman suggests the “premortem” (think of it as the analog to a post mortem).  When a decision is being formulated (not yet made), convene a group meeting with the following premise: It is a year from now, we have implemented the decision under consideration, it has been a complete disaster.  Have each individual write down “what happened?”

The objective of the premortem is to legitimize dissent and minimize the innate “bias toward optimism” in decision analysis.  It is based on the observation that as organizations converge toward a decision, dissent becomes progressively more difficult and costly and people who warn or dissent can be viewed as disloyal.  The premortem essentially sets up a competitive situation to see who can come up with the flaw in the plan.  In essence everyone takes on the role of dissenter.  Kahneman’s belief is that the process will yield some new insights - that may not change the decision but will lead to adjustments to make the decision more robust. 

Kahneman’s ideas about decisions resonate with our thinking that the most useful focus for nuclear safety culture is the quality of organizational decisions.  It also contrasts with a recent instance of a nuclear plant run afoul of the NRC (Browns Ferry) and now tagged with a degraded cornerstone and increased inspections.  As usual in the nuclear industry, TVA has called on an outside contractor to come in and perform a safety culture survey, to “... find out if people feel empowered to raise safety concerns….”***  It may be interesting to see how people feel, but we believe it would be far more powerful and useful to analyze a significant sample of recent organizational decisions to determine if the decisions reflect an appropriate level of concern for safety.  Feelings (perceptions) are not a substitute for what is actually occurring in the decision process. 

We have been working to develop ways to grade whether decisions support strong safety culture, including offering opportunities on this blog for readers to “score” actual plant decisions.  In addition we have highlighted the work of Constance Perin including her book, Shouldering Risks, which reveals the value of dissecting decision mechanics.  Perin’s observations about group and individual status and credibility and their implications for dissent and information sharing directly parallel Kahneman’s focus on the need to legitimize dissent.  We hope some of this thinking ultimately overcomes the current bias in nuclear organizations to reflexively turn to surveys and the inevitable retraining in safety culture principles.

*  "Daniel Kahneman on behavioral economics," McKinsey Quarterly video interview (May 2008).

** M. Popova, "The Anti-Gladwell: Kahneman's New Way to Think About Thinking," The Atlantic website (Nov. 1, 2011).

*** A. Smith, "Nuke plant inspections proceeding as planned," Athens [Ala.] News Courier website (Nov. 2, 2011).

Vermont Yankee (part 5) - Muddy Water

In our April 5, 2010 post re Vermont Yankee we provided some initial thoughts on the report of the independent investigator regarding misleading statements provided by Entergy personnel to Vermont regulators, as contained in Entergy’s March 31, 2010 response to a March 1, 2010 NRC Demand for Information.* The Entergy filing also provides more detail on follow-up actions including an assessment of current site safety culture. In this post, we offer some additional observations and questions.
First, in our initial March 3, 2010 post regarding the VY situation, we disputed a prediction made by a third party that the administrative actions taken by Entergy for certain employees might have a detrimental effect on the safety culture at the plant - due to the way Entergy is treating its employees. In reality it appeared to us that any detrimental impact on safety culture would be more likely if Entergy had not taken appropriate actions. In Entergy’s report to the NRC, they provide the results of a follow-up assessment confirming that after the personnel actions employees were even more likely to raise concerns.

Also in our initial post we speculated that the Vermont Yankee events could have consequences for Entergy’s proposed spinout of six nuclear plants into a separate subsidiary. Since then Entergy has announced the cancellation of the spinout after a decision by New York re the extension of permits for their Indian Point plants.

However, after a careful review of the March 31 Entergy response, we are still left with water that is more than a little bit muddy. Entergy says a Synergy assessment a few months before the reporting event found safety culture at Vermont Yankee to be strong. After the event, Entergy states safety culture is strong or stronger, and with regard to the replaced staff, Entergy “continues to have confidence in the integrity of the affected employees.” Strong safety culture and organizational integrity are not supposed to add up to this kind of outcome. How then did things go wrong? How did the misleading statements to Vermont regulators come about and what was the cause?

A fundamental element of all nuclear plant problem resolution/corrective action programs is a determination of not just what happened, but why. Cause in other words, and in significant situations, the root cause. The root cause that led to eleven employees, including managers and site executives, being relieved of duties and disciplined is not contained in the Entergy materials. In fact, most of the focus appears to be on the safety culture of the plant staff both before and after the incident came to light and the personnel actions taken. Those actions and information appear to be reassuring in regards to the plant staff - but the plant staff was not where the problem occurred. There is also considerable emphasis on the fact that the managers have been replaced with competent substitutes. But haven’t those new managers been placed in exactly the same situation as the former managers were in? If it is not clear why the former managers failed to meet performance standards, then how is one confident that the replacements will do so?

As we have pointed out in other posts, the response to safety culture failures too often stresses the “values and beliefs” of personnel as the beginning and end of safety culture. We have argued that the situational parameters, including competing goals and interests, are at least as important if not paramount in trying to understand such issues.

What was the situation at Vermont Yankee and to what extent, if any, did it have an effect? The VY management team was operating in an environment where significant business decisions were in play. One was the extension of the operating license for VY which required approval by both the NRC and by the Vermont Senate. A second was the pending proposed spinout of nuclear units, including VY, into a separate subsidiary, a spinout that was expected to be worth billions of dollars to Entergy. SEC and other regulatory filings had been made by Entergy for the spinout and approvals were being sought from state regulators and the NRC.

Entergy’s March 31 NRC submittal also states, “Finally, neither the underlying report of investigation which led to the discipline, nor the interviews of the AFEs, identified any credible evidence to suggest that any weakness in the work environment or site safety culture contributed to a reluctance by anyone to provide clarifying or supplemental information to the relevant state officials. Indeed, there is no credible evidence that any of the AFEs are -- or were -- reluctant to report safety concerns or any other matter of potential regulatory significance or legal non-compliance.”

Does this mean that situational factors such as business priorities were evaluated and found not to be a contributor? If so, how was this done and what is the basis for such a conclusion? Or were such competing priorities acknowledged as potential influences and able to be dealt with as part of the management system? What other situational factors might have been present and to what effect?

On April 9, 2010 the NRC notified Entergy that the NRC was going to “initiate a detailed review and independent assessment of the acceptability of Entergy’s conclusions. . . .” ** We will keep you posted.

*ADAMS Accession Number ML100910420

**ADAMS Accession Number ML100990409