Friday, September 27, 2013

Four Years of Safetymatters

Aztec Calendar
Over the four plus years we have been publishing this blog, regular readers will have noticed some recurring themes in our posts.  The purpose of this post is to summarize our perspective on these key themes.  We have attempted to build a body of work that is useful and insightful for you.

Systems View

We have consistently considered safety culture (SC) in the nuclear industry to be one component of a complicated socio-technical system.  A systems view provides a powerful mental model for analyzing and understanding organizational behavior. 

Our design and explicative efforts began with system dynamics as described by authors such as Peter Senge, focusing on characteristics such as feedback loops and time delays that can affect system behavior and lead to unexpected, non-linear changes in system performance.  Later, we expanded our discussion to incorporate the ways systems adapt and evolve over time in response to internal and external pressures.  Because they evolve, socio-technical organizations are learning organizations but continuous improvement is not guaranteed; in fact, evolution in response to pressure can lead to poorer performance.

The systems view, system dynamics and their application through computer simulation techniques are incorporated in the NuclearSafetySim management training tool.

Decision Making

A critical, defining activity of any organization is decision making.  Decision making determines what will (or will not) be done, by whom, and with what priority and resources.  Decision making is  directed and constrained by factors including laws, regulations, policies, goals, procedures and resource availability.  In addition, decision making is imbued with and reflective of the organization's values, mental models and aspirations, i.e., its culture, including safety culture.

Decision making is intimately related to an organization's financial compensation and incentive program.  We've commented on these programs in nuclear and non-nuclear organizations and identified the performance goals for which executives received the largest rewards; often, these were not safety goals.

Decision making is part of the behavior exhibited by senior managers.  We expect leaders to model desired behavior and are disappointed when they don't.  We have provided examples of good and bad decisions and leader behavior. 

Safety Culture Assessment


We have cited NRC Commissioner Apostolakis' observation that “we really care about what people do and maybe not why they do it . . .”  We sympathize with that view.  If organizations are making correct decisions and getting acceptable performance, the “why” is not immediately important.  However, in the longer run, trying to identify the why is essential, both to preserve organizational effectiveness and to provide a management (and mental) model that can be transported elsewhere in a fleet or industry.

What is not useful, and possibly even a disservice, is a feckless organizational SC “analysis” that focuses on a laundry list of attributes or limits remedial actions to retraining, closer oversight and selective punishment.  Such approaches ignore systemic factors and cannot provide long-term successful solutions.

We have always been skeptical of the value of SC surveys.  Over time, we saw that others shared our view.  Currently, broad-scope, in-depth interviews and focus groups are recognized as preferred ways to attempt to gauge an organization's SC and we generally support such approaches.

On a related topic, we were skeptical of the NRC's SC initiatives, which culminated in the SC Policy Statement.  As we have seen, this “policy” has led to back door de facto regulation of SC.

References and Examples

We've identified a library of references related to SC.  We review the work of leading organizational thinkers, social scientists and management writers, attempt to accurately summarize their work and add value by relating it to our views on SC.  We've reported on the contributions of Dekker, Dörner, Hollnagel, Kahneman, Perin, Perrow, Reason, Schein, Taleb, Vaughan, Weick and others.

We've also posted on the travails of organizations that dug themselves into holes that brought their SC into question.  Some of these were relatively small potatoes, e.g., Vermont Yankee and EdF, but others were actual disasters, e.g., Massey Energy and BP.  We've also covered DOE, especially the Hanford Waste Treatment and Immobilization Plant (aka the Vit plant).

Conclusion

We believe the nuclear industry is generally well-managed by well-intentioned personnel but can be affected by the natural organizational ailments of complacency, normalization of deviation, drift, hubris, incompetence and occasional criminality.  Our perspective has evolved as we have learned more about organizations in general and SC in particular.  Channeling John Maynard Keynes, we adapt our models when we become aware of new facts or better ways of looking at the data.  We hope you continue to follow Safetymatters.  

Tuesday, September 24, 2013

Safety Paradigm Shift

We came across a provocative and persuasive presentation by Jean Pariès Dédale, "Why a Paradigm Shift Is Needed" from the IAEA Experts Meeting in May of this year.*  Many of the points resonate with our views on nuclear safety management; in particular complexity, the fallacy of the "predetermination envelope"- making a system more reliable within its design envelope but more susceptible outside that envelope; deterministic and probabilistic rationalization that avoids dealing with complexity of the system; and unknown-unknowns.  We also believe it will take a paradigm shift, however unlikely it may be at least in the U.S. nuclear industry.  Interestingly, Dédale does not appear to have a nuclear power background and develops his paradigm argument across multiple events and industries.

Dédale poses a very fundamental question: since the current safety construct has shown vulnerabilities to actual off-normal events should the response be, do more of the same but better and with more rigor? Or should the safety paradigm itself be challenged?  The key issue underlying the challenge to this construct is how to cope with complexity.  He means complexity in the same terms we have posted about numerous times.

Dédale notes “The uncertainty generated by the complexity of the system itself and by its environment is skirted through deterministic or probabilistic rationality.” (p. 8)  We agree.  Any review of condition reports and Tech Spec variances indicates a wholesale reliance on risk based rationale for deviations from nominal requirements.  And the risk based argument is almost always based on an estimated small probability of an event that would challenge safety, often enhanced by a relatively short exposure time frame.  As we highlighted in a prior post, Nick Taleb has long cautioned against making decisions based on assessments of probabilities, which he asserts we cannot know, versus consequences which are (sometimes uncomfortably) knowable.

How does this relate to safety management issues including culture?

We see a parallel between constructs for nuclear safety and safety culture.  The nuclear safety construct is constrained both in focus and evolution, heavily reliant on the design basis philosophy (what Dédale labels “predetermination fallacy”) dating back to the 1960s.  Little has changed over the succeeding 50 years; even the advent of PRA has been limited to “informing” the implementation of this approach.  Safety culture has emerged over the last 10+ years as an added regulatory emphasis though highly constrained in its manifestation as a policy statement.  (It is in fact still quite difficult to square the NRC’s characterization of safety culture as critical to safety** yet stopping way short of any regulation or requirements.)  The definitional scope of safety culture is expressed in a set of traits and related values and behaviors.  As with nuclear safety it has a limited scope and relies on abstractions emphasizing, in essence, individual morality.  It does not look beyond people to the larger environment and “system” within which people function.  This environment can bring to bear significant influences that can challenge the desired traits and values of safety culture policy and muddle their application to decisions and actions.  The limitations can be seen in the assessments of safety culture (surveys and similar) as well as the investigation of specific events, violations or non-conformances by licensees and the NRC.  We’ve read many of these and rarely have we encountered any probing of the “why” associated with perceived breakdowns in safety culture.

One exception and a very powerful case in point is contained in our post dated July 29, 2010.  The cited reference is an internal root cause analysis performed by FPL to address employee concerns and identified weaknesses in their corrective action program.  They cite production pressures as negatively impacting employee trust and recognition, and perceptions of management and operational decisions.  FPL took steps to change the origin and impact of production pressures, relieving some of the burden on the organization to contain those influences within the boundaries of safe operation.

Perhaps the NRC believes that it does not have the jurisdiction to probe these types of issues or even require licensees to assess their influence.  Yet the NRC routinely refers to “licensee burden” - cost, schedule, production impacts - in accepting deviations from nominal safety standards.****  We wonder if a broader view of safety culture in the context of the socio-technical system might better “inform” both regulatory policy and decisions and enhance safety management.


*  J.P. Dédale, "Why a Paradigm Shift Is Needed," IAEA International Experts’ Meeting on Human and Organizational Factors in Nuclear Safety in the Light of the Accident at the Fukushima Daiichi Nuclear Power Plant, Vienna May 21-24, 2013.


**  The NRC’s Information Notice 2013-15 states that safety culture is “essential to nuclear safety in all phases…”
 

***  "NRC Decision on FPL (Part 2)," Safetymatters (July 29, 2010).  See slide 18, Root Cause 2 and Contributing Causes 2.2 and 2.4. 

****  10 CFR 50.55a(g)(6)(i) states that the Commission may grant such relief and may impose such alternative requirements as it determines is authorized by law and will not endanger life or property or the common defense and security and is otherwise in the public interest, given the consideration of the burden upon the licensee (emphasis added).

Tuesday, September 17, 2013

Even Macy’s Does It

We have long been proponents of looking for innovative ways to improve safety management training for nuclear professionals.  We’ve taken the burden to develop a prototype management simulator, NuclearSafetySim, and made it available to our readers to experience for themselves (see our July 30, 2013 post).  In the past we have also noted other industries and organizations that have embraced simulation as an effective management training tool.

An August article in the Wall Street Journal* cites several examples of new approaches to manager training.  Most notable in our view is Macy’s use of simulations to have managers gain decision making experience.  As the article states:

“The simulation programs aim to teach managers how their daily decisions can affect the business as a whole.”

We won’t revisit all the arguments that we’ve made for taking a systems view of safety management, focusing on decisions as the essence of safety culture and using simulation to allow personnel to actualize safety values and priorities.  All of these could only enrich, challenge and stimulate training activities. 

A Clockwork Magenta

 
On the other hand what is the value of training approaches that reiterate INPO slide shows, regulatory policy statements and good practices in seemingly endless iterations?  Brings to mind the character Alex, the incorrigible sociopath in A Clockwork Orange with an unusual passion for classical music.**  He is the subject of “reclamation treatment”, head clamped in a brace and eyes pinned wide open, forced to watch repetitive screenings of anti-social behavior to the music of Beethoven’s Fifth.  We are led to believe this results in a “cure” but does it and at what cost?

Nuclear managers may not be treated exactly like Alex but there are some similarities.  After plant problems occur and are diagnosed, managers are also declared “cured” after each forced feeding of traits, values, and the need for increased procedure adherence and oversight.  Results still not satisfactory?  Repeat.



*  R. Feintzeig, "Building Middle-Manager Morale," Wall Street Journal (Aug. 7, 2013).  Retrieved Sept. 24, 2013.

**  M. Amis, "The Shock of the New:‘A Clockwork Orange’ at 50,"  New York Times Sunday Book Review (Aug. 31, 2013).  Retrieved Sept. 24, 2013.

Thursday, September 12, 2013

Bad Eggs?

We’ve often thought that intentional or willful violations of safety/regulatory requirements could provide a useful window into the dynamics of safety culture.  Now the NRC has just issued an Information Notice* listing recent instances of willful violations.  The Notice is titled “Willful Misconduct/Record Falsification and Nuclear Safety Culture” and reports on seven recent instances of such conduct.  From the title and throughout the notice the NRC asserts a link between willful violations and nuclear safety culture.  To wit it states, “An effective safety-culture is essential to nuclear safety at all phases of design, construction and operation and can help prevent willful misconduct by ensuring expectations and consequences are clearly stated and understood.” (p. 5)  The NRC adds, “The above willful misconduct issues and discussion highlights the need... to establish and implement an effective nuclear safety-culture.  This includes training, adequate oversight, and frequent communications especially for workers new to the nuclear industry.” (p. 6)

What we see here is consistent with the NRC’s pro forma approach to organizational safety performance issues.  The problem is culture; the answer is more training, more clarity of expectations, more oversight.**  Oh, and disciplinary actions for the errant individuals.  

Are we to take from this that the individuals involved in these situations are just “bad eggs”?  And the answer is some punishment and re-education?  Is this even consistent with the nature of willful violations and does the sheer number of recent experiences raise more fundamental questions, the most basic of which is “Why?”

Let’s start with what is different about willful violations.  Willful violations are deliberate, intentional and knowing.  In other words the individual knows his/her actions are against established policies or procedures.  This is not a case of carelessness or lack of knowledge of what is expected.  Thus it is hard to understand what would be achieved by more training and reinforcement of expectations.  The prescription for more oversight is also puzzling.  It appears to assume that violations will continue unless there is strict monitoring of behaviors.  Interestingly it is reliance on more oversight by managers who apparently weren’t providing the necessary oversight in the first place.

So on the one hand the corrective actions identified in the these events do not appear well suited to the nature of a willful violation.  Perhaps more importantly this treatment of the problem obscures deeper analysis of why such violations are occurring in the first place.  Why are personnel deciding to intentionally do something wrong?  Often willful acts have their basis in personal gain or covering up some other misdeed.  Nothing in the seven instances in the Notice even hint at this type of motivation.  Could it be an intent to do harm to the organization due to some other personal issue - a problem with a supervisor, being passed by for a promotion, etc?  Hmmm, I guess it’s possible but again there does not appear to be any hint of this in the available documentation. Or could it be that the individuals were responding to some actual or perceived pressure to get something done - more quickly, at less cost, or to avoid raising an issue that itself would cost time or money?  Again there was no exploration of motive for these violations in the NRC’s or licensee’s investigations.***

The apparent failure to fully investigate the motive for these violations is unfortunate as it leaves other critical factors unexplored and untreated.  Goal pressures almost always have their origin higher up in the organization.  Defaulting to reinforcing the culture side of the equation may not be effective due to the inherent contradiction in signals from upper management. 

In a prior post we suggested that safety culture be thought of as a “pressure boundary”, specifically “the willingness and ability of an organization to resist undue pressure on safety from competing business priorities”.   When resistance breaks down it can lead to shading of safety assessments, a decided lack of rigor in pursuing causes and extent of condition - or it can even lead to willful violations.  Relieving business pressure may be the far more effective antidote.


*  NRC Information Notice 2013-15: Willful Misconduct/Record Falsification and Nuclear Safety Culture (Aug. 23, 2013).  ADAMS ML13142A437.

**  In two instances modest civil penalties were also assessed.

***  We would remind our readers of our post dated April 2, 2012 regarding the guilty plea of one of the Massey coal mine supervisors to intentional violations of the law.  The stated reason: following the law would decrease coal production.