IAEA Nuclear Safety Culture Conference

The International Atomic Energy Agency (IAEA) recently sponsored a week-long conference* to celebrate 30 years of interest and work in safety culture (SC).  By our reckoning, there were about 75 individual presentations in plenary sessions and smaller groups; dialog sessions with presenters and subject matter experts; speeches and panels; and over 30 posters.  It must have been quite a circus.

We cannot justly summarize the entire conference in this space but we can highlight material related to SC factors we’ve emphasized or people we’ve discussed on Safetymatters, or interesting items that merit your consideration.

Topics We Care About

A Systems Viewpoint

Given that the IAEA has promoted a systemic approach to safety and it was a major conference topic it’s no surprise that many participants addressed it.  But we were still pleased to see over 30 presentations, posters and dialogues that included mention of systems, system dynamics, and systemic and/or holistic viewpoints or analyses.  Specific topics covered a broad range including complexity, coupling, Fukushima, the Interaction between Human, Technical and Organizational Factors (HTOF), error/incident analysis, regulator-licensee relationships, SC assessment, situational adaptability and system dynamics.

Role of Leadership

Leadership and Management for Safety was another major conference topic.  Leadership in a substantive context was mentioned in about 20 presentations and posters, usually as one of multiple success factors in creating and maintaining a strong SC.  Topics included leader/leadership commitment, skills, specific competences, attributes, obligations and responsibilities; leadership’s general importance, relationship to performance and role in accidents; and the importance of leadership in nuclear regulatory agencies. 

Decision Making

This was mentioned about 10 times, with multiple discussions of decisions made during the early stages of the Fukushima disaster.  Other presenters described how specific techniques, such as Probabilistic Risk Assessment and Human Reliability Analysis, or general approaches, such risk control and risk informed, can contribute to decision making, which was seen as an important component of SC.

Compensation and Rewards

We’ve always been clear: If SC and safety performance are important then people from top executives to individual workers should be rewarded (by which we mean paid money) for doing it well.  But, as usual, there was zero mention of compensation in the conference materials.  Rewards were mentioned a few times, mostly by regulators, but with no hint they were referring to monetary rewards.  Overall, a continuing disappointment.   

Participants Who Have Been Featured in Safetymatters

Over the years we have presented the work of many conference participants to Safetymatters readers.  Following are some familiar names that caught our eye.  Page numbers refer to the conference “Programme and Abstracts” document.
 
We have to begin with Edgar Schein, the architect of the cultural construct used by almost everyone in the SC space.  His discussion paper (p. 47) argued that the SC components in a nuclear plant depend on whether the executives actually create the climate of trust and openness that the other attributes hinge on.  We’ve referred to Schein so often he has his own label on Safetymatters.

Mats Alvesson’s presentation (p. 46) discussed “hyper culture,” the vague and idealistic terms executives often promote that look good in policy documents but seldom work well in practice.  This presentation is consistent with his article on Functional Stupidity which we reviewed on Feb. 23, 2016.

Sonja Haber’s paper (p. 55) outlined a road map for the nuclear community to move forward in the way it thinks about SC.  Dr. Haber has conducted many SC assessments for the Department of Energy that we have reviewed on Safetymatters. 

Ken Koves of INPO led or participated in three dialogue sessions.  He was a principal researcher in a project that correlated SC survey data with safety performance measures which we reviewed on Oct. 22, 2010 and Oct. 5, 2014.

Najmedin Meshkati discussed (p. 60) how organizations react when their control systems start to run behind environmental demands using Fukushima as an illustrative case.  His presentation draws on an article he coauthored comparing the cultures at TEPCO’s Fukushima Daiichi plant and Tohoku Electric’s Onagawa plant which we reviewed on Mar. 19, 2014.

Jean-Marie Rousseau co-authored a paper (p. 139) on the transfer of lesson learned from accidents in one industry to another industry.  We reviewed his paper on the effects of competitive pressures on nuclear safety management issues on May 8, 2013.

Carlo Rusconi discussed (p. 167) how the over-specialization of knowledge required by decision makers can result in pools of knowledge rather than a stream accessible to all members of an organization.  A systemic approach to training can address this issue.  We reviewed Rusconi’s earlier papers on training on June 26, 2013 and Jan. 9, 2014.

Richard Taylor’s presentation (p. 68) covered major event precursors and organizations’ failure to learn from previous events.  We reviewed his keynote address at a previous IAEA conference where he discussed using system dynamics to model organizational archetypes on July 31, 2012.

Madalina Tronea talked about (p. 114) the active oversight of nuclear plant SC by the National Commission for Nuclear Activities Control (CNCAN), the Romanian regulatory authority.  CNCAN has developed its own model of organizational culture and uses multiple methods to collect information for SC assessment.  We reviewed her initial evaluation guidelines on Mar. 23, 2012. 

Our Perspective

Many of the presentations were program descriptions or status reports related to the presenter’s employer, usually a utility or regulatory agency.  Fukushima was analyzed or mentioned in 40 different papers or posters.  Overall, there were relatively few efforts to promote new ideas, insights or information.  Having said that, following are some materials you should consider reviewing.

From the conference participants mentioned above, Haber’s abstract (p. 55) and Rusconi’s abstract (p. 167) are worth reading.  Taylor’s abstract (p. 68) and slides are also worth reviewing.  He advocates using system dynamics to analyze complicated issues like the effectiveness of organizational learning and how events can percolate through a supply chain.

Benoît Bernard described the Belgian regulator’s five years of experience assessing nuclear plant SC.  Note that lessons learned are described in his abstract (p. 113) but are somewhat buried in his presentation slides.

If you’re interested in a systems view of SC, check out Francisco de Lemos’ presentation (p. 63) which gives a concise depiction of a complex system plus a Systems Theoretic Accident Models and Processes (STAMP) analysis.  His paper is based on Nancy Leveson’s work which we reviewed on Nov. 11, 2013.

Diana Engström argued that nuclear personnel can put more faith in reported numbers than justified by the underlying information, e.g., CAP trending data, and thus actually add risk to the overall system.  We’d call this practice an example of functional stupidity although she doesn’t use that term in her provocative paper.  Both her abstract (p. 126) and slides are worth reviewing.

Jean Paries gave a talk on the need for resilience in the management of nuclear operations.  The abstract (p. 228) is clear and concise; there is additional information in his slides but they are a bit messy.

And that’s it for this installment.  Be safe.  Please don’t drink and text.


*  International Atomic Energy Agency, International Conference on Human and Organizational Aspects of Assuring Nuclear Safety: Exploring 30 years of Safety Culture (Feb. 22–26, 2016).  This page shows the published conference materials.  Thanks to Madalina Tronea for publicizing them.  Dr. Tronea is the founder/moderator of the LinkedIn Nuclear Safety Culture discussion group. 

Leadership and Safety Culture

Cover of the first issue

It’s an election year in America and voters are assessing candidates who all claim they can provide the leadership the country needs.  A recent article* in The New Yorker offers a primer on the nature of leadership.  The article is engaging because we talk a lot about leadership in the nuclear industry in areas ranging from general management to molding or influencing culture.**  Following are some highlights from the article.

For starters, leadership can mean different things to different people.  The article cites a professor who found more than 200 definitions in the modern leadership literature.  Of necessity, the author focused on a small subset of the literature, starting with sociologist Max Weber who distinguished between “charismatic” and “bureaucratic” leadership.

The charismatic model is alive and well; it’s reflected in the search for CEOs with certain traits, e.g., courage, decisiveness, intelligence or attractiveness, especially during periods of perceived crisis.  Unfortunately, the track record of such people is mixed; according to one researcher, “The most powerful factor determining a company’s performance is the condition of the market in which it operates.” (p. 67)

The bureaucratic model focuses on process, i.e., what a leader actually does.  Behaviors might include gathering information on technology and competitors, setting goals, assembling teams and tracking progress, in other words, the classic plan, organize, staff, direct and control paradigm.  But a CEO candidate’s actual process might not be visible or not what he says it is.  And, in our experience, if the CEO cannot bring strategic insight or a robust vision to the table, the “process” is a puerile exercise.

So how does one identify the right guy or gal?  Filtering is one method to reduce risk in the leader selection process.  Consider the nuclear industry’s long infatuation with admirals.  Why?  One reason is they’ve all jumped through the same hoops and tend to be more or less equally competent—a safe choice but one that might not yield out-of-the-ballpark results.  A genuine organizational crisis might call for an unfiltered leader, an outsider with a different world view and experience, who might deliver a resounding success (e.g., Abraham Lincoln).  Of course, the downside risk is the unfiltered leader may fail miserably.

If you believe leadership is learnable, you’re in luck; there is a large industry devoted to teaching would-be leaders how to empower and inspire their colleagues and subordinates, all the while evidencing a set of pious virtues.  However, one professor thinks this is a crock and what the leadership industry actually does is “obscure the degree to which companies are poorly and selfishly run for the benefit of the powerful people in charge.” (p. 68)

The author sees hope in approaches that seek to impart more philosophy or virtue to leaders.  He reviews at length the work of Elizabeth Samet, an English professor at the U.S. Military Academy (West Point).  She presents leadership through a wide-angle lens, from General Grant’s frank memoirs to a Virginia Woolf essay.  To gain insight into ambition, her students read “Macbeth.”  (Ooops!  I almost typed “MacTrump.”)    

Our Perspective

The New Yorker article is far from a complete discussion of leadership but it does spur one to think about the topic.  It’s worth a quick read and some of the author’s references are worth additional research.  If you want to skip all that, what you should know is “. . . leaders in formal organizations have the power and responsibility to set strategy and direction, align people and resources, motivate and inspire people, and ensure that problems are identified and solved in a timely manner.”***

At Safetymatters, we believe effective leadership is necessary, but not sufficient, to create a strong safety culture (SC).  Not all aspects of leadership are important in the quest for a strong SC.  Leaders need some skills, e.g., the ability to communicate their visions, influence others and create shared understanding.  But the critical aspects are decision-making and role modeling.

Every decision the leader makes must show respect for the importance of safety.  The people will be quick to spot any gap between words and decisions.  Everyone knows that production, schedule and budget are important—failure to perform eventually means jobs and careers go away—but safety must always be a conscious and visible consideration.

Being a role model is also important.  Again, the people will spot any disregard or indifference to safety considerations, rules or practices.

There is no guarantee that even the most gifted leader can deliver a stronger SC.  Although the leader may create a vision for strong SC and attempt to direct behavior toward that vision, the dynamics of SC are complex and subject to multiple factors ranging from employees’ most basic values to major issues that compete for the organization’s attention and resources. 

To close on a more upbeat note, effective leadership is open to varying definitions and specifications but, to borrow former Supreme Court Justice Potter Stewart’s famous phrase, we know it when we see it.****


*  J. Rothman, “Shut Up and Sit Down,” The New Yorker (Feb. 29, 2016), pp. 64-69.

**  For INPO, leadership is sine qua non for an effective nuclear organization.

***  This quote is not from The New Yorker article.  It is from a review of SC-related social science literature that we posted about on Feb. 10, 2013.

****  Justice Stewart was talking about pornography but the same sort of Kantian knowing can be applied to many topics not amenable to perfect definition.

Cultural Tidbits from McKinsey

We spent a little time poking around the McKinsey* website looking for items that could be related to safety culture and found a couple.  They do not provide any major insights but they do spur us to think of some questions for you to ponder about your own organization.

One article discussed organizational redesign** and provided a list of recommended rules, including establishing metrics that show if success is being achieved.  Following is one such metric.

“One utility business decided that the key metric for its efficiency-driven redesign was the cost of management labor as a proportion of total expenditures on labor.  Early on, the company realized that the root cause of its slow decision-making culture and high cost structure had been the combination of excessive management layers and small spans of control.  Reviewing the measurement across business units and at the enterprise level became a key agenda item at monthly leadership meetings.” (p. 107)

What percent of total labor dollars does your organization spend on “management”?  Could your organization’s decision making be speeded up without sacrificing quality or safety?  Would your organization rather have the “right” decision (even if it takes a long time to develop) or no decision at all rather than risk announcing a “wrong” one?

A second article discussed management actions to create a longer view among employees,*** including clearly identifying and prioritizing organizational values.  Following is an example of action related to values.

“The pilots of one Middle East–based airline frequently write incident reports that candidly raise concerns, questions, and observations about potential hazards.  The reports are anonymous and circulate internally, so that pilots can learn from one another and improve—say, in handling a particularly tricky approach at an airport or dealing with a safety procedure.  The resulting conversations reinforce the safety culture of this airline and the high value it places on collaboration.  Moreover, by making sure that the reporting structures aren’t punitive, the airline’s executives get better information and can focus their attention where it’s most needed.”

How do your operators and other professionals share experiences and learning opportunities among themselves at your site?  How about throughout your fleet?  Does documenting anything that might be construed as weakness require management review or approval?  Is management (or the overall organization) so fearful of such information being seen by regulators or the public, or discovered by lawyers, that the information is effectively suppressed?  Is your organization paranoid or just applying good business sense?  Do you have a culture that would pass muster as “just”?

Our Perspective

Useful nuggets on management or culture are where you find them.  Others’ experiences can stimulate questions; the answers can help you better understand local organizational phenomena, align your efforts with the company’s needs and build your professional career.


*  McKinsey & Company is a worldwide management consulting firm.

**  S. Aronowitz et al, “Getting organizational redesign right,” McKinsey Quarterly, no. 3 (2015), pp. 99-109.

***  T. Gibbs et al, “Encouraging your people to take the long view,” McKinsey Quarterly (Sept. 2012).

Obtain Better Decisions by Asking Better Questions

We’re currently experiencing a reduced flow of quality feedstock into our safety culture mill.  But we did see a reference to a Harvard Business Review (HBR) article* that’s worth a quick read.

The authors’ thesis is the pressure on business to make decisions ever more quickly means important questions may never get asked, or even considered, which leads to poor decision-making.  Their proposed fix is to ask more, better questions to help frame decisions.  They suggest four types of questions, presented in the consultant’s favorite typology:  the two-by-two matrix.  In this case, one axis is the View of the Problem (wide or narrow) and the other is the Intent of the Question (to affirm or discover), as shown in the following figure.

Types of Questions to Improve Decision Making  (Source Mu Sigma)

Clarifying questions are focused on helping participants or managers understand what has happened so far, e.g., the data gathered or partial decisions already made.  People often don’t ask these questions because of cultural pressures to move forward, or they tend to make assumptions and fill in any missing parts themselves.**

Adjoining questions explore related aspects of the problem utilizing available information, e.g., how the results of this analysis could be applied elsewhere. 

Funneling questions are focused on learning more about the analysis to date.  How was an answer derived?  What were your assumptions?  What are the root causes of this problem?  The authors opine that most analytical teams usually do a good job of asking this type of question.

Elevating questions raise broader issues and create opportunities to make new connections between individual decisions, e.g., what are the larger issues or trends we should be concerned about?

There is a cultural dimension to question asking, particularly the unspoken rules about what types of questions can be asked, and by whom, in the decision making process.  Leaders need to encourage people to ask questions and co-workers need to be tolerant of the question askers rather than pushing to obtain and deliver an answer.

Our Perspective

The information in this article is hardly magical.  Most of us recognize that the best investigators and managers know What kind of questions they are asking and Why.  But we do have a few exercises for you to think about.   

For starters, look at the questions suggested or prescribed in your official problem-solving or problem analysis recipes.  Do they omit any types of questions that could add value to your immediate situation, bigger picture issues or the overall process?

What’s your problem solving culture like?  How are people treated who ask questions, especially devil’s advocate questions, that don’t add instant value to the search for an answer?

Finally, consider Millstone’s issue with a turbine-driven auxiliary feedwater pump (which we reviewed on Jan. 15, 2015).  Could more extensive questioning during the initial analysis phase have more quickly led the investigators to a correct understanding of the problem?    


*  T. Pohlmann and N.M. Thomas, “Relearning the Art of Asking Questions,” Harvard Business Review on-line (Mar. 27, 2015).  The authors are not famous professors.  They are two consultants with a Mu Sigma, a Big Data company, who are publishing under the HBR aegis.  That doesn’t disqualify their work, it’s just something to keep mind as they describe a construct their firm uses.

**  For an informative and entertaining essay on how people develop their own models of what’s going on in the world, even when they are wildly misinformed, check out “We Are All Confident Idiots.”

Training....Yet Again

U.S. Navy SEALS in Training

We have beat the drum on the value of improved and innovative training techniques for improving safety management performance for some time.  Really since the inception of this blog where our paper, “Practicing Nuclear Safety Management,”* was one of the seminal perspectives we wanted to bring to our readers.  We continue to encounter knowledgeable sources that advocate practice-based approaches and so continue to bring them to our readers’ attention.  The latest is an article from the Harvard Business Review that calls attention to, and distinguishes, “training” as an essential dimension of organizational learning.  The article is “How the Navy SEALS Train for Leadership Excellence.”**  The author, Michael Schrage,*** is a research fellow at MIT who reached out to a former SEAL, Brandon Webb, who transformed SEAL training.  The author contends that training, as opposed to just education or knowledge, is necessary to promote deep understanding of a business or market or process.  Training in this sense refers to actually performing and practicing necessary skills.  It is the key to achieving high levels of performance in complex environments. 

One of Webb’s themes that really struck a chord was: “successful training must be dynamic, open and innovative…. ‘It’s every teacher’s job to be rigorous about constantly being open to new ideas and innovation’, Webb asserts.”  It is very hard to think about much of the training in the nuclear industry on safety culture and related issues as meeting these criteria.  Even the auto industry has recently stepped up to require the conduct of decision simulations to verify the effectiveness of corrective actions - in the wake of the ignition switch-related accidents. (see our May 22, 2014 post.)

In particular the reluctance of the nuclear industry and its regulator to address the presence and impact of goal conflicts on safety continues to perplex us and, we hope, many others in the industry.   It was on the mind of Carlo Rusconi more than a year ago when he observed: “Some of these conflicts originate high in the organization and are not really amenable to training per se” (see our Jan. 9, 2014 post.)  However a certain type of training could be very effective in neutralizing such conflicts - practicing making safety decisions against realistic fact-based scenarios.  As we have advocated on many occasions, this process would actualize safety culture principles in the context of real operational situations.  For the reasons cited by Rusconi it builds teamwork and develops shared viewpoints.  If, as we have also advocated, both operational managers and senior managers participated in such training, senior management would be on the record for its assessment of the scenarios including how they weighed, incorporated and assessed conflicting goals in their decisions.  This could have the salutary effect of empowering lower level managers to make tough calls where assuring safety has real impacts on other organizational priorities.  Perhaps senior management would prefer to simply preach goals and principles, and leave the tough balancing that is necessary to implement the goals to their management chain.  If decisions become shaded in the “wrong” direction but there are no bad outcomes, senior management looks good.  But if there is a bad outcome, lower level managers can be blamed, more “training” prescribed, and senior management can reiterate its “safety is the first priority” mantra.


*  In the paper we quote from an article that highlighted the weakness of “Most experts made things worse.  Those managers who did well gathered information before acting, thought in terms of complex-systems interactions instead of simple linear cause and effect, reviewed their progress, looked for unanticipated consequences, and corrected course often. Those who did badly relied on a fixed theoretical approach, did not correct course and blamed others when things went wrong.”  Wall Street Journal, Oct. 22, 2005, p. 10 regarding Dietrich Dörner’s book, The Logic of Failure.  For a comprehensive review of the practice of nuclear safety, see our paper “Practicing Nuclear Safety Management”, March 2008.

**  M. Schrage, "How the Navy SEALS Train for Leadership Excellence," Harvard Business Review (May 28, 2015).

***  Michael Schrage, a research fellow at MIT Sloan School’s Center for Digital Business, is the author of the book Serious Play among others.  Serious Play refers to experiments with models, prototypes, and simulations.

Nuclear Safety Assessment Principles in the United Kingdom

A reader sent us a copy of “Safety Assessment Principles for Nuclear Facilities” (SAPs) published by the United Kingdom’s Office for Nuclear Regulation (ONR).*  For documents like this, we usually jump right to the treatment of safety culture (SC).  However, in this case we were impressed with the document’s accessibility, organization and integrated (or holistic) approach so we want to provide a more general review.

ONR uses the SAPs during technical assessments of nuclear licensees’ safety submissions.  The total documentation package developed by a licensee to demonstrate high standards of nuclear safety is called the “safety case.”

Accessibility

The language is clear and intended for newbies as well as those already inside the nuclear tent.  For example, “The SAPs contain principles and guidance.  The principles form the underlying basis for regulatory judgements made by inspectors, and the guidance associated with the principles provides either further explanation of a principle, or their interpretation in actual applications and the measures against which judgements can be made.” (p. 11) 

Also furthering ease of use, the document is not strewn with acronyms.  As a consequence, one doesn’t have to sit with glossary in hand just to read the text.

Organization

ONR presents eight fundamental principles including responsibility for safety, limitation of risks to individuals and emergency planning.  We’ll focus on another fundamental principle, Leadership and Management (L&M) because (a) L&M activities create the context and momentum for a positive SC and (b) it illustrates holistic thinking.

L&M is comprised of four subordinate (but still high-level) inter-related principles: leadership, capable organization, decision making and learning.  “Because of their inter-connected nature there is some overlap between the principles. They should therefore be considered as a whole and an integrated approach will be necessary for their delivery.” (p. 18)

Drilling down further, the guidance for leadership includes many familiar attributes.  We want to acknowledge attributes we have been emphasizing on Safetymatters or reflect new thoughts.  Specifically, leaders must recognize and resolve conflict between safety and other goals, ensure that the reward systems promote the identification and management of risk, encourage safe behavior and discourage unsafe behavior or complacency; and establish a common purpose and collective social responsibility for safety. (p.19) 

Decision making (another Safetymatters hot button issue) receives a good treatment.  Topics covered include explicit recognition of goal conflict; appreciating the potential for error, uncertainty and the unexpected; and the essential functions of active challenges and a questioning attitude.

We do have one bone to pick under L&M: we would like to see words to the effect that safety performance and SC should be significant components of the senior management reward system.

Useful Points

Helpful nuggets pop up throughout the text.  A few examples follow.

“The process of analysing safety requires creativity, where people can envisage the variety of routes by which radiological risks can arise from the technology. . . . Safety is achieved when the people and physical systems together reliably control the radiological hazards inherent in the technology. Therefore the organizational systems (ie interactions between people) are just as important as the physical systems, . . . “ (pp. 25-26)

“[D]esigners and/or dutyholders may wish to put forward safety cases that differ from [SAP] expectations.   As in the past, ONR inspectors should consider such submissions on their individual merits. . . . ONR will need to be assured that such cases demonstrate equivalence to the outcomes associated with the use of the principles here,. . .” (p. 14)  The unstated principle here is equifinality; in more colorful words, there is more than one way to skin a cat.

There are echoes of other lessons we’ve been preaching on Safetymatters.  For example “The principle of continuous improvement is central to achieving sustained high standards of nuclear safety. . . . Seeking and applying lessons learned from events, new knowledge and experience, both nationally and internationally, must be a fundamental feature of the safety culture of the nuclear industry.” (p. 13)

And, in a nod to Nicholas Taleb, if a “hazard is particularly high, or knowledge of the risk is very uncertain, ONR may choose to concentrate primarily on the hazard.” (p. 8)

Our Perspective

Most of the content of the SAPs will be familiar to Safetymatters readers.  We suggest you skim the first 23 pages of the document covering introductory material and Leadership & Management.  SAPs is an excellent example of a regulator actually trying to provide useful information and guidance to current and would-be licensees and is far better than the simple-minded laundry lists promulgated by IAEA.


*  Office for Nuclear Regulation, “Safety Assessment Principles for Nuclear Facilities” Rev. 0 (2014).  We are grateful to Bill Mullins for forwarding this document to us.

DNFSB Hearings on Safety Culture, Round Three

DNFSB Headquarters

On October 7, 2014 the Defense Nuclear Facilities Safety Board (DNFSB) held its third and final hearing* on safety culture (SC) at Department of Energy (DOE) nuclear facilities.  The original focus was on the Hanford Waste Treatment Plant (WTP) but this hearing also discussed the Waste Isolation Pilot Plant (WIPP), the Pantex plant and other facilities.  There were three presenters: DOE Secretary Moniz and two of his top lieutenants.  A newspaper article** published the same day reported key points made during the hearing and you should read that article along with this post.  This post focuses on items not included in the newspaper article, including the tone of the hearing and other nuances.  The presenters used no slides and the hearing transcript has not yet been released.  The only current record of the hearing is a DNFSB video.

Secretary Moniz

Moniz has been Secretary for about a year-and-a-half.  In his view, the keys to improving SC are training, consistent senior management attention, and procurement modifications, i.e., DOE’s intent to revise RFP and contracting processes to include SC expectations.  He also said fostering the consideration of SC in all decisions, including resource allocation, is important.  Board member Sullivan asked about the SC issues at Pantex and Moniz provided a generic answer about improving self-assessments and sharing lessons learned but ultimately punted to the next presenter, Ms. Creedon.

Principal Deputy Administrator Creedon, National Nuclear Security Administration (NNSA)

Creedon has been in her position for two months.  She believes NNSA employees get the job done in spite of bureaucracy but they need greater trust in senior management who, in turn, must work harder to engage the workforce.  Returning to the Pantex*** issues, Sullivan asked why the recommendations of the plant’s outside technical advisors had been ignored for years.  Creedon said she would work to improve communications up and down the organization.  In a separate exchange, she provided an example of positive reinforcement where NNSA employees can receive cash awards ($500) for good work. 

Creedon’s  prior position was in the Department of Defense.  To the extent she has the warfighter mentality (“Anything, anywhere, anytime…at any cost”)**** then balancing mission and safety may not be natural for her.  Her response to a question on this topic was not encouraging; she claimed the motto du jour for NNSA (“Mission First, People Always”) adequately addresses safety's prioity but it obviously doesn’t even mention safety.

Acting Assistant Secretary for Environmental Management Whitney

Whitney is also new in his job but not to DOE, coming from DOE Oak Ridge.  He laid out his goals of establishing trust, a questioning attitude and mutual respect.  He was asked about a SC assessment finding that DOE senior managers don’t feel responsible for safety, rather it belongs to the site leads or one of the EM mission support units.  Whitney said that was unacceptable and described the intent to add SC factors to senior management evaluations.  He also repeated the plan to upgrade the WTP contractor evaluation to include SC factors.  He noted that most employees stay at one site for their entire career, making it hard to transfer SC from site to site.

Our Perspective

The overall tone of the hearing was collegial.  The Board expressed support and encouragement for the presenters, all of whom are relatively new in their jobs.  The presenters all stayed on message and reinforced each other.  For example, for WTP one message is “We know there are still significant SC issues at WTP but we have the right team in place and are taking action and making progress.  Changing a decades-old culture takes time.”  Whitney received more of a (polite) grilling probably because the WTP and the WIPP are under his purview.

We are totally supportive of DOE’s stated intent to add SC factors to contracts and senior management evaluations.  When players have skin in the game, the chances of seeing desired behavioral changes are greatly increased.  We are equally supportive of Secretary Moniz’ desire to create a culture that incorporates safety considerations in all decisions.

DOE is trying to make its employees more conscious of safety’s importance; two thousand mangers have gone through SC training and there’s more to come.  Now we’re starting to worry about the drumbeat of SC creating a Weltanschauung where a strong SC is sine quo non for good outcomes and a weak SC is always present when bad outcomes occur.  Organizational reality is more complicated.  An organization with a mediocre SC can achieve satisfactory results if other effective controls and incentives are in place; an organization with a strong SC can still make poor decisions.  And luck can run good or bad for anyone.

*  DNFSB Oct. 7, 2014 Safety Culture Public Meeting and Hearing.  We posted on the first hearing on June 9, 2014 and the second hearing on Sept. 4, 2014.

**  A. Cary, “Moniz says safety culture at Hanford vit plant led to problems,” Tri-City Herald (Oct. 7, 2014).

***  NNSA's responsibilities include Pantex which has recognized SC issues.

****  See the third footnote in our Sept. 4, 2014 post.

1995 ANS Safety Culture Conference: A Portal to the Past

In April 1995 the American Nuclear Society (ANS) sponsored a nuclear safety culture (SC) conference in Vienna.  This was a large undertaking, with over 80 presentations; the proceedings are almost 900 pages in length.*  Presenters included industry participants, regulators, academics and consultants.  1995 was early in the post-Soviet era and the new openness (and concerns about Soviet reactors) led to a large number of presenters from Russia, Ukraine and Eastern Europe.  This post presents some conference highlights on topics we emphasize on Safetymatters.

Decision Making

For us, decision making should be systemic, i.e., consider all relevant inputs and the myriad ways a decision can affect consequences.  The same rigor should be applied to all kinds of decisions—finance, design, operations, resource allocation, personnel, etc.  Safety should always have the highest priority and decisions should accord safety its appropriate consideration.  Some presenters echoed this view.

“Safety was (and still is) seen as being vital to the success of the industry and hence the analysis and assessment of safety became an integral part of management decision making” (p. 41); “. . . in daily practice: overriding priority to safety is the principle, to be taken into account before making any decision” (p. 66); and “The complexity of operations implies a systemic decision process.” (p. 227)

The relationship between leadership and decisions was mentioned.  “The line management are a very important area, as they must . . . realise how their own actions and decisions affect Safety Culture.  The wrong actions, or perceived messages could undermine the work of the team leaders” (p. 186); “. . . statements alone do not constitute support; in the intermediate and long-term, true support is demonstrated by behavior and decision and not by what is said.” (p. 732)

Risk was recognized as a factor in decision making.  “Risk culture yields insights that permit balanced safety vs.cost decisions to be made” (p. 325); “Rational decision making is based on facts, experience, cognitive (mental) models and expected outcomes giving due consideration to uncertainties in the foregoing and the generally probabilistic nature of technical and human matters.  Conservative decision making is rational decision making that is risk-averse.  A conservative decision is weighted in favor of risk control at the expense of cost.” (p. 435)

In sum, nuclear thought leaders knew what good decision making should look like—but we still see cases that do not live up to that standard.

Rewards

Rewards or compensation were mentioned by people from nuclear operating organizations.  Incentive-based compensation was included as a key aspect of the TEPCO management approach (p. 551) and a nuclear lab manager recommended using monetary compensation to encourage cooperation between organizational departments. (p. 643)  A presenter from a power plant said “A recognition scheme is in place . . . to recognise and reward individuals and teams for their contribution towards quality improvement and nuclear safety enhancement.” (p. 805)

Rewards were also mentioned by several presenters who did not come from power plants.  For example, the reward system should stress safety (p. 322); rewards should be given for exhibiting a “caring attitude” about SC (p. 348) and to people who call attention to safety problems. (p. 527)  On the flip side, a regulator complained about plants that rewarded behavior that might cause safety to erode. (pp. 651, 656) 

Even in 1995 the presentations could have been stronger since INSAG-4** is so clear on the topic: “Importantly, at operating plants, systems of reward do not encourage high plant output levels if this prejudices safety.  Incentives are therefore not based on production levels alone but are also related to safety performance.” (INSAG-4, p. 11)  Today, our own research has shown that nuclear executives’ compensation often favors production.   

Systems Approach

We have always favored nuclear organizational mental models that consider feedback loops, time delays, adaptation, evolution and learning—a systems approach.  Presenters’ references to a system include “commercial, public, and military operators of complex high reliability socio-technical systems” (p. 260); “. . . assess the organisational, managerial and socio-technical influences on the Safety Culture of socio-technical systems such as nuclear power plants” (p. 308); “Within the complex system such as . . . [a] nuclear power plant there is a vast number of opportunities for failures to stay hidden in the system” (p. 541); and “It is proposed that the plant should be viewed as an integrated sociotechnical system . . .” (p. 541)

There are three system-related presentations that we suggest you read in their entirety; they have too many good points to summarize here.  One is by Electricité de France (EdF) personnel (pp. 193-201), another by Constance Perin (pp. 330-336) and a third by John Carroll (pp. 338-345). 

Here’s a sample, from Perin: “Through self-analysis, nuclear organizations can understand how they currently respond socially, culturally, and technically to such system characteristics of complexity, density, obscured signals, and delayed feedback in order to assure their capacities for anticipating, preventing, and recovering from threats to safety.” (p. 330)  It could have been written yesterday.

The Role of the Regulator

By 1995 INSAG-4 had been published and generally accepted by the nuclear community but countries were still trying to define the appropriate role for the regulator; the topic merited a half-dozen presentations.  Key points included the regulator (1) requiring that an effective SC be established, (2) establishing safety as a top-level goal and (3) performing some assessment of a licensee’ safety management system (either directly or part of ordinary inspection duties).  There was some uncertainty about how to proceed with compliance focus vs. qualitative assessment.

Today, at least two European countries are looking at detailed SC assessment, in effect, regulating SC.  In the U.S., the NRC issued a SC policy statement and performs back-door, de facto SC regulation through the “bring me another rock” approach.

So conditions have changed in regulatory space, arguably for the better when the regulator limits its focus to truly safety-significant activities.

Our Perspective

In 1995, some (but not all) people held what we’d call a contemporary view of SC.  For example, “Safety culture constitutes a state of mind with regard to safety: the value we attribute to it, the priority we give it, the interest we show in it.  This state of mind determines attitudes and behavior.” (p. 495)

But some things have changed.  For example, several presentations mentioned SC surveys—their design, administration, analysis and implications.  We now (correctly) understand that SC surveys are a snapshot of safety climate and only one input into a competent SC assessment.

And some things did not turn out well.  For example, a TEPCO presentation said “the decision making process is governed by the philosophy of valuing harmony highly so that a conclusion preferred by all the members is chosen as far as possible when there are divided opinions.” (p. 583)  Apparently harmony was so valued that no one complained that Fukushima site protection was clearly inadequate and essential emergency equipment was exposed to grave hazards. 


*  A. Carnino and G. Weimann, ed., “Proceedings of the International Topical Meeting on Safety Culture in Nuclear Installations,” April 24-28, 1995 (Vienna: ANS Austria Local Section, 1995).  Thanks to Bill Mullins for unearthing this document.

**  International Nuclear Safety Advisory Group, “Safety Culture,” Safety Series No. 75-INSAG-4, (Vienna: IAEA, 1991). INSAG-4 included a definition of SC, a description of SC components, and illustrative evidence that the components exist in a specific organization.

GM Part 3 - Lawyers, Decision Making and...Simulation?

The GM story continues to unfold on a daily basis.  We’ve already lost track of the number of recalls as it appears that any and every possible safety defect from prior years has been added to the recall list.  This is reminiscent of the “problem” nuclear plants in the 1990s - NRC mandated improvement programs precipitated an avalanche of condition reports into the plants’ Corrective Action Programs, requiring immense resources and time to sort out and prioritize the huge volume of issues.

In our prior post on GM product safety issues, we critiqued the structure of management’s independent review being conducted by attorney Anton Valukas based in part on the likelihood that GM’s legal department would be a subject of the review.  Asking the chairman of a law firm with a long standing relationship with GM, to pull this off seemed, at a minimum, to be unnecessary, and potentially could undermine the credibility of the assessment.  Now we see in further reporting of the GM issues by the New York Times* that in fact GM’s lawyers are becoming a key focus of the investigation.  The implication is that GM’s lawyers may have been the gate keepers on information related to the Cobalt ignition switches and/or been enablers of a decision process that did not result in aggressive action.

Of greater interest is the Consent Order** entered into by GM and the United States Department of Transportation, National Highway Traffic Safety Administration.  The headline was the $35 million civil penalty but there were more interesting nuggets within the order.  Among a series of required actions by GM to improve timeliness and data to support safety defect evaluations were three actions specifically focusing on safety decision making.  One is to ensure that safety issues are expeditiously brought to the attention of “committees and individuals with authority to make safety recall decisions.” (p. 10)***  Second, GM will have to meet with the NHTSA on a monthly basis for one year to review its decision making on potential safety issues.  And third,

“GM shall meet with NHTSA no later than 120 calendar days after execution of this Consent Order to conduct simulations—i.e., an exercise to discuss hypothetical scenarios, for the purpose of assessing the effectiveness of the improvements [in processes and analytics to identify safety-related defects]…” (p. 9, emphasis added)  We find the emphasis of the Consent Order both fascinating and appropriate.  It emphasizes decision making - the process, timeliness, engagement of appropriate participants, and transparency - as essential to assuring appropriate outcomes.  It opens that process to scrutiny by the NHTSA through monthly reviews of actual decisions.  And most strikingly, it requires the conduct of decision simulations to verify the effectiveness of the improvements.

The provisions of the Consent Order establish a fundamentally new and better approach to rectifying deficiencies in safety performance and are consistent with themes we have been advocating for some time.  It departs from the simplistic - blame some individuals, reinforce expectations, emphasize values and improve processes - catechism that is pursued within the nuclear industry and others as well.  It seems to recognize that safety related decisions constitute the essence of assuring safety.  Rather than just reviewing and investigating bad outcomes, the Consent Order opens the door to making the results of all ongoing decisions transparent and reviewable.  Further it even calls for practicing the decision making process - through simulations - to verify the effectiveness of the process and the results.  Practicing complex and nuanced safety decisions to improve the process and decision making skills - what an idea.

It is no news flash to our readers that we have not only advocated these approaches, we have developed prototype tools for these purposes.  We have made the NuclearSafetySim simulation tool available for almost a year via this blog and linked to its website.  What has been the result?  While it is clear there have been many viewings of these materials, there has not been a single inquiry or follow-up by the nuclear industry, the NRC or INPO.****  At the same time there have been no initiatives within those groups to develop new or improved tools and methods for improving safety management.  Why?


*  B. Vlasic, “Inquiry by General Motors Is Said to Focus on Its Lawyers,” New York Times (May 17, 2014).  Retrieved May 22, 2014. 

**  Consent Order between the National Highway Traffic Safety Administration and General Motors Company re: NHTSA’s Timeliness Query TQ14-001 (May 16, 2014).

***  Including GM’s Executive Field Action Decision Committee and Field Performance Evaluation Recommendation Committee. (p. 9)

****  Ironically, the only serious interest has been expressed within the oil/gas industry which appears much more open to exploring innovative approaches.

DOE Report on WIPP's Safety Culture

On Feb. 14, 2014, an incident at the Department of Energy (DOE) Waste Isolation Pilot Plant (WIPP) resulted in the release of radioactive americium and plutonium into the environment.  This post reviews DOE’s Phase 1 incident report*, with an emphasis on safety culture (SC) concerns.

From the Executive Summary

The Accident Investigation Board (the Board) concluded that a more thorough hazard analysis, coupled with a better filter system could have prevented the unfiltered above ground release. (p. ES-1)

The root cause of the incident was Nuclear Waste Partnership’s (NWP**, the site contractor) and the DOE Carlsbad Field Office’s (CBFO) failure to manage the radiological hazard. “The cumulative effect of inadequacies in ventilation system design and operability compounded by degradation of key safety management programs and safety culture [emphasis added] resulted in the release of radioactive material . . . and the delayed/ineffective recognition and response to the release.” (pp. ES 6-7)

The report presents eight contributing causes, most of which point to NWP deficiencies.  SC was included as a site-wide concern, specifically the SC does not fully implement DOE safety management policy, “[t]here is a lack of a questioning attitude, reluctance to bring up and document issues, and an acceptance and normalization of degraded equipment and conditions.”  A recent Safety Conscious Work Environment (SCWE) survey suggests a chilled work environment. (p. ES-8)

The report includes 31 conclusions, 4 related to SC.  “NWP and CBFO have allowed the safety culture at the WIPP project to deteriorate . . . Questioning attitudes are not welcomed by management . . . DOE has exacerbated the safety culture problem by referring to numbers of [problem] reports . . . as a measure of [contractor] performance . . . . [NWP and CBFO] failed to identify weaknesses in . . . safety culture.” (pp. ES 14-15, 19-20)

The report includes 47 recommendations (called Judgments of Need) with 4 related to SC.  They cover leadership (including the CBFO site manager) behavior, organizational learning, questioning attitude, more extensive use of existing processes to raise issues, engaging outside SC expertise and improving contractor SC-related processes. (ibid.)

Report Details

The body of the report presents the details behind the conclusions and recommendations.  Following are some of the more interesting SC items, starting with our hot button issues: decision making (esp. the handling of goal conflict), corrective action, compensation and backlogs. 

Decision Making

The introduction to section 5 on SC includes an interesting statement:  “In normal human behavior, production behaviors naturally take precedence over prevention behaviors unless there is a strong safety culture - nurtured by strong leadership.” (p. 61)

The report suggests nature has taken its course: WIPP values production first and most.  “Eighteen emergency management drills and exercises were cancelled in 2013 due to an impact on operations. . . .Management assessments conducted by the contractor have a primary focus on cost and schedule performance.” (p. 62)  “The functional checks on CAMs [continuous air monitors] were often delayed to allow waste-handling activities to continue.” (p. 64)  “[D]ue consideration for prioritization of maintenance of equipment is not given unless there is an immediate impact on the waste emplacement processes.” (p. ES-17)  These observations evidence an imbalance between the goals of production and prevention (against accidents and incidents) and, following the logic of the introductory statement, a weak SC.

Corrective Action

The corrective action program has problems.  “The [Jan. 2013] SCWE Self-Assessment . . . identified weaknesses in teamwork and mutual respect . . . Other than completing the [SCWE] National Training Center course, . . . no other effective corrective actions have been implemented. . . . [The Self-Assessment also ]“identified weaknesses in effective resolution of reported problems.” (p. 63)  For problems that were reported, “The Board noted several instances of reported deficiencies that were either not issued, or for which corrective action plans were not developed or acted on for months.” (p. 65)

Compensation

Here is the complete text of Conclusion 14, which was excerpted above: “DOE has exacerbated the safety culture problem by referring to numbers of ORPS [incident and problem] reports and other deficiency reporting documents, rather than the significance of the events, as a measure of performance by Source Evaluation Boards during contract bid evaluations, and poor scoring on award fee determinations.  Directly tying performance to the number of occurrence reports drives the contractor to non-disclosure of events in order to avoid the poor score. [emphasis added]  This practice is contrary to the Department’s goals of the development and implementation of a strong safety culture across our projects.” (p. ES-15)  ‘Nuff said. 

Backlogs

Maintenance was deferred if it interfered with production.  Equipment and systems were  allowed to degrade (pp. ES-7, ES-17, C-7)  There is no indication that maintenance backlogs were a problem; the work simply wasn’t done.

Other SC Issues

In addition to our Big Four and the issues cited from the Executive Summary, the report mentions the following concerns.  (A listing of all SC deficiencies is presented on p. D-3.)

• Delay in recognizing and responding to events,
• Bias for negative conclusions on Unreviewed Safety Question Determinations, and
• Infrequent presence of NWP management in the underground and surface.

Our Perspective

For starters, the Board appears to have a limited view of what SC is.  They see it as a cause for many of WIPP's problems but it can be fixed if it is “nurtured by strong leadership” and the report's recommendations are implemented.  The recommendations are familiar and can be summed up as “Row harder!”***  In reality, SC is both cause (it creates the context for decision making) and consequence (it is influenced by the observed actions of all organization members, not just senior management).  SC is an organizational property that cannot be managed directly.  

The report is a textbook example of linear, deterministic thinking, especially Appendix E (46 pgs.) on events and causal factors related to the incident.  The report is strong on what happened but weak on why things happened.  Going through Appendix E, SC is a top-level blanket cause of nuclear safety program and radiological event shortcomings (and, to a lesser degree, ventilation, CAMs and ground control problems) but there is no insight into how SC interacts with other organizational variables or with WIPP’s external (political, regulatory, DOE policy) environment. 

Here’s an example of what we’re talking about, viz., how one might gain some greater insight into a problem by casting a wider net and applying a bit of systems thinking.  The report faults DOE HQ for ineffective oversight, providing inadequate resources and not holding CBFO accountable for performance.  The recommended fix is for DOE HQ “to better define and execute their roles and responsibilities” for oversight and other functions. (p. ES-21)  That’s all what and no why.  Is there some basic flaw in the control loop involving DOE HQ, CBFO and NWP?  DOE HQ probably believes it transmits unambiguous orders and expectations through its official documents—why weren’t they being implemented in the field and why didn’t DOE know it?  Is the information flow from DOE to CBFO to NWP clear and adequate (policies, goals); how about the flow in the opposite direction (performance feedback, problems)?  Is something being lost in the translation from one entity to another?  Does this control problem exist between DOE HQ and other sites, i.e., is it a systemic problem?  Who knows.****

Are there other unexamined factors that make WIPP's problems more likely?  For example, has WIPP escaped the scrutiny and centralized controls that DOE applies to other entities?  As a consequence, has WIPP had too much autonomy to adjust its behavior to match its perception of the task environment?  Are DOE’s and WIPP’s mental models of the task environment similar or even adequate?  Perhaps WIPP (and possibly DOE) see the task environment as simpler than it actually is, and therefore the strategies for handling the environment lack requisite variety.  Was there an assumption that NWP would continue the apparently satisfactory performance of the previous contractor?  It's obvious these questions do not specifically address SC but they seek to ascertain how the organizations involved are actually functioning, and SC is an important variable in the overall system.

Contrast with Other DOE SC Investigations 

This report presents a sharp contrast to the foot-dragging that takes place elsewhere in DOE.  Why can’t DOE bring a similar sense of urgency to the SC investigations it is supposed to be conducting at its other facilities?  Was the WIPP incident that big a deal (because it involved a radioactive release) or is it merely something that DOE can wrap its head around?  (After all, WIPP is basically an underground warehouse.)  In any event, something rang DOE’s bell because they quickly assembled a 5 member board with 16 advisor/consultants and produced a 300 page report in less than two months.*****

Bottom line: You don't need to pore over this report but it provides some perspective on how DOE views SC and demonstrates that a giant agency can get moving if it's motivated to do so.


*  DOE Office of Environmental Management, “Accident Investigation Report: Radiological Release Event at the Waste Isolation Pilot Plant on February 14, 2014, Phase 1” (April 2014).  Retrieved April 30, 2014.  Our thanks to Mark Lyons who posted this report on the LinkedIn Nuclear Safety group discussion board.

**  NWP LLC was formed by URS Energy and Construction, Inc. and Babcock & Wilcox Technical Services Group, Inc.  Their major subcontractor is AREVA Federal Services, LLC.  All three firms perform work at other, i.e., non-WIPP, DOE facilities.  NWP assumed management of WIPP on Oct. 1, 2012.  From NWP website.  Retrieved May 2, 2014.

***  To the Board's credit, they did not go looking for individual scapegoats to blame for WIPP's difficulties.

****  In fairness, the report has at least one example of a feedback loop in the CBFO-NWP sub-system: CBFO's use of the condition reports as an input to NWP’s compensation review and NWP's predictable reaction of creating fewer condition reports.

*****  The Accident Investigation Board was appointed on Feb. 27, 2014 and completed its Phase 1 investigation on March 28, 2014.  The Phase 1 report was released to the public on April 22, 2014.