The Study of Organizational Culture: History, Assessment Methods, and Insights

We came across an academic journal article* that purports to describe the current state of research into organizational culture (OC).  It’s interesting because it includes a history of OC research and practice, and a critique of several methods used to assess it.  Following is a summary of the article and our perspective on it, focusing on any applicability to nuclear safety culture (NSC).

History

In the late 1970s scholars studying large organizations began to consider culture as one component of organizational identity.  In the same time frame, practicing managers also began to show an interest in culture.  A key driver of their interest was Japan’s economic ascendance and descriptions of Japanese management practices that depended heavily on cultural factors.  The notion of a linkage between culture and organizational performance inspired non-Japanese managers to seek out assistance in developing culture as a competitive advantage for their own companies.  Because of the sense of urgency, practical applications (usually developed and delivered by consultants) were more important than developing a consistent, unified theory of OC.  Practitioners got ahead of researchers and the academic world has yet to fully catch up.

Consultant models only needed a plausible, saleable relationship between culture and organizational performance.  In academic terms, this meant that a consultant’s model relating culture to performance only needed some degree of predictive validity.  Such models did not have to exhibit construct validity, i.e., some proof that they described, measured, or assessed a client organization’s actual underlying culture.  A second important selling point was the consultants’ emphasis on the singular role of the senior leaders (i.e., the paying clients) in molding a new high-performance culture.

Over time, the emphasis on practice over theory and the fragmented efforts of OC researchers led to some distracting issues, including the definition of OC itself, the culture vs. climate debate, and qualitative vs. quantitative models of OC. 

Culture assessment methods 

The authors provide a detailed comparison of four quantitative approaches for assessing OC: the Denison Organizational Culture Survey (used by more than 5,000 companies), the Competing Values Framework (used in more than 10,000 organizations), the Organizational Culture Inventory (more than 2,000,000 individual respondents), and the Organizational Culture Profile (OCP, developed by the authors and used in a “large number” of research studies).  We’ll spare you the gory details but unsurprisingly, the authors find shortcomings in all the approaches, even their own. 

Some of this criticism is sour grapes over the more popular methods.  However, the authors mix their criticism with acknowledgement of functional usefulness in their overall conclusion about the methods: because they lack a “clear definition of the underlying construct, it is difficult to know what is being measured even though the measure itself has been shown to be reliable and to be correlated with organizational outcomes.” (p. 15)

Building on their OCP, the authors argue that OC researchers should start with the Schein three-level model (basic assumptions and beliefs, norms and values, and cultural artifacts) and “focus on the norms that can act as a social control system in organizations.” (p. 16)  As controllers, norms can be descriptive (“people look to others for information about how to act and feel in a given situation”) or injunctive (how the group reacts when someone violates a descriptive norm).  Attributes of norms include content, consensus (how widely they are held), and intensity (how deeply they are held).

Our Perspective

So what are we to make of all this?  For starters, it’s important to recognize that some of the topics the academics are still quibbling over have already been settled in the NSC space.  The Schein model of culture is accepted world-wide.  Most folks now recognize that a safety survey, by itself, only reflects respondents’ perceptions at a specific point in time, i.e., it is a snapshot of safety climate.  And a competent safety culture assessment includes both qualitative and quantitative data: surveys, focus groups, interviews, observations, and review of artifacts such as documents.

However, we may still make mistakes.  Our mental models of safety culture may be incomplete or misassembled, e.g., we may see a direct connection between culture and some specific behavior when, in reality, there are intervening variables.  We must acknowledge that OC can be a multidimensional sub-system with complex internal relationships interacting with a complicated socio-technical system surrounded by a larger legal-political environment.  At the end of the day, we will probably still have some unknown unknowns.

Even if we follow the authors’ advice and focus on norms, it remains complicated.  For example, it’s fairly easy to envision that safety could be a widely agreed upon, but not intensely held, norm; that would define a weak safety culture.  But how about safety and production and cost norms in a context with an intensely held norm about maintaining good relations with and among long-serving coworkers?  That could make it more difficult to predict specific behaviors.  However, people might be more likely to align their behavior around the safety norm if there was general consensus across the other norms.  Even if safety is the first among equals, consensus on other norms is key to a stronger overall safety culture that is more likely to sanction deviant behavior.
 
The authors claim culture, as defined by Schein, is not well-investigated.  Most work has focused on correlating perceptions about norms, systems, policies, procedures, practices and behavior (one’s own and others’) to organizational effectiveness with a purpose of identifying areas for improvement initiatives that will lead to increased effectiveness.  The manager in the field may not care if diagnostic instruments measure actual culture, or even what culture he has or needs; he just wants to get the mission accomplished while avoiding the opprobrium of regulators, owners, bosses, lawmakers, activists and tweeters. If your primary focus is on increasing performance, then maybe you don’t need to know what’s under the hood. 

Bottom line: This is an academic paper with over 200 citations but is quite readable although it contains some pedantic terms you probably don’t hear every day, e.g., the ipsative approach to ranking culture attributes (ordinary people call this “forced choice”) and Q factor analysis.**  Some of the one-sentence descriptions of other OC research contain useful food for thought and informed our commentary in this write-up.  There is a decent dose of academic sniping in the deconstruction of commercially popular “culture” assessment methods.  However, if you or your organization are considering using one of those methods, you should be aware of what it does, and doesn’t, incorporate. 


*  J.A. Chatman and C.A. O’Reilly, “Paradigm lost: Reinvigorating the study of organizational culture,” Research in Organizational Behavior (2016).  Retrieved May 28, 2019.

**  “Normal factor analysis, called "R method," involves finding correlations between variables (say, height and age) across a sample of subjects. Q, on the other hand, looks for correlations between subjects across a sample of variables. Q factor analysis reduces the many individual viewpoints of the subjects down to a few "factors," which are claimed to represent shared ways of thinking.”  Wikipedia, “Q methodology.”   Retrieved May 28, 2019.

“New” IAEA Nuclear Safety Culture Self-Assessment Methodology

IAEA report cover

The International Atomic Energy Agency (IAEA) touted its safety culture (SC) self-assessment methodology at the Regulatory Cooperation Forum held during the recent IAEA 61st General Conference.  Their press release* refers to the methodology as “new” but it’s not exactly fresh from the factory.  We assume the IAEA presentation was based on a publication titled “Performing Safety Culture Self-assessments”** which was published in June 2016 and we reviewed on Aug. 1, 2016.  We encourage you to read our full review; it is too lengthy to reasonably summarize in this post.  Suffice to say the publication includes some worthwhile SC information and descriptions of relevant SC assessment practices but it also exhibits some execrable shortcomings.


*  IAEA, “New IAEA Self-Assessment Methodology and Enhancing SMR Licensing Discussed at Regulatory Cooperation Forum” (Sept. 22, 2017).

**  IAEA, “Performing Safety Culture Self-assessments,” Safety Reports Series no. 83 (Vienna: IAEA, 2016).

Nuclear Safety Culture Update at Pilgrim and Watts Bar

Pilgrim

Watts Bar

A couple of recent reports address the nuclear safety culture (NSC) problems at Pilgrim and Watts Bar.  This post summarizes the reports and provides our perspective on their content.  Spoiler alert: there is not much new in this news.

Pilgrim

The NRC issued their report* on phase C of their IP 95003 inspection at Pilgrim.  This is the phase where the NRC conducts its own assessment of the plant’s NSC.  The overall finding in the cover letter is: “The NRC determined that programs and processes at PNPS [Pilgrim] adequately support nuclear safety and that PNPS should remain in Column 4.”  However, the letter goes on to detail a host of deficiencies.  The relative good news is that Pilgrim’s NSC shortcomings weren’t sufficiently serious or interesting to merit mention in the cover letter.

But the NRC had plenty to say about NSC in the main report.  Highlights include the finding that NSC is a “fundamental problem” at Pilgrim.  NSC gradually deteriorated over time and “actions to balance competing priorities, manage problems, and prioritize workload resulted in reduced safety margins.”  Staffing reduction initiatives exacerbated plant performance problems.  Personnel were challenged to exhibit standards and expectations in conservative decision-making, work practices, and procedure use and adherence.  Contributing factors to performance shortcomings include lack of effective benchmarking of industry standards and the plant’s planned 2019 permanent shutdown.  The NRC also noted weaknesses in the Executive Review Board, Employee Concerns Program and the Nuclear Safety Culture Monitoring Panel. (pp. 8-10)

Watts Bar

In April the TVA inspector general (IG) issued a report** castigating TVA management for allowing a chilled work environment (CWE) to continue to exist at Watts Bar.  The IG report’s findings included: TVA's analyses and its response to the NRC’s CWE letter were incomplete and inadequate; TVA's planned corrective actions are unlikely to have long-term effectiveness; precursors of the CWE went unrecognized by management; and management has inappropriately influenced the outcome of analyses and investigations pertaining to Watts Bar NSC/SCWE issues.  Staff stress, fear and trust issues also exist.

In response, TVA management pointed out the corrective actions that were taken or are underway since the first draft of the IG report was issued.  Additionally, TVA management “has expressly acknowledged management's role in creating the condition and its responsibility for correcting it."

Our Perspective

This is merely a continuation of a couple of sad stories we’ve been reporting on for a long time.  Click on the Entergy, Pilgrim, TVA or Watts Bar labels to get our earlier reports. 

The finding that Pilgrim did not adequately benchmark against industry standards is appalling.  Entergy operates a fleet of nuclear plants and they don’t know what industry standards are?  Whatever.  Entergy is closing all the plants they purchased outside their service territory, hopefully to increase their attention on their utility-owned plants (where Arkansas Nuclear One remains a work in progress). 

We applaud the TVA IG for shining a light on the agency’s NSC issues.  In response to the IG report, TVA management put out a typical mea culpa accompanied by claims that their current corrective actions will fix the CWE and other NSC problems.  Well, their prior actions were ineffective and these actions will also probably fall short.  It doesn’t really matter.  TVA is too big to fail, both politically and economically, and their nuclear program will likely continue to plod along forever.


*  D.H. Dorman (NRC) to J. Dent (TVA), “Pilgrim Nuclear Power Station – Supplemental Inspection Report (Inspection Procedure 95003 Phase ‘C’) 05000293/2016011 and Preliminary Greater-than-Green Finding” (May 10, 2017).  ADAMS ML17129A217.

**  TVA Inspector General, “NTD Consulting Group, LLC's Assessment of TVA's Evaluation of the Chilled Work Environment at Watts Bar Nuclear Plant - 2016-16702” (April 19, 2017).  Also see D. Flessner, “TVA inspector general says safety culture problems remain at Watts Bar,” Chattanooga Times Free Press (April 21, 2017).  Retrieved May 25, 2017.

Is TEPCO’s Nuclear Safety Culture Still Weak?

Cover of TEPCO Self-Assessment

Tokyo Electric Power Co. (TEPCO) recently conducted a self-assessment* (SA) to ascertain progress vis-à-vis the goals in their Nuclear Safety Reform Plan.  The SA covered both Fukushima Daiichi and the undamaged plants.  It was approved by TEPCO’s president and reported to the Nuclear Reform Monitoring Committee (NRMC), an independent group of experts that advises TEPCO’s board of directors.  The committee reviewed the SA and communicated their evaluation to the board.  This post reviews both the SA and committee documents focusing on findings and observations related to safety culture (SC).

The TEPCO Self-Assessment

Comments with actual or potential relevance for culture appear throughout the 20-page SA report and are summarized below.  It appears about half of the SA findings and concrete action plans could have some connection to organizational culture.

Scores of employees have received awards for achieving goals related to stronger nuclear safety culture (NSC) and senior managers have been emphasizing nuclear safety but weaknesses still exist in implementing all the traits of a strong NSC.  Unsafe behaviors with respect to industrial safety are being corrected and the need for stronger nuclear safety is being established.  However, communications to the worker level with respect to nuclear safety may be insufficient.  The importance of nuclear safety is emphasized in new employee training and in meetings with contractor representatives. (pp. 4-5)  This is a mixed bag in the part of the SA most likely to be concerned with SC, viz., “Management Reforms Prioritizing Nuclear Safety.”

The company is working on strengthening work processes to improve risk management but employees report processes are unchanged and no clear priorities are established, factors that may increase fatigue and decrease motivation, both of which challenge the development of NSC. (p. 6)

Stricter safety regulations are being implemented and are no longer resisted based on their cost or operating impact. At Fukushima, the emphasis when performing work has shifted from speed to risk reduction to proceed safely. (pp. 8-9)  All good news.

Daily meetings share information on operating experience and near-misses but the overall information set is not leveraged because it is not managed to lead to long-term improvements. (p. 10)  In other words, continuous improvement is still a goal, not a reality.

Training is good for teaching employees how to complete tasks but the curriculum is insufficient to cultivate and inculcate a high level of safety performance.  The need for a more systematic approach to training is recognized but has not been realized. (pp. 12-13)

Overall findings of the SA emphasize the need to enhance a questioning attitude, strengthen supervision, and upgrade education and training. (p. 20)

NRMC Report and Cover Letter

The NRMC reviewed the SA in a 7-page report.**  One improvement noted by the NRMC was “Safety culture awareness has permeated throughout the organization and has improved significantly.”  However, the first three items on the list of nine Recommendations deal with NSC:

“Consistent efforts should be made to build a strong nuclear safety culture and instill the nuclear safety culture in an organizational culture.  The need for formal training and/or professional facilitation for the managers should be evaluated to instill a strong safety culture in the organization. . . . a safety culture program should be developed to the same standards” for contractors. (p. 5) 

There is also a one-page cover letter to the report.  Its primary focus is SC:

“TEPCO has made significant progress but must not become complacent . . . . TEPCO should instill . . . a strong safety culture throughout the organization, . . . TEPCO is encouraged to take further actions for the safety culture alignment at all levels of the organization . . .”

Our Perspective

SC and NSC occupy much of the space in all these documents.  What should we make of that, if anything?  One possibility is SC is acceptable but can always be improved or strengthened.  After all, as the NRMC notes “any self-assessment process must be critical by nature and therefore should identify areas for future improvement.”

A darker possibility is that TEPCO’s SC is still weak.  The NRMC’s report doesn’t have the language we usually see in the typical U.S. NSC report which says or implies “The plant is operating safely (indicating the NSC is at least minimally acceptable) but has improvement opportunities.”  We have to ask the NRMC: Is TEPCO’s current NSC acceptable or not?  Everyone understands Fukushima Daiichi is not operating, in fact, it’s still a mess where finding a lost fuel rod is world-wide news, but are current clean-up efforts occurring in an adequately safety-conscious environment?  The disaster occurred in 2011; some of the shortcomings noted in the SA should have been squared away by now.

On a different note, how does the SA address some topics dear to us?  Goal conflict is addressed when safety is mentioned as the primary goal and improvements are being made without cost being a major consideration.  The corrective action program (CAP) is mentioned but only as a tool for implementing improvement in the operating experience program.  Decision making is not mentioned at all so we don’t know how safety is being integrated into the decision making process at any level.  Another mixed bag.

Bottom line: Is SC front and center in all these documents because it is not yet acceptable?


*  “Report on TEPCO’s Self-Assessment of Progress” (Jan. 2017).

**  Nuclear Reform Monitoring Committee, “Review of the TEPCO’s Self-Assessment Effort on Nuclear Safety Reform,” (Jan. 30, 2017).

***  Nuclear Reform Monitoring Committee, Cover letter to "Review of the TEPCO's Self-Assessment Effort on Nuclear Safety Reform," (Jan. 30, 2017).  The public versions of all these TEPCO documents are copy protected so quotes have been retyped.

Canadian Draft Regulation on Nuclear Safety Culture

Draft REGDOC cover

The Canadian Nuclear Safety Commission (CNSC) has published a draft regulatory document REGDOC-2.1.2, “Safety Culture” for comment*  The REGDOC will be a requirement for nuclear power plants and provide guidance for other nuclear entities and activities.  

The REGDOC establishes “requirements and guidance for fostering and assessing safety culture.” (p. 1)  The CNSC’s purpose is to promote a healthy safety culture (SC) which they say “is a key factor in reducing the likelihood of safety-related events and mitigating their potential impact, and in continually improving safety performance.” (ibid.)

Section 2 specifies five characteristics of a healthy SC: Safety is a clearly recognized value, accountability for safety is clear, a learning organization is built around safety, safety is integrated into all activities in the organization, and a safety leadership process exists in the organization.  For each characteristic, the document lists observable indicators. 

Sections 3 and 4 describe how licensees should perform SC assessments.  Specifically, assessments should be empirical, valid, practical and functional.  Each of these three characteristics is fleshed out with relevant criteria.  The document goes on to discuss the mechanics of performing assessments: developing a communications strategy, defining the assessment framework, selecting team members, planning and conducting assessments, developing findings and recommendations, writing reports, etc.

Our Perspective

The REGDOC is clear and relatively brief.  None of the content is controversial or even new; the document is based on multiple International Atomic Energy Agency (IAEA) publications.  (14 of 15 references in the document are from IAEA.  The “Additional Information” page includes items from INPO, NEI and WANO.)

Here’s how the REGDOC addresses SC topics that are important to us:

Decision making - Satisfactory

The introduction to the SC characteristics says “The highest level of governing documentation should make safety the utmost priority – overriding the demands of production and project schedules . . .” (p. 4)  The specific SC indicators include “Timely decisions are made that reflect the value and relative priority placed on safety.” (ibid.)  “Workers are involved in risk assessment and decision-making processes.” (p. 5)  “A proactive and long-term approach to safety is demonstrated in decision making.” (p. 6)  We would have liked a more explicit treatment of safety-production-cost goal conflict but what the CNSC has included is OK.

Taking a systems view of SC - Unacceptable

This topic is only mentioned in a table of SC maturity model indicators that is in an appendix to the REGDOC.  The links between SC and other important organizational attributes must be inferred from the observable indicators.  There is no discussion of the interrelationship between SC and other important organizational attributes, e.g., the safety conscious work environment, management’s commitment to safety, or workers’ trust in management to do the right thing.

Rewards and compensation - Unacceptable 

The discussion is limited to workers.  What about senior management compensation and incentives?  How much are senior managers paid, if anything, for establishing and maintaining a healthy SC?

The discussion on performing assessments refers several times to a SC maturity model that is appended to the REGDOC.  The model has three stages of organizational maturity—requirement driven, goal driven and continually improving, along with specific observable behaviors associated with each stage.  The model can be used to “describe and interpret the organization’s safety culture, . . .” (p. 10)  Nowhere does the REGDOC explicitly state that stage 3 (a continually improving organization) is the desired configuration.  This is a glaring omission in the REGDOC.

Bottom line: If you keep up with IAEA’s SC-related publications, you don’t need to look at this draft REGDOC which adds zero value to our appreciation or understanding of SC.


*  Canadian Nuclear Safety Commission, draft regulatory document REGDOC-2.1.2, “Safety Culture” (Sept. 2016).  The CNSC is accepting public comments on the document until Jan. 31, 2017.

Nuclear Safety Culture Self-Assessment Guidance from IAEA

IAEA report cover

The International Atomic Energy Agency (IAEA) recently published guidance on performing safety culture (SC) self-assessments (SCSAs).  This post summarizes the report* and offers our perspective on its usefulness.

The Introduction presents some general background on SC and specific considerations to keep in mind when conducting an SCSA, including a “conscious effort to think in terms of the human system (the complex, dynamic interaction of individuals and teams within an organization) rather than the technological system.” (p. 2)  Importantly, an SCSA is not based on technical skills or nuclear technology, nor is it focused on immediate corrective actions for observed problems.

Section 2 provides additional information on SC, starting with the basics, e.g., culture is one way of explaining why things happen in organizations.  The familiar iceberg model is presented, with the observable artifacts above the surface and the national, ethnic and religious values that underlie culture way below the waterline.  Culture is robust (it cannot be changed rapidly) and complicated (subcultures exist).  So far, so good.

Then things start to go off the rails.  The report reminds us that the IAEA SC framework** has five SC characteristics but then the report introduces, with no transition, a four-element model for envisioning SC; naturally, the model elements are different from the five SC characteristics previously mentioned.  The report continues with a discussion of IAEA’s notion of “shared space,” the boundary area where working relationships develop between the individual and other organizational members.  We won’t mince words: the four-component model and “shared space” are a distraction and zero value-added.

Section 3 explores the characteristics of SCSAs.  Initially, an SCSA focuses on developing an accurate description of the current culture, the “what is.”  It then moves on to evaluating a SC’s strengths and weaknesses by comparing “what is” with “what should be.”  An SCSA is different from a typical audit in numerous ways, including the need for specialized training, a focus on organizational dynamics and an understanding of the complex interplay of multicultural dimensions of the organization.

SCSAs require recognition of the biases present when a culture examines itself.  Coupling this observation with an earlier statement that effective SCSAs require understanding of the relevant social sciences, the report recommends obtaining qualified external support personnel (at least for the initial efforts at conducting SCSAs).  In addition, there are many risks (the report comes up with 17) associated with performing an SCSA that have to be managed.  All of these aspects are important and need to be addressed.

Section 4 describes the steps in performing an SCSA.  The figure that purportedly shows all the steps is unapproachable and unintelligible.  However, the steps themselves—prepare the organization, the team and the SCSA plan; conduct the pre-launch and the SCSA; analyze the results; summarize the communicate the findings; develop actions; capture lessons learned; and conduct a follow-up—are reasonable.

The description of SCSA team composition, competences and responsibilities is also reasonable.  Having a team member with a behavioral science background is highly desirable but probably not available internally in other than the largest organizations. 

Section 5 covers SCSA methods: document review, questionnaires, observations, focus groups and interviews.  For each method, the intent, limitations and risks, and intended uses are discussed.  Each method requires specific skills.  The purpose is to develop an overall view of the culture.  Because of the limitations of individual methods, multiple (and preferably all) methods should be used.  Overall, this section is a pretty good high-level description of the different investigative methods.

Section 6 describes how to perform an integrated analysis of the information gathered.  This involves working iteratively with parallel information sets.  There is a lengthy discussion of how to develop cultural themes from the different data sources.  Themes are combined into an overall descriptive view of the culture which can then be compared to the IAEA SC framework (a normative view) to identify relative strengths and weaknesses, and improvement opportunities.

Section 7 describes approaches to communicating the findings and transitioning into action.  It covers preparing the SCSA report, communicating the results to management and the larger organization, possible barriers to implementing improvement initiatives and maintaining continuous improvement in an organization’s SC.

The report has an extensive set of appendices that illustrate how an SCSA can be conducted.  Appendix I is a laundry list of potential areas for inquiry.  Appendices II-VIII present a case study using all the SCSA methods in Section 5, followed by some example overall conclusions.  Appendix IX is an outline of an SCSA final report.  The guidance on using the SCSA methods is acceptably complete and clear.

A 28-page Annex (including 8 pages of references) describes the social science underlying the recommended methodology for performing SCSAs.  It covers too much ground to be summarized here.  The writing is uneven, with some topics presented in a fluid style (probably a single voice) while others, especially those referring to many different sources, are more ragged.  Because of the extensive use of in-line references, the reader can easily identify source materials.   

Our Perspective

There’s good news and bad news in this Safety Report.  The good news is that when IAEA collates and organizes the work of others, e.g., academics, SC practitioners or industry best practices, IAEA can create a readable, reasonably complete reference on a subject, in this case, SCSA.

The bad news is that when IAEA tries to add new content with their own concepts, constructs, figures and such, they fail to add any value.  In fact, they detract from the total package.  It seems to never have occurred to the IAEA apparatchiks to circulate their ideas for new content for substantive review and comment.


*  International Atomic Energy Agency, “Performing Safety Culture Self-assessments,” Safety Reports Series no. 83 (Vienna: IAEA, 2016).  Thanks to Madalina Tronea for publicizing this report.  Dr. Tronea is the founder/moderator of the LinkedIn Nuclear Safety Culture discussion group.

**  Interestingly, the IAEA SC framework (SC definition, key characteristics and attributes) is mentioned without much discussion; the reader is referred to other IAEA documents for more details.  That’s OK.  For purposes of SCSA, it’s only important that the organization, including the SCSA team, agree on a SC definition and its associated characteristics and attributes.  This will give everyone involved a shared normative view for linking the SCSA findings to a picture of what the SC should look like.

Nuclear Safety Culture at ANO—the NRC Weighs In

Arkansas Nuclear One (credit: Edibobb)

On June 25, 2015 we posted about Arkansas Nuclear One’s (ANO) performance problems (a stator drop, inadequate flood protection and unplanned scrams) and the Nuclear Regulatory Commission’s (NRC's) reaction.  The NRC assigned ANO to column 4 of the Action Matrix where it receives the highest level of oversight for an operating plant.  As part of this increased oversight, the NRC conducted a comprehensive inspection of ANO performance, programs and processes.  A lengthy inspection report* was recently issued.

According to the NRC press release** the inspection team identified the following major issues:

“Resource reductions and leadership behaviors were the most significant causes for ANO’s declining performance. . . . ANO management did not reduce workloads through efficiencies or the elimination of unnecessary work, . . . Leaders . . . did not address expanding work backlogs***. . . . An unexpected increase in employee attrition between 2012 and 2014 caused a loss in experienced personnel, . . . Since 2007, the reduced resources created a number of changes that slowly began to impact equipment reliability.  The Entergy fleet reduced preventive maintenance and extended the time between some maintenance activities.”

The press release goes on to list numerous ANO corrective actions and NRC observations that suggest the potential for improved plant performance.

What About ANO’s Safety Culture?

The press release also mentions that the inspection team evaluated the adequacy of a 2015 Third Party Nuclear Safety Culture Assessment (TPNSCA) conducted at ANO.  The press release gives short shrift to the key role a weak safety culture (SC) played in creating ANO’s problems in the first place and the extensive SC questions raised and diagnostics performed by the NRC inspection team.

Last June, based on NRC and ANO meeting presentations, we concluded “the ANO culture endorses a “blame the contractor” attitude, accepts incomplete investigations into actual events and potential problems, and is content to let the NRC point out problems for them.”  These are serious deficiencies.  Do the same or similar problems appear in the inspection report?  To answer that question, we need to dig into the details of the 243 page report.

The Cover Letter

Top-level SC problems are included in the NRC cover letter which says “The inspection team identified what it considered to be missed opportunities for ANO to have promptly initiated performance improvements since being placed in Column 4.  More specifically, ANO: 1) was slow to implement corrective actions to address the findings from the Corrective Action Program cause evaluation and the Third Party Nuclear Safety Culture Assessment; 2) did not perform an evaluation of the causes for safety culture problems; . . .” (letter, p. 2)

Executive Summary

The report's Executive Summary says “The Third Party Nuclear Safety Culture Assessment identified that ANO personnel tolerated, and at times normalized, degraded conditions.”  Expanding on the missed opportunities comment in the cover letter, “the NRC team’s independent safety culture evaluation noted limited improvement in safety culture since the completion of ANO’s independent Third Party Nuclear Safety Culture Assessment.” (report p. 5)  “ANO did not create a specific improvement plan to address the findings of the safety culture assessments, choosing to address selected safety culture attributes that were associated with root cause evaluations rather than treating the findings in the context of a separate problem area.  By not performing a cause evaluation for safety culture, ANO management missed the opportunity to address the full scope of safety culture weaknesses.” (pp. 5-6)

Review of ANO Recovery Plan 

The NRC’s critique of ANO’s Recovery Plan included “The NRC team questioned the recovery team’s decision not to perform casual evaluations of the PAs [Problem Areas].  In response, ANO performed apparent cause evaluations (ACEs) or gap analyses for each PA.  The NRC team questioned the recovery team’s decision not to perform causal evaluations for the safety culture attributes identified in [a 2014] . . . safety culture survey, the TPNSCA, and the RCEs [Root Cause Evaluations].  The team also questioned the recovery team’s decision not to treat safety culture as a separate problem area.” (p. 21)

This is an example where the NRC was still identifying ANO’s overarching problems for the plant staff.

Review of RCEs for Fundamental Problem Areas

“ANO’s Vendor Oversight RCE identified weak implementation of administrative controls and placing undue confidence in vendor services as common cause failures. However, ANO did not assess the underlying safety culture aspects.” (p. 110, emphasis added)

This is not “blame the vendor” but is a different serious problem, viz., an over-reliance on vendor activities to protect the customer.  (This problem is not unique to ANO; it also might exist at the Waste Isolation Pilot Plant.  See our May 3, 2016 post for details.)

Inspection Report Chapter on SC

The NRC team conducted its own assessment of ANO’s SC. The NRC team interviewed personnel at all levels, conducted focus group discussions, performed behavioral observations, reviewed documents and relevant plant programs, and evaluated plant management meetings.  Overall, they assessed all ten SC traits using the full set of SC attributes contained in NRC documentation.  For each trait, the report includes its attributes, inspection team observations and findings, and relevant ANO corrective actions.

The team also reviewed seven RCEs and concluded ANO addressed the major SC attributes identified in each RCE.  However, “The NRC team noted that ANO identified that some safety culture attributes were contributors to several of the RCE problem statements, but ANO did not consider the collective significance.” (p. 184)

ANO took the hint.  “In response to the NRC team’s concerns, ANO performed a common cause analysis of all of the safety culture attributes that were identified in the recovery RCEs in order to assess the collective significance and causes.” (p. 185)  ANO developed a SC Area Action Plan (AAP) and the NRC concluded “The corrective actions identified in the NSC AAP were comprehensive and appropriate to address the causes for safety culture weaknesses.” (p. 186)

“The NRC team’s graded safety culture assessment independently confirmed the results from the TPNSCA.” (p. 188)

“The NRC team was concerned that the SCLT’s [Safety Culture Leadership Team, senior managers] conclusion that ANO’s safety culture was “adequate” in August 2015 did not appropriately reflect the data provided by, or the recommendations from, the NSCMP [Nuclear Safety Culture Monitoring Panel, mid-level personnel].  This SCLT conclusion did not reflect the declining condition with respect to safety culture and indicated a lack of awareness that improvements in safety culture at ANO were needed.”  The SCLT eventually came around and in December 2015 declared that ANO’s SC was not acceptable. (p. 192)

Our Perspective

The NRC is optimistic that ANO has correctly identified the root causes of its performance problems and has undertaken corrective actions that will ultimately prove effective.  We hope so but we’ll go with “trust but verify” on this one.  ANO still exhibits problems with incomplete analyses and leaning on the NRC to identify systemic deficiencies.

The NRC team took a good look at ANO's SC.  Quite frankly, their effort was more comprehensive than we expected.  They used an acceptable methodology for their SC assessment.  The fact that their assessment findings were consistent with the TPNSCA is not surprising.  SC evaluation is a robust social science activity and qualified SC evaluators using similar techniques should obtain generally comparable results.

We believe the NRC’s SC professionals are qualified and competent but probably encouraged to support the overall inspection findings.  The elephant in the room is that SC is a policy, not a regulation.  Would the NRC keep a plant in column 4 based solely on their belief that the plant SC is deficient?  Look at the contortions the agency performed at Palisades as that plant’s SC somehow went from weak, with constant problems, to “improving” and, we inferred, acceptable.  (See our Jan. 30, 2013 post for details.)

There may have been a bit of similar magical thinking at ANO.  In the inspection report, every SC trait had examples of shortcomings but also had “appropriate” corrective actions to improve performance.****  How can this be when ANO (and Entergy) have been so slow to grasp the systemic nature of their SC problems?

Let’s close on a different note.  Earlier this year ANO named a full-time SC manager, a person whose background is in plant security.  On the surface, this is an “unfiltered” choice.  (See our March 10, 2016 post for a discussion of filtering in personnel decisions.)  He may be exactly the type of person ANO needs to make SC improvements happen.  We wish him well.


*  M. L. Dapas (NRC) to J. Browning (ANO), “Arkansas Nuclear One – NRC Supplemental Inspection Report 05000313/2016007 and 05000368/2016007” (June 9, 2016).  ADAMS ML16161B279.

**  V. Dricks, Press Release, “NRC Issues Comprehensive Inspection Report on Arkansas Nuclear One” (June 13, 2015).

***  We have often noted that large backlogs, especially of safety-related work, are an artifact of a weak SC.

****  One trait was judged to have no significant issues so corrective action was not needed.

Targeted Safety Culture Assessment at Columbia Generating Station

Columbia Generating Station

The Columbia Generating Station (CGS) got into trouble with the NRC when two members of the security department were found to have been willfully inattentive to their duties on multiple occasions over three years (2012-2014).  What they were doing was not disclosed because it was a security-related matter.  The situation was summarized in a recent newspaper article* and the relevant NRC documents** provide some additional details.

Energy Northwest (CGS’s owner) opted for the NRC’s Alternative Dispute Resolution (ADR) process.  The agreed-upon corrective actions and penances are typical for ADR settlements: conduct a common cause evaluation, install new cameras and increase supervision if the cameras aren’t working, revise and present training, prepare a statement on willful misconduct’s consequences and have personnel sign it, prepare a "lessons learned" presentation for plant personnel and an industry gathering (aka public atonement), revise procedures and conduct a targeted nuclear safety culture (SC) assessment of the security organization at CGS.  Oh, and pay a $35K fine.

Our Perspective

The security SC assessment caught our eye because it is being conducted by a law firm, not a culture assessment specialist.  Maybe that’s because the subject is security-related, therefore sensitive, and this approach will ensure the report will never be made public.  It also ensures that the report will focus on the previously identified “bad apples” (who no longer work at the plant) and the agreed-upon ADR actions; the assessment will not raise any awkward management or systemic issues.


*  A. Cary, “Energy Northwest pays fine over Richland nuclear security,” Tri-City Herald (Jan. 5, 2015.)

**  A. Vegel (NRC) to M. Reddermann (Energy NW), Columbia Generating Station – NRC Security Inspection Report 05000397/2015405 and NRC Investigation Report No. 4-2014-009 (June 25, 2015).  ADAMS ML15176A599.  M. Dapas (NRC) to M. Reddermann (Energy NW), Confirmatory Order - NRC Security Inspection Report 05000397/2015407 AND NRC Investigation Report 4-2014-009 Columbia Generating Station (Sept. 28, 2015).  ADAMS ML15271A078.

New Safety Culture Assessment at the Hanford Waste Treatment Plant

Hanford WTP

The Department of Energy (DOE) recently released the latest safety culture (SC) assessment report* for the Hanford Waste Treatment Plant (WTP or “vit plant”) project.  The 2015 report follows similar SC assessments conducted in 2011 and 2014, all of which were inspired by the Defense Nuclear Facilities Safety Board’s scathing 2011 report on SC at the WTP.  This post provides a brief overview of the report’s findings then focuses on the critical success factors for a healthy SC.

Assessment Overview

The 2011, 2014 and 2015 assessments used the same methodology, with multiple data collection methods, including interviews, Behavioral Anchored Rating Scales (BARS)** and a SC survey.  Following are selected highlights from the 2015 report.

DOE’s Office of River Protection (ORP) has management responsibility for the WTP project.  In general, ORP personnel feel more positive about the organization’s SC than they did during the 2014 assessment.  Feelings of confusion about ORP’s more collaborative relationship with Bechtel (the prime contractor) have lessened.  ORP management is perceived to be more open to constructive criticism.  Concerns remain with lack of transparency, trust issues and the effectiveness of the problem resolution process.

Bechtel personnel were more positive than in either previous SC assessment.  Bechtel has undertaken many SC-related initiatives including the promotion of a shared mental model of the project by senior Bechtel managers.  In 2014, Bechtel Corporate’s role in project decision making was perceived to skew against SC concerns.  The creation of a new Bechtel nuclear business unit has highlighted the special needs of nuclear work. (pp. 2, 39)  On the negative side, craft workers remain somewhat suspicious and wary of soft retributions, e.g., being blamed for their own industrial mishaps or having their promotion or layoff chances affected by reporting safety issues.

See this newspaper article*** for additional details on the report’s findings. 

Critical Success Factors for a Healthy SC

We always look at the following areas for evidence of SC strength or weakness: management’s decision making process, recognition and handling of goal conflicts, the corrective action program and financial incentives.

Decision Making

Both ORP and Bechtel interviewees complained of a lack of basis or rationale for different types of decisions. (pp. 9, 16)  Some ORP and Bechtel interviewees did note that efforts to clarify decision making are in process. (pp. 13, 32)  Although the need to explain the basis for decisions was recognized, there was no discussion of the decision making process itself.  This is especially disappointing because decision making is one of the possible behaviors that can be included in a BARS analysis, but was not chosen for this assessment.

Goal Conflicts

Conflicts among cost, schedule and safety goals did not rise to the level of a reportable problem.  ORP interviewees reported that cost and schedule do not conflict with safety in their individual work. (p. 6)  Most Bechtel interviewees do not perceive schedule pressures to be the determining factor while completing various tasks. (p. 23)  Overall, this is satisfactory performance.

Corrective Action Program

We believe how well an organization recognizes and permanently resolves its problems is important.  Problem Identification and Resolution was one of the traits evaluated in the assessment.  ORP interviewees said that current safety concerns are being addressed.  The historical lack of management feedback on problem resolution is still a disincentive for reporting problems. (pp. 8-9)  Some Bechtel interviewees said “issue resolution with management engagement was the single most positive improvement in problem resolution, . . .” (p. 24)  This performance is minimally acceptable but needs ongoing attention.

Financial Incentives

DOE’s contract with Bechtel now includes incentives for Bechtel if it self-identifies problems (rather than waiting for DOE or some other party to identify them).  ORP believes the incentives are a positive influence on contractor performance. (p. 8)  Bechtel interviewees also believe the new contract has had a positive impact on the project.  However, Bechtel has a goal to reduce legacy issues and some believe the contract’s emphasis on new issues distracts from addressing legacy problems. (pp. 24-25)  The assessment had no discussion of either ORP or Bechtel senior management financial incentives.  The new contract conditions are good; ignoring senior management incentives is unacceptable.

Safety Conscious Work Environment (SCWE)

We usually don’t pay much attention to SCWE at nuclear power plants because it is part of the larger cultural milieu.  But SCWE has been a long-standing issue at various DOE facilities, as well as the impetus for the series of WTP SC assessments, so we’ll look at a few highlights from the SC survey data.

For ORP, mean responses to five of the six SCWE questions were higher (better) in 2015 vs 2014, and 2014 vs 2011.  However, for one question “Concerns raised are addressed” the mean is lower (worse) in 2015 vs 2014, and significantly lower in 2015 vs 2011.  This may indicate an issue with problem resolution. (p. B-2) 

For Bechtel, mean responses to all six SCWE questions were significantly higher (better) in 2015 vs 2014.  However, the 2011 data were not included so we cannot make any inference about possible longer-term trends. (p. B-5)  What is shown is good news because it appears people feel freer to raise safety concerns.  Interestingly, Bechtel’s mean 2015 responses were 5-13% higher (better) than ORP’s for all questions.

Both ORP and Bechtel are showing acceptable performance but continued improvement efforts are warranted.

Our Perspective

The Executive Summary and Conclusions suggest ORP and especially Bechtel have turned the corner since 2014. (pp. v, 37)  This is arguably true for SCWE but we’d say the jury is still out on improvement in the broader SC, based on our look at the BARS data.

For ORP, the BARS data mean scores are higher for 4 (out of 10) behaviors in 2015 vs 2014, but only higher for 1 behavior in 2015 vs 2011. (p. B-1)  The least charitable interpretation is ORP’s view of itself has not yet re-achieved 2011 levels.  For Bechtel the BARS data shows a bit brighter picture.  Mean scores are higher for 6 (out of 10) behaviors in 2015 vs 2014, and higher for 4 behaviors for 2015 vs 2011. (p. B-4)

The format of the report is probably intended to be reader-friendly but it mixes qualitative interview data and selected quantitative data from BARS and the survey.  The use of modifiers like “many” and “some” creates a sense of relative frequency or importance but no real specificity.  It’s impossible to say how much (if any) cherry picking of the interview data occurred.****

We also wonder about the evaluation team’s level of independence and optimism.  This is the first time DOE has performed a WTP SC assessment without the extensive use of outside consultants.  Put bluntly, how independent was the team’s effort given DOE Headquarters’ desire to see improvements at WTP?  And it’s not just HQ; DOE is under the gun from Congress, the DNFSB, the Government Accountability Office, and environmental activists and regulators to clean up their act at Hanford.

We want to see a stronger SC at Hanford but we’ll go with Ronald Reagan on this report: “Trust, but verify.”


*  DOE Office of Enterprise Assessments, “Follow-up Assessment of Safety Culture at the Hanford Site Waste Treatment and Immobilization Plant” (June, 2015).  We have followed the WTP saga for years; please click on the Vit Plant label to see our related posts.

**  Behavioral Anchored Rating Scales (BARS) quantitatively summarize interviewees’ perceptions of their organization using specific examples of good, moderate, and poor performance.   There are 17 possible organizational behaviors in a BARS analysis, but only 10 were used in this assessment:  Attention to Safety, Coordination of Work, Formalization, Interdepartmental Communication, Organizational Learning, Performance Quality, Problem Identification and Resolution, Resource Allocation, Roles and Responsibilities and Time Urgency. (p. C-2)

***  A. Cary, “DOE: Hanford vit plant safety culture shows improvement,” Tri-City Herald (June 26, 2015).

****  The report also includes multiple references to the two organizations’ behavioral norms that were inferred from the survey data.  It’s not exactly consultant mumbo-jumbo but it’s too complicated to attempt to explain in this space.

Safety Culture at Arkansas Nuclear One

Arkansas Nuclear One (credit: Edibobb)

Everyone has heard about the March 31, 2013 stator drop at Arkansas Nuclear One (ANO).  But there was also unsatisfactory performance with respect to flood protection and unplanned scrams.  As a consequence, ANO has been assigned to column 4 of the NRC’s Action Matrix where it will receive the highest level of oversight for an operating plant.

When a plant is in column 4 the NRC takes a particular interest in its safety culture (SC) and ANO is no exception.  NRC required ANO to have an independent (i.e., outside third party) SC assessment, which was conducted starting in late 2014.  While the assessment report is not public, some highlights were discussed during the May 21, 2015 NRC staff briefing of the Commissioners on the results of the April 15, 2015 Agency Action Review Meeting.*

NRC Presentation

The bulk of the staff presentation was a soporific review of agency progress in a variety of areas.  But when the topic turned to ANO, the Regional Administrator responsible for ANO was quite specific and minced no words.  Following are the key problems he reviewed.  See if you can connect the dots on SC issues based on these artifacts.

Let’s start with the stator drop.  ANO’s initial root cause evaluation did not identify any root or contributing causes related to ANO’s own performance, but rather focused solely on the contractor.  After the NRC identified ANO’s failure to follow its load handling procedure, ANO conducted another root cause evaluation and identified their own organizational performance issues such as inadequate project oversight and non-conservative decision making. (pp. 28-29)

The stator drop damaged a fire main which caused localized flooding.  This led to an extended condition review which identified various equipment and structures that could be subject to flooding.  The NRC inspectors pointed out deficiencies in the condition review and identified corrective actions that likely would not work.  In addition, earlier flooding walkdowns completed as part of the NRC’s post-Fukushima requirements failed to identify the majority of the flood protection deficiencies.  These walkdowns were also performed by a contractor.  (pp. 29-31)

Finally, ANO did not report an April 2014 Unit 2 trip as an unplanned scram because the trip occurred during a planned down power evolution.  After prodding by the NRC inspectors, ANO reclassified this event as an unplanned scram. (pp. 31-32)

Overall, the NRC felt it was driving ANO to perform complete evaluations and develop effective corrective actions.  NRC believes that ANO’s “cause evaluations typically don't provide for a thorough assessment of organizational and programmatic contributors to events or issues.” (p. 35)  Later, in response to a question, the Regional Administrator said “I think the licensee clearly needs to own the performance gaps, ensure that their assessments in the various areas are comprehensive and then identify appropriate actions, and then engage and ensure those actions are effective. . . . I don't want to be in a position where our inspection activities are the means for identifying the performance gaps.” (p. 44)

Responding to a question about ANO’s independent SC assessment, he said “one of the key findings . . . was that there's an urgent need to internalize and communicate the seriousness of performance problems and engage the site in their strategy for improvement.” (p. 45)

Entergy Presentation

A team of Entergy (ANO’s owner) senior managers presented their action plan for ANO.  They said they would own their own problems, improve contractor oversight, identify their own issues, increase corporate oversight and improve their CAP.

With respect to culture, they said “We're going to change the culture to promote a healthy, continuous improvement and to not only achieve, but also to sustain excellence.” (pp. 70-71)  They are benchmarking other plants, analyzing ANO’s issues and adding resources including people with plant performance recovery experience. 

They took comfort from the SC assessment conclusion “That although weaknesses exist, the overall safety culture at ANO is sufficient to support safe operation." (p. 72)

In response to a question about important takeaways from the SC assessment, Entergy referred to the need for the plant to recognize that performance has got to improve, the CAP must be more effective and organizational programmatic elements are important.  In addition, they vowed to align the organization on the performance gaps (and their significance) and establish a sense of urgency in order to fix them. (pp. 80-81)

Our Perspective

Not to be too cynical, but what else could Entergy say?  When your plant is in column 4, a mega mea culpa is absolutely necessary.  But Entergy’s testimony read like generic management arm-waving invoking the usual set of fixes.

Basically, the ANO culture endorses a “blame the contractor” attitude, accepts incomplete investigations into actual events and potential problems, and is content to let the NRC point out problems for them.  Where did those values come from?  Is “increased oversight” sufficient to create a long-term fix?

ANO naturally gives a lot of weight to the SC assessment because its findings appear relatively simple and apparently actionable.   Somewhat surprisingly, the NRC also appears to give this assessment broad credibility.  We think that’s misplaced.  The chances are slim of such an assessment identifying deep, systemic cultural issues although we admit we don’t know the assessment details.  Did the assessment team perform document reviews, conduct focus groups or interviews?  If it was a survey, it only identified the most pressing issues in the plant’s safety climate.

Taking a more systemic view, we note that Entergy has a history of SC issues over many plants in its fleet.  Check out our Feb. 20, 2015 post for highlights on some of their problems.  Are ANO’s problems just the latest round of SC Whac-A-Mole at Entergy?

Entergy has always had a strong Operations focus at its plants.  The NRC’s confidence in ANO’s operators is the main reason that plant is not shut down.  But continuously glorifying the operators, particularly their ability to respond successfully to challenging conditions, is like honoring firefighters while ignoring the fire marshal.  The fire marshal role at a nuclear plant is played by Engineering and Maintenance, groups whose success is hidden (thus under-appreciated) in an ongoing series of dynamic, non-events, viz., continuous safe plant operation.  That’s a cultural issue.  By the way, who gets the lion’s share of praise and highest status at your plant?


*  “Briefing on Results of the Agency Action Review Plan Meeting,” public meeting transcript (May 21, 2015).  ADAMS ML15147A041.

The Agency Action Review Meeting (AARM) “is a meeting of the senior leadership of the agency, and its goals are to review the appropriateness of agency actions taken for reactor material licensees with significant performance issues.” (pp. 3-4)

Safety Culture at the 2015 NRC Regulatory Information Conference

NRC Public Meeting

The Nuclear Regulatory Commission (NRC) held its annual Regulatory Information Conference (RIC) on March 10-12, 2015.  As usual, safety culture (SC) played a minor supporting role: it was the topic of one technical session out of 37 total.  The SC session focused on assessing and/or measuring SC.  It featured a range of presentations—from NRC, Duke Energy, DOE and a SC consultant—which are summarized below.*

NRC

This presentation consisted of one (sic) slide recounting the NRC’s SC outreach program during the past year including the Trait Talk brochures, SC case studies and meetings with other nuclear regulatory bodies.

Duke Energy

The presenter provided a list of internal (CAP, Employee Concerns Program )and external (INPO, NRC) information, and management activities (Nuclear SC Monitoring Panel, Site Leadership team, Corporate Nuclear SC Monitoring Panel, Fleet Nuclear SC Monitoring Panel, Executive Nuclear Safety Council) that are used to assess equipment, processes and people across the Duke fleet.  There was no information on how these activities are integrated to describe plant or fleet SC, or if any SC issues have been identified or corrective actions taken; the slides were basically a laundry list.

Department of Energy (DOE)

The speaker was from DOE’s Office of Environment, Health, Safety and Security.  He reviewed the safety mission and goals related to DOE’s Integrated Safety Management program, DOE’s SC focus areas (leadership, employee/worker engagement and organizational learning) and SC-related activities (extent of condition reviews, self‐assessments, sustainment plans, independent assessments and the SC Improvement Panel.) 

The presentation covered the challenges in relating SC to safety management performance (mostly industrial safety metrics) and in implementing cultural changes.  Factors that make SC improvement difficult include production vs. safety goal conflict, fiscal pressures, leadership changes and internal inertia (resistance to change).

This presentation covered the basics of SC, as customized for DOE, but had no supporting details or any mention of the SC issues that have arisen at various DOE facilities, e.g., Hanford, Pantex and the Waste Isolation Pilot Plant.  We have posted many times on DOE SC; please click on the DOE label to retrieve these posts.

SC Consultant

The presenter was Sonja Haber.  She reviewed the fundamentals of the linkage between culture, behavior and ultimate performance, and the Schein three-level model of culture.

She also covered the major considerations for conducting SC assessments including having a diversity of expertise in assessing culture, using multiple methods of data collection, understanding how cultural complexity impacts performance and considering the interaction of human, organizational and technological factors.

Our Perspective

This was thin gruel compared to the 2014 RIC SC session (which we reviewed April 25, 2014).  Based on the slides, there was not much “there” there at this session.  The speaker who offered the most was Dr. Haber, not a surprise given that she has been involved in SC evaluations at various DOE facilities and testified at a Defense Nuclear Facilities Safety Board hearing on SC (which we reviewed June 9, 2014).

If a webcast of the SC technical session becomes available, we will review it to see if any useful additional information was presented or arose during the discussion.


*  The SC technical session presentations are available on the NRC website.

A Safety Culture Guide for Regulators

This paper* was referenced in a safety culture (SC) presentation we recently reviewed.  It was prepared for Canadian offshore oil industry regulators.  Although not nuclear oriented, it’s a good introduction to SC basics, the different methods for evaluating SC and possible approaches to regulating SC.  We’ll summarize the paper then provide our perspective on it.  The authors probably did not invent anything other than the analysis discussed below but they used a decent set of references and picked appropriate points to highlight.

Introduction to SC and its Importance
 
The paper provides some background on SC, its origins and definition, then covers the Schein three-tier model of culture and the difference between SC and safety climate.  The last topic is covered concisely and clearly: “. . . safety climate is an outward manifestation of culture. Therefore, safety culture includes safety climate, but safety culture uniquely includes shared values about risk and safety.” (p. 11)  SC attributes (from the Canadian Nuclear Safety Commission) are described.  Under attributes, the authors stress one of our basic beliefs, viz., “The importance of safety is made clear by the decisions managers make and how they allocate resources.” (p. 12)  The authors also summarize the characteristics of High Reliability Organizations, Low Accident Organizations, and James Reason’s model of SC and symptoms of poor SC.

The chapter on SC as a causal factor in accidents contains an interesting original analysis.  The authors reviewed reports on 17 offshore or petroleum related accidents (ranging from helicopter crashes to oil rig explosions) and determined for each accident which of four negative SC factors (Normalization of deviance, Tolerance of inadequate systems and resources, Complacency, Work pressure) were present.  The number of negative SC factors per accident ranged from 0 (three instances) to 4 (also three instances, including two familiar to Safetymatters readers: BP Texas City and Deepwater Horizon).  The negative factor that appeared in the most accidents was Tolerance of inadequate systems and resources (10) and the least was Work pressure (4).

Assessing SC
 
The authors describe different SC assessment methods (questionnaires, interviews, focus groups, observations and document analysis) and cover the strengths and weaknesses of each method.  The authors note that no single method provides a comprehensive SC assessment and they recommend a multi-method approach.  This is familiar ground for Safetymatters readers; for other related posts, click on the “Assessment” label in the right hand column.

A couple of highlights stand out.  Under observations the authors urge caution:  “The fact that people are being observed is likely to influence their behaviour [the well-known Hawthorne Effect] so the results need to be treated with caution. The concrete nature of observations can result in too much weight being placed on the results of the observation versus other methods.“ (p. 37)  A strength of document analysis is it can evidence how (and how well) the organization identifies and corrects its problems, another key artifact in our view.

Influencing SC

This chapter covers leadership and the regulator’s role.  The section on leadership is well-trod ground so we won’t dwell on it.  It is a major (but in our opinion not the only) internal factor that can influence the evolution of SC.  The statement that “Leaders also shape the safety culture through the allocation of resources” (p. 42) is worth repeating.

The section on regulatory influence is more informative and describes three methods: the regulator’s practices, promotion of SC, and enforcement of SC regulations.  Practices refer to the ways the regulator goes about its inspection and enforcement activities with licensees.  For example, the regulator can promote organizational learning by requiring licensees to have effective incident investigation systems and monitoring how effectively such systems are used in practice. (p. 44)  In the U.S. the NRC constantly reinforces SC’s importance and, through its SC Policy Statement, the expectation that licensees will strive for a strong SC.

Promoting SC can occur through research, education and direct provision of SC-related services.  Regulators in other countries conduct their own surveys of industry personnel to appraise safety climate or they assess an organization’s SC and report their findings to the regulated entity.**  (pp. 45-46)  The NRC both supports and cooperates with industry groups on SC research and sponsors the Regulatory Information Conference (which has a SC module).

Regulation of SC means just what it says.  The authors point out that direct regulation in the offshore industry is controversial. (p. 47)  Such controversy notwithstanding, Norway has developed  regulations requiring offshore companies to promote a positive SC.  Norway’s experience has shown that SC regulations may be misinterpreted or result in unintended consequences. (pp. 48-50)  In the nuclear space, regulation of SC is a popular topic outside the U.S.; the IAEA even has a document describing how to go about it, which we reviewed on May 15, 2013.  More formal regulatory oversight of SC is being developed in Romania and Belgium.  We reported on the former on April 21, 2014 and the latter on June 23, 2014.

Our Perspective
 
This paper is written by academics but intended for a more general audience; it is easy reading.  The authors score points with us when they say: “Importantly, safety culture moves the focus beyond what happened to offer a potential explanation of why it happened.” (p. 7)  Important factors such as management decision making and work backlogs are mentioned.  The importance of an effective CAP is hinted at.

The paper does have some holes.  Most importantly, it limits the discussion on influencing SC to leadership and regulatory behavior.  There are many other factors that can affect an organization’s SC including existing management systems; the corporate owner’s culture, goals, priorities and policies; market factors or economic regulators; and political pressure.  The organization’s reward system is referred to multiple times but the focus appears to be on lower-level personnel; the management compensation scheme is not mentioned.

Bottom line: This paper is a good introduction to SC attributes, assessments and regulation.


*  M. Fleming and N. Scott, “A Regulator’s Guide to Safety Culture and Leadership” (no date).

**  No regulations exist in these cases; the regulator assesses SC and then uses its influence and persuasion to affect regulated entity behavior.

IAEA OSART in USA

The International Atomic Energy Agency (IAEA) offers an Operational Safety Review Team (OSART) program where international teams of experts conduct in-depth reviews of nuclear power plant operational safety performance.  Performance is reviewed in various areas such as management, training, operations, maintenance, etc.  The reports also include comments about plant safety culture (SC).  IAEA has conducted scores of OSART reviews, including U.S. power plants.  This post covers SC-related findings for the U.S. plants; we are interested in what kinds of SC strengths and weaknesses the teams identify during their approximately two-week visits.

North Anna (2000)

This is the earliest OSART report available in the NRC ADAMS database.  There are two versions of the report, one distributed to NRC Commissioners on Nov. 30, 2000* and another released by IAEA on Feb. 12, 2001.**  Both versions include generally complimentary language related to SC saying management is committed to safety and continuous improvement is nurtured. (p. 7 in 2000 report)  In the Operations area the report notes “Policies and management verbal communication are consistent and emphasize the importance of safety first.  Safety culture is referred to at several hierarchical levels of the plant.” (p. 31 in 2000 report)

There is an interesting discrepancy between the two versions.  The first report says “The emphasis the plant is making on self-assessment and improving human performance to bring about a sustaining safety culture is presently balancing the perception of increased cost drivers and short term budgeting.” (p. 13)  This hints at a cost vs. safety goal conflict, a serious challenge to maintaining a strong SC.  However, the second report does not include that statement, instead noting elsewhere that “In the teams opinion, however, there is presently little resource margin to absorb increased demands and although the highly motivated staff continue to look for ways to improve, the potential for complacency and demotivation from the prospect of long term challenges with aging facilities and equipment and tightening budget poses a risk to maintaining that performance.” (p. 9)  Is this just a more politic way of saying the same thing or does it reflect an actual watering down of the report?

Brunswick (2005)

The report*** covers the initial visit and the follow-up visit 19 months later.  SC is treated more thoroughly than in earlier reviews: “An important element of the OSART review is the identification of those findings that exhibit positive and negative attributes of safety culture.” (p. 7)  At Brunswick positive SC attributes included a questioning attitude of all personnel, a strong self-assessment program, and adherence to established procedures.  Improvement opportunities included the need for increased attention for continuous improvement, greater use of self criticism when observing safety systems and components, and further expanding external operating experience activities. (p. 10) 

The team’s in-depth review of the corrective action program (CAP) provides a look at the SC in practice.  “The corrective action programme is very strong and closely monitored.” (p. 97)  Aging mechanisms, especially corrosion, are a challenge for the corrective action program (p. 69) but the plant’s response led to this issue being closed during the follow-up visit.  Rework is analyzed by a committee to address effective corrective actions (p. 57) but maintenance backlogs require constant attention. (p. 59)  After the team’s initial visit the plant developed a new system for classifying condition reports; at follow-up the team observed “Based on the new graded system, the plant is improving opportunities to focus analyses and resources in accordance with the significance of the events.” (p. 100)  Overall, this seems reflective of a SC that supports both the CAP and efforts to improve it.

Another finding reflecting SC was that the plant Management Succession Planning and Development Program did not include safety performance in talent assessment, ranking or as a specified core skill.  Brunswick claimed these factors were implicitly considered but strengthened the program to explicitly include nuclear safety, radiation safety and industrial safety goals for each plant staff member.  The review team found this action sufficient to close the issue. (pp. 21-22)  This is also indicative of a culture that encourages improvement efforts.

Arkansas Nuclear One (2008) 

The positive SC attributes in this report**** include a strong Human Performance program with a focus on continuous improvement, a rigorous and conservative approach when planning and performing tasks, and a willingness of staff to correct each other’s behavior without waiting to be corrected by management.  In addition, conservative decision making is a core value within the management team.  Problem areas include long standing defects throughout the plant (and first attempts to fix problems that do not always work), policies such as overtime and time pressure on outage schedule that suggest the plant is “cost” driven, and the benchmarking program does not work from a broad international base. (pp. 4-5, 34).  Both an ineffective CAP and a cost-driven plant (i.e, goal conflict between cost and safety) reflect a weak SC.

Seabrook (2011)

In the Seabrook report***** SC strengths include consistent reinforcement of key values by the management team and no evidence of production taking precedence over safety.  Opportunities for improvement include a more aggressive approach to addressing long term issues, thus reducing the potential for staff normalization to less than excellent conditions. (pp. 4, 45)

A positive observation is that plant communications stress safety as the no. 1 priority. (p. 7)  Another is that Seabrook has a problem reporting culture based on a low-threshold and high-volume reporting system. (pp. 2, 56)  Personnel who report problems outside their area of responsibility are recognized with a “Good Catch” award.  Personnel writing lots of condition reports is generally a good thing but some corrective actions are closed prematurely. (pp. 54-56, 59)

More significantly, plant problems include a lack of resolution of long term issues, plant material condition deterioration and degraded equipment conditions. (pp. 2, 44-45)  Significant backlogs exist in the CAP, work orders and procedure change requests. (p. 8)  Work backlogs are an indication of a culture that lives with issues rather than resolving them.

Seabrook SC standards are summarized as “very positive” (p. 4) and maybe they are, but the reality of backlogs and unresolved long-term issues is inconsistent with a strong SC. 

Our Perspective

It’s probably unrealistic to expect a group of foreigners to visit a U.S. plant for two weeks and come up with significant SC insights.^  The summaries of SC positives and negatives appear to be relatively superficial but findings in the functional areas can offer a look at the actual underlying SC.  Some of the OSART observations and findings on key artifacts provide information from which we can infer the strength of SC at a plant being reviewed. 

We would like to see greater attention to how important decisions are made at a plant.  Decision making was only mentioned in passing (North Anna, p. 31) or a simple statement like plant performance indicators are a key input to decision making. (ANO, p. 6)

Finally, compensation and reward systems appear to be outside the OSART scope but we know these are critical to reinforce safety-related behavior and overall SC.


*  J.D. Lee (NRC) to NRC Commissioners, “IAEA Report on OSART Mission at North Anna Power Station, January 22 to February 10, 2000” (Nov. 30, 2000).  ADAMS ML010160525.

**  IAEA, “Report of the OSART Mission to the USA North Anna Nuclear Power Plant 22 January to 10 February 2000” (Released Feb. 12, 2001).  ADAMS ML010470115.

***  IAEA, “Report of the OSART Mission to the Brunswick Nuclear Plant United States of America 9-25 May 2005 and Follow Up Visit 4-8 December 2006” no date.  ADAMS ML071100006.

****  IAEA, “Report of the OSART Mission to the Arkansas Nuclear One Nuclear Power Plant United States of America 15 June – 2 July 2008” no date.  ADAMS ML083440148.

*****  P. Freeman (Seabrook) to NRC, “IAEA Final Report OSART Mission to Seabrook Nuclear Power Plant USA 6-23 June 2011” (Mar. 16, 2012).  ADAMS ML12081A105.

^  A partial view is suggested in the Foreword to the Seabrook report where the authors note that an OSART review represents a “snapshot in time.”  In fairness, it would be equally unrealistic to expect a team of Americans to visit a foreign plant and develop a deep understanding of plant culture in a similar time period.