Friday, January 25, 2013

Safety Culture Assessments: the Vit Plant vs. Other DOE Facilities

The Vit Plant
 As you recall, the Defense Nuclear Facilities Safety Board (DNFSB) set off a little war with DOE when DNFSB published its blistering June 2011 critique* of the Hanford Waste Treatment Plant's (Vit Plant) safety culture (SC).  Memos were fired back and forth but eventually things settled down.  One of DOE's resultant commitments was to assess SC at other DOE facilities to see if  SC concerns identified at the Vit Plant were also evident elsewhere.  Last month DOE transmitted the results of five assessments to DNFSB.**  The following facilities were evaluated:

• Los Alamos National Laboratory Chemistry and Metallurgy Research Replacement Project (Los Alamos)
• Y-12 National Security Complex Uranium Processing Facility Project (UPF)
• Idaho Cleanup Project Sodium Bearing Waste Treatment Project (Idaho)
• Office of Environmental Management Headquarters (EM)
• Pantex Plant
 


The same protocol was used for each of the assessments: DOE's Health, Safety and Security organization formed a team of its own assessors and two outside experts from the Human Performance Analysis Corporation (HPA).  Multiple data collection tools, including functional analysis, semi-structured focus group and individual interviews, observations and behavioral anchored rating scales, were used to assess organizational behaviors.  The external experts also conducted a SC survey at each site.

A stand-alone report was prepared for each facility, consisting of a summary and recommendation (ca. 5 pages) and the outside experts' report (ca. 25 pages).  The outside experts organized their observations and findings along the nine SC traits identified by the NRC, viz.,

• Leadership Safety Values and Actions
• Problem Identification and Resolution
• Personal Accountability
• Work Processes
• Continuous Learning
• Environment for Raising Concerns
• Effective Safety Communication
• Respectful Work Environment
• Questioning Attitude.

So, do Vit Plant SC concerns exist elsewhere?

That's up to the reader to determine.  The DOE submittal contained no meta-analysis of the five assessments, and no comparison to Vit Plant concerns.  As far as I can tell, the individual assessments made no attempt to focus on whether or not Vit Plant concerns existed at the reviewed facilities.

However, my back-of-the-envelope analysis (no statistics, lots of inference) of the reports suggests there are some Vit Plant issues that exist elsewhere but not to the degree that riled the DNFSB when it looked at the Vit Plant.  I made no effort to distinguish between issues mentioned by federal versus contractor employees, or by different contractors.  Following are the major Vit Plant concerns, distilled from the June 2011 DNFSB letter, and their significance at other facilities.

Schedule and/or budget pressure that can lead to suppressed issues or safety short-cuts
 

This is the most widespread and frequently mentioned concern.  It appears to be a significant issue at the UPF where the experts say “the project is being driven . . . by a production mentality.”  Excessive focus on financial incentives was also raised at UPF.  Some Los Alamos interviewees reported schedule pressure.  So did some folks at Idaho but others said safety was not compromised to make schedule; financial incentives were also mentioned there.  At EM, there were fewer comments on schedule pressure and at Pantex, interviewees opined that management shielded employees from pressure and tried to balance the message that both safety and production are important.

A chilled atmosphere adverse to safety exists

The atmosphere is cool at some other facilities, but it's hard to say the temperature is actually chilly.  There were some examples of perceived retaliation at Los Alamos and Pantex.  (Two Pantex employees reported retaliation for raising a safety concern; that's why Pantex, which was not on the original list of facilities for SC evaluation, was included.)  Fear of retaliation, but not actual examples, was reported at UPF and EM.  Fear of retaliation was also reported at Pantex. 

Technical dissent is suppressed

This is a minor issue.  There were some negative perceptions of the differing professional opinion (DPO) process at Los Alamos.  Some interviewees thought the DPO process at EM could be better utilized.  The experts said DPO needed to be better promoted at Pantex. 

Processes for raising and resolving SC-related questions exist but are neither trusted nor used

Another minor issue.  The experts said the procedures at Los Alamos should be reevaluated and enforced.

Conclusion

I did not read every word of this 155 page report but it appears some facilities have issues akin to those identified at the Vit Plant but their scope and/or intensity generally appear to be less.

The DOE submittal is technically responsive to the DNFSB commitment but is not useful without further analysis.  The submittal evidences more foot dragging by DOE to cover up the likely fact that the Vit Plant's SC problems are more significant than other facilities' and buy time to attempt to correct those problems.


* Defense Nuclear Facilities Safety Board, Recommendation 2011-1 to the Secretary of Energy "Safety Culture at the Waste Treatment and Immobilization Plant" (Jun 9, 2011).  We have posted on the DOE-DNFS imbroglio here, here and here.
   
**  G.S. Podansky (DOE) to P.S. Winokur (DNFSB), letter transmitting five independent safety culture assessments (Dec. 12, 2012).

3 comments:

  1. Lew,

    These supplemental assessments of other DOE enterprises, after the Vit Plant’s three reviews in two years, probably seemed like a good idea when the 2011-1 Implementation Plan was being put together. However, in that sense, the commitment to do them might also seem very much in the pattern of Stimulus-Response Conditioning (cf. Milgram Experiment). I would offer the possibility that SRC has become the normal state of affairs regarding the DNFSB’s oversight feedback (so-called “action-forcing) expectations, particularly in the use of its Statutory Powers of Recommendation. This Normalization of Deviation might be seen to go back to at least 2004 or so.
    Considering the out-of-budget and opportunity costs to conduct these reviews, the return in these reports seems thin gruel indeed. A pretty good argument could be made that the rather trivial and highly formulaic report conclusions serve to invalidate the utility of the assessment method employed. Evidently, however, it was not an objective of this round-robin to achieve such a validation – the reports state that a high-confidence level in the survey method and sense-making protocols pre-existed the conduct of these specific reviews.
    Rather than do an academic critique of the baseline assertions in all those repetitious boilerplate paragraphs, it seems more important to note some of the “elephants in the room” at the non-Vit Plant sites that go unremarked about in these reports. What might these suggest about missing observations regarding the Vit Plant itself?
    For example, CMRR project has ground to a halt – after a decade’s work and many millions spent, for a combination of unresolved safety technical issues leading (?) to cost and schedule uncertainties too high to sustain in the midst of other NNSA capital construction priorities. The UPF project was the subject of a Stop Work ($500M in) this past Fall owing to the discovery that the preliminary design stage foot print was 25% too small to meet the facility functional requirements DOE contracted for.
    The construction completion and initial facility startup at the SBWTP in Idaho is proving challenging (not unexpectedly) but it appears that DOE and contractor personnel “have higher priority things to do” than stop to reflect on the state of Safety Culture. A year ago, the prime contractor was in a fee-return penalty situation for a number of months owning to delays from poor construction quality. That might have been the time to have done the Safety Culture assessment!

    (to be continued)

    ReplyDelete
  2. Over the past 20 years, Pantex Plant has probably received more DNFSB “quality” time for performance reliability oversight than any other DOE facility (which is fine by me given they are the ones taking nuclear weapons apart). One familiar with the record of that oversight, the improvements of all kinds instituted over the years, and with the cadre of very dedicated and up to date performance improvement staff at Pantex might question if adding Pantex to the list really makes much sense. As a manufacturing-production facility they are worlds apart from the circumstances of the other organizations surveyed.
    By comparison, the report of assessment in the DOE-HQ EM enterprise is remarkable for its rather “disconnected from reality” air – represented in conclusions like this:
    “The data collected during this assessment indicates a homogeneous organization with respect to the perceptions and beliefs across the EM-HQ Organization. The lack of statistically significant differences across work groups and employee categories on most of the measures being evaluated is indicative of a pervasive culture. Combined with the perception by employees’ sense of tenuousness with several managers in acting positions, and the reductions in staffing mandated by budget decisions, the effective implementation of many of the new initiatives (cited above) in EM will be a challenge.”
    “Pervasive culture” as in “stacked cord wood” – this conclusion is suggestive of the common weakness observed in these type reviews: they comprise a snapshot in the moment looking at the wake of the ship, without suggesting what resilience for the challenges ahead may or may not exist. That and the seeming fact that at no point does one of these surveys “dig deeper” no matter how curious the information being gathered – where’s that “questioning attitude” when you need it?
    Later the recommendation appears:
    “1. As the EM Organization transitions into its new structure, it should seek to provide more clarity regarding the roles and responsibilities of each of the Office’s units. More formalization around the processes and procedures that are expected would also be beneficial for those trying to implement new initiatives. The oversight function in particular should be clearly identified and defined going forward.”
    Given that on the previous page it was reported that the new structure was emphasizing Integrated Project Teams and similar cross-functional collaboration as a way of improving field support, and that the organization is facing reduced FTE levels, how does this recommendation for more “paper boundary setting” make sense in terms of building and sustaining an enterprise culture facing “stormy seas ahead.”

    ReplyDelete
  3. The earlier VIT Plant surveys took the evidence of whistleblowers, anonymous employee concerns, differing professional opinion reports, and nuclear safety with design inter-departmental friction – as well as lousy overall mission performance – as evidence that “safety culture weakness” was implicated as a prime causal factor.
    Three years ago, no one was challenged to prove or disprove that allegation – in all likelihood that would have proven quite a task given the complete lack of benchmark surveys in comparably Complex, High-Consequence, one-off design circumstances. Perhaps that would have been a worthwhile thing to contemplate, perhaps it would have been hard to “see the forest for the trees” – today with these additional reports it would seem that seeing the forest would be a lot easier.
    Severe human performance difficulties on the Vit Plant project have been pandemic (in some observer’s views) going back more than 10 years. On the hypothesis, that Vit Plant nuclear safety culture frictions are symptomatic of faulty Acquisition Strategy; insufficient Program management “competence commensurate with [the] responsibilities” of that “world class” acquisition challenge and working contract administration we are left to wonder how these follow on surveys enlighten us.
    I would suggest there is a lot of evidence that “weak nuclear safety culture” – when “captured” by the surveys reported here – is not serving well as either a diagnostic nor remedy indicator for what are recurrent failures of Integrated Safety Management achievement to the regulations incorporated in each of the site contracts, standards that are also applicable to the EM establishment. Sadly these surveys, modeled as they are around power reactor operations don’t “shine a light in that corner.”

    ReplyDelete

Thanks for your comment. We read them all. We would like to display them under their respective posts on our main page but that is not how Blogger works.