When concerns are raised about the safety culture of an organization with very significant safety responsibilities what’s one to do? How about, bring in the lawyers. That appears to be the news out of the Vit Plant* in Hanford, WA. With considerable fanfare Bechtel unveiled a new website committed to their management of the vit plant. The site provides an array of policies, articles, reports, and messages regarding safety and quality.
One of the major pieces of information on the site is a recent assessment of the state of safety culture at the vit plant.** The conclusion of the assessment is quite positive: “Overall, we view the results from this assessment as quite strong, and similar to prior assessments conduct [sic] by the Project.” (p. 16) The prior assessments were the 2008 and 2009 Vit Plant Opinion Surveys.
However our readers may also recall that earlier this year the Defense Nuclear Facilities Safety Board (DNFSB) issued its report that at the safety culture at the WTP plant is “flawed”. In a previous post we quoted from the DNFSB report as follows:
“The HSS [DOE's Office of Health, Safety and Security] review of the safety culture on the WTP project 'indicates that BNI [Bechtel National Inc.] has established and implemented generally effective, formal processes for identifying, documenting, and resolving nuclear safety, quality, and technical concerns and issues raised by employees and for managing complex technical issues.' However, the Board finds that these processes are infrequently used, not universally trusted by the WTP project staff, vulnerable to pressures caused by budget or schedule [emphasis added], and are therefore not effective.”
Thus the DNFSB clearly has a much different view of the state of safety culture at the vit plant than does DOE or Bechtel. We note that the DNFSB report does not appear to be one of the numerous references available at the new website. Links to the original DOE report and the recent assessment are provided. There is also a November 17, 2011 message to all employees from Frank Russo, Project Director*** which introduces and summarizes the 2011 Opinion Survey on the project’s nuclear safety and quality culture (NSQC). Neither the recent assessment nor the opinion survey addresses the issues raised by the DNFSB; it is as if the DNFSB review never happened.
What really caught our attention in the recent assessment is who wrote the report - a law firm. Their assessment was based on in-depth interviews of 121 randomly selected employees using a 19 question protocol (the report states that the protocol is attached however it is not part of the web link). But the law firm did not actually conduct the interviews - “investigators” from the BSII internal audit department did so and took notes that were then provided to the lawyers. This may give new meaning to the concept of “defense in depth”.
The same law firm also analyzed the results from the 2011 Opinion Survey. In the message to employees from , Russo asserts that the law firm has “substantial experience in interpreting [emphasis added] NSQC assessments”. He goes on to say that the questions for the survey were developed by the WTP Independent Safety and Quality Culture Assessment (ISQCA) Team. In our view, this executive level team has without question “substantial experience” in safety culture. Supposedly the ISQCA team was tasked with assessing the site’s culture - why then did they only develop the questions and a law firm interpret the answers? Strikes us as very odd.
We don’t know the true state of safety culture at the vit plant and unfortunately, the work sponsored by vit plant management does little to provide such insight or to fully vet and respond to the serious deficiencies cited in the DNFSB assessment. If we were employees at the plant we would be anxious to hear directly from the ISQCA team.
Reading the law firm report provides little comfort. We have commented many times about the inherent limitations of surveys and interviews to solicit attitudes and perceptions. When the raw materials are interview notes of a small fraction of the employees, and assessed by lawyers who were not present in the interviews, we become more skeptical. Several quotes from the report related to the Employee Concerns Program illustrate our concern.
“The overwhelming majority of interviewees have never used ECP. Only 6.5% of the interviewees surveyed had ever used the program. [Note: this means a total of nine interviewees.] There is a major difference between the views of interviewees with no personal experience with ECP and those who have used the program: the majority of the interviewees who have not used the program have a positive impression of the program, while more than half of the interviewees who have used the program have a negative impression of it.” (p. 5, emphasis added)
Our favorite quote out of the report is the following. “Two interviewees who commented on the [ECP] program appear to have confused it with Human Resources.” (p. 6) One only wonders if the comments were favorable.
Eventually the report gets around to a conclusion that we probably could not say any better. “We recognize that an interview population of nine employees who have used the ECP in the past is insufficient to draw any meaningful conclusions about the program.” (p. 17)
We’re left with the following question: Why go about an assessment of safety culture in such an obtuse manner, one that is superficial in its “interpretation” of very limited data, laden with anecdotal material, and ultimately over reaching in its conclusions?
* The "Vit Plant" is the common name for the Hanford Waste Treatment Plant (WTP).
** Pillsbury Winthrop Shaw Pittman, LLP, "Assessment of a Safety Conscious Work Environment at the Hanford Waste Treatment Plant" (undated). The report contains no information on when the interviews or analysis were performed. Because a footnote refers to the 2009 Opinion Survey and a report addendum refers to an October, 2010 DOE report, we assume the assessment was performed in early-to-mid 2010.