Case Study

Solvay plant Reduces Downtime and Improves Safety through Alarm Management

The Solvay Novecare plant in Pasadena, Texas had nuisance alarms, critical alarms that weren’t (critical), and production delays due to alarms being missed. A stronger alarm management program was needed to create a safer and more productive plant.

The Pasadena plant, which opened in November of 2015, operates a large-scale, "on pipe" alkoxylation unit as part of a batch process. Situated in the integrated industrial campus of LyondellBasell’s Equistar Chemicals affiliate, the plant is supplied the key raw material ethylene oxide via pipeline, enhancing sustainability and surety of supply. Alkoxylates are used as emulsifiers, detergents and wetting agents and are the chemical foundation for a wide range of Solvay Novecare specialty surfactants.

Challenges

Alarm management problems can impact operations directly and indirectly. In Solvay’s case, operators were overloaded with nuisance alarms, oftentimes filling three to four pages of the alarm summary with stale alarms. This caused the operators to ignore alarms, reduced their situation awareness, and made it hard to notice new alarms. Because of the volume of nuisance alarms, the alarm horn was left disabled after commissioning so that no audible notification was generated for alarms or for batch recipe prompts. The lack of audible notification made it more challenging for the operators to “stay on top” of the process and to know when new alarms came in. Occasionally, this led to production interruptions when key utility equipment shutdown that could have been addressed promptly, but instead went unnoticed.

A large percentage of alarms were Critical priority (the highest priority) by default; this made it hard for operators to distinguish importance and was misleading since many of these were not Critical. As Solvay automation engineer Tracy Blauvelt-Heilaman says, “If everything is critical, then nothing is critical. This puts extra burden and stress on the operator.” There was also no training for engineers about basic alarm management principles, so even they overlooked the lurking alarm problem in favor of chasing today’s production troubleshooting. 

Answer: Developing an Alarm Management Program that could be Rolled Out to Other Sites

Solvay contracted with exida, Emerson, and Puffer-Sweiven to implement a holistic alarm management approach, starting with the Pasadena plant, and intentionally designed as a baseline for other Solvay Novecare sites.  Applying the ISA-18.2 standard, which is considered recognized and generally accepted good engineering practice (RAGAGEP) by OSHA, is key to developing a successful alarm management program and is the basis of exida’s recommendations.  ISA-18.2 defines a recommended workflow, called the alarm management lifecycle, which Solvay embraced as their foundation for success moving forward.  Ryan Scofield, Novecare Global Automation Manager, explained “Most of us jump into alarm remediation (bad actor knockdown) without first establishing guidelines or setting goals. For this project we wanted to discuss existing alarms, as well as find missing ones. The lifecycle approach defined in ISA-18.2 is really the right way to achieve that.”

Step 1: Benchmark Performance

One of the initial steps was to benchmark the performance of the alarm system. Benchmarking identifies systematic issues and sets the baseline for judging performance improvement. ISA-18.2 defines recommended KPIs (e.g., average number of alarms/day, alarm priority distribution, number of stale alarms), so with the help of DeltaV Analyze, the Pasadena site was able to compare actual to recommended performance.

Step 2: Develop an Alarm Philosophy Document

With the benchmarking step complete, the team moved on to developing an alarm philosophy document (APD) as the foundation of their alarm management program. The process started with exida leading a one-day training session for the site alarm management team, a cross-functional representation from Operations, Engineering, and Maintenance.  With training, the team was able to align their scattered ideas with the actual RAGAGEP practices found in ISA-18.2. The training also reviewed the alarm management functionality available directly in the DeltaV control system and how it could be used to improve performance of operator response to alarms.

Following training, exida led a two-day workshop with the team to develop the APD. The APD template provided by exida is comprehensive; it addresses the entire lifecycle to ensure that the necessary work practices are in place to drive performance improvement and to sustain it over time. "Our operations groups at every site were asking for a more specific starting point so that they did not have to reinvent the wheel”, said Scofield. “Getting started is the hardest part.  The APD was written with the understanding that it could be tweaked with minor effort for each site and would eliminate starting from scratch over and over."

Establishing a consistent and objective methodology for alarm prioritization was an important goal for the alarm philosophy. Solvay aligned their alarm priority matrix with their existing corporate Risk Management documents, including consideration for risks to people, environment, assets, quality, and productivity. This ensures a common approach to consequence evaluation is used for process hazard analysis and alarm rationalization.  

“The alarm philosophy document is truly the core of our program. Developing it forced us to think about purposeful alarm management. What alarms are actually significant?  What’s in the way of recognizing them now?  How can we provide crystal-clear information for the very human operator? How do we resist the next urge to add yet another brighter light or louder horn to bother the operator about the latest perceived imperative?  Without the structure of our APD, we’d just be playing ‘whac-a-mole’ forever.”

Tracy Blauvelt-Heilaman

Novecare Automation Engineer, Solvay

Step 3: Identify & Rationalize Alarms

Having a solid alarm philosophy in place, the team was ready for rationalization. Solvay selected the SILAlarm tool, an Emerson partner product for alarm management by exida, to be their master alarm database (MADB).  SILAlarm was seeded with Solvay-specific details defined in the APD, such as the priority matrix, the DeltaV color scheme, and the urgency matrix.  All DeltaV configured alarms were also imported directly into SILAlarm.  

With initial coaching from exida, Solvay used the SILAlarm MADB guided tools to rationalize and document each alarm. The purpose of rationalization is to ensure that all alarms presented to the operator are (a) actionable, (b) represent an unexpected situation, and (c) prioritized to indicate their relative importance and urgency. Rationalization is a systematic, bottoms-up approach that ensures alarm management fundamentals are met; “Bad actor knockdown” is a reactive process more like the children’s game “whac-a-mole”. 

The Solvay team knew that following the APD and implementing alarm rationalization represented a paradigm shift for operations. To sow the seeds of success the team made sure that operations was heavily involved in rationalization with participation from all shifts. Direct involvement in the improvement process increased enthusiasm amongst all plant personnel. Operators gained a new perspective on their role and a better appreciation for engineering. Operators and engineering are collaborators now. As a result, operators are more likely to raise concerns and potential issues to the alarm team.  This also enabled transition out of “project” mode and into ongoing continuous improvement mode, which is the essence of the lifecycle approach. 

An important part of the rationalization process is to document in the MADB the likely cause, consequence, corrective action, and allowable time to respond for each alarm. The senior operators provided a great deal of insight, effectively transforming rationalization into a knowledge capture process. They identified redundant alarms, which alarms would come first in a sequence, as well as causal events and patterns. Junior operators grew from the experience (what does it mean when I get this alarm?) and thought about things differently next time they were back at the HMI console. 

Rationalization takes effort and requires patience. The Solvay team rationalized approximately 9000 alarms over a 6-month timeframe. To prevent burnout, meetings were held twice monthly, Tuesday through Thursday, for no more than 5 ½ hours at a time. As a parallel benefit, the rationalization exercise even uncovered maintenance issues such as transmitter problems, equipment problems, and process problems.

Step 4: Implement while Leveraging the Capability of the DeltaV Alarm System

Best of all, SILAlarm provides functionality such that all of the rationalization results in the MADB can be imported directly into DeltaV, automatically updating alarm configuration and DeltaV Alarm Help.  This saved significant time and prevented errors from manual entry. Now when a new alarm occurs the operators can quickly call up Alarm Help to review the cause, consequence, corrective action and time to respond information for that alarm as it was captured during rationalization. 

One of the outcomes of the APD was definition of how to best leverage the alarm management functionality of the control system to better support operations. The team tailored the alarm priority system to differentiate and segregate alarms from batch prompts (with audible annunciation and without), bypasses (interlocks, equipment modules, and phases), indication of simulated / forced variables, and report (situations that don’t require a timely response or require no response at all). The team at Puffer Sweiven, the Emerson Local Impact Partner, created a dedicated summary display for the different categories with operator access as appropriate from the banner, main toolbar, dashboard, and overview display. These displays provide a way to improve the operator’s situation awareness while improving their ability to focus on alarms and batch status (now that non-alarms are no longer mixed in). The APD defined a consistent approach to audible annunciation (alarm horns) for the control room and in the field which was implemented accordingly in DeltaV.

Rationalization defined some scenarios where conditional alarming was warranted. Conditional alarming allows an alarm to be dynamically enabled / disabled based on a status or indication. This allows, for example, a low discharge pressure alarm on the outlet of a pump to reference the pump RUN status, in addition to the pressure value, when determining whether to annunciate. Once a common nuisance alarm, now the low discharge pressure alarm is not triggered just by virtue of the pump being turned off. The Puffer Sweiven team was an instrumental resource in implementing these conditional alarms and other advanced logic to make the alarms most useful for the operators. 

Results

By following the philosophy’s alarm priority matrix during rationalization, alarms were reduced and reprioritized based on importance.  Even the distribution of alarm priorities was brought in line with ISA-18.2 recommendations (~5% high, ~15% medium, ~80% low). Previously a majority of alarms were default categorized as Critical, making it impossible for operators to distinguish their true importance. Now operators have a clearer understanding of what is important. Critical alarms are rare, and really get the operator’s attention when they do occur.

Alarm rationalization successfully eliminated nuisance alarms and reduced alarm load by 84%. The three to four pages of standing alarms has been reduced to only 10 – 12 alarms active at a time. The alarm horn was turned on. Now if utility upsets occur, the operator quickly notices and can investigate or call for maintenance to resolve the issue before resulting in a production delay.

“Operator involvement was key to the success of implementing SILAlarm.  Operators got a better understanding of the process and the initiative, and gave Alarm Help suggestions relevant to the work operators do.  They see the difference alarm rationalization has made, pay more attention to the alarms they do get, and are more likely to make suggestions for future alarm improvement. ”

Susan Chambers

Pasadena Automation Engineer, Solvay

One of the most notable benefits of the alarm management program was achieving "Highly Protected Risk" (HPR) status from Solvay’s insurance underwriter. The HPR designation resulted in a reduction in the plant’s insurance premium. It meant that the Pasadena facility meets the highest industry standards for property protection and risk management. Only 6% of the insurer’s customers reach this status. The Pasadena plant was only the 8th Solvay plant in the world to reach that status.

Next Steps

“We’ve made substantial improvements for Pasadena’s alarm program.  It relieves sensory stress for the Pasadena operators and allows them to focus on the important information.  We have work still to do, but that’s the nature of a lifecycle approach”, says Blauvelt-Heilaman. “The support of exida’s alarm management professionals and their SILAlarm tool, along with the alarm tools in DeltaV, have absolutely facilitated our success.  Going forward we want to multiply the accomplishments from Pasadena by applying them at other Solvay Novecare facilities. “