Case Study: Integrating PlantPAx into an Existing Process Cell for a Pharmaceutical Company
A major pharmaceutical company was in the process of commissioning a control system for a new process cell and asked ECS to provide site support to complete the commissioning and qualification.
The control system supplier had taken a simplistic approach to keep costs down. A single controller with multiple Panelviews provided the heart of the control system that also interfaced with some OEM skids. The control hardware consisted of a quality Rockwell ControlLogix controller with multiple Panelviews. The controller code was all custom with little use of off-the-shelf products and was an assortment of custom blocks of code tied together, providing minimal functionality.
Little to no thought had been put into the operator and how they would run the system to create quality product. Standard operating procedures had not yet been developed. Producing a quality product was dependent upon the operator selecting the correct sequence of functions from operator stations at multiple vessels. This approach exposed the company to a great deal of risk in producing a quality product and controlling the rate of production.
In addition, since procedures had not been considered to this point, the system was not well documented and inevitably in store for substantial changes through the commissioning and qualification phases of the project.
As the owners started to think practically about how operators would make product, change requests started to trickle in – and eventually pour in. Changes to the custom logic were challenging. It was difficult to determine what effect small changes would have in other areas of the program.
ECS evaluated the situation and recommended a migration to Rockwell’s PlantPAx, providing FactoryTalk Batch for the batch automation with Rockwell’s proven Process Objects in the controller replacing much of the custom code.
The probability of errors in production led ECS Solutions to recommend that many activities carried out by the operators be significantly reduced. This was achievable by automating the operator activities related to procedural control, parameter transfers, and data gathering of the system. ECS proposed that FactoryTalk Batch be integrated into the system to provide a layer of automation that would reliably set the parameters for the Equipment Module and Sequences at the right time, every time. The company agreed to add this layer of automation but requested that the implementation of FactoryTalk Batch not interfere with the ongoing activities and that all existing code and functionality be preserved without any changes being made.
With the proposed integration of FactoryTalk Batch, the operator would be required to simply select a recipe (stored by the sequencing engine) and select which equipment was needed to make the batch. The FTBatch sequencing engine coordinated all activities, including the transfer of parameter values, and the capturing of Reports information. Furthermore, the system prompted the operator when a task required operator interaction. This work was carried out in such a way as to have no impact on the functionality of the existing Equipment Modules or Sequences.
To do this ECS Solutions created an area model in FactoryTalk Batch to represent the existing equipment in the process area, with Phase classes and Phases representing the individual PLC Equipment Modules and PLC Sequences. A wrapper logic was created as needed in Phase Manager Phases to allow FactoryTalk Batch to properly control the existing Equipment Modules and Sequences. This resulted in a system capable of executing recipes without parameters being entered manually or equipment being started by an operator while capturing all pertinent report values. This dramatically improved the reliability and repeatability of the existing system by reducing (if not eliminating) human error.
At this point to make a batch recipe, ECS created procedures to encompass everything that needed to be done with every piece of equipment. Phases were created that could talk directly to the coordination sequences or coordination Equipment Modules. A wrapper was created for all their Equipment Modules, which lacked some of the flexibility required by the company. It should be noted that a Rockwell PlantPAx Solution was integrated with FactoryTalk Batch, including key procedures and direct signatures. Existing custom code was replaced with a Rockwell Process objects standard to provide a more reliable and sustainable solution.
The absence of a standard Process object was the major driver for the migration. The custom code in the existing system had some limitations and issues in terms of coding, and later the control module was migrated to a PlantPAx standard. However, the Equipment Module layer remained standard since it was clearly commissioned. Some problems with the control module layer were identified by ECS- inconsistent logic, absence of signal filtering, and software anomalies- so that the entire standard was replaced by a global standard library (PlantPAx) custom code, with standardized objects in place of the custom code.
A significant amount of IO was migrated, and at the same time, ECS was commissioning other systems. It became critical to maintain the timeline as the code was replaced and ensure that it was thoroughly tested prior to validation. This required a risk response plan (familiar to ECS), recognizing that errors due to migration would occur. Each controller was tested in a similar environment to prevent issues in terms of coding and ensuring sufficient memory in the processor to perform the migration. In terms of architecture ECS initially created a simulation system at the ECS facilities. This system allowed for testing and building recipes, and a test could be performed before implementing them in the live system. The simulation system was identical to the production system, giving latitude to things that could be done, and implementation was less invasive in the current architecture.
Three different workstations, two of which were automation workstations, provided access to FactoryTalk and how the system was configured. At the automation work stations, recipes could be edited, changes made to the HMI, or the controllers could be configured. Thin manager clients are available, and it is only necessary to provide the IP address and credentials of the Thin manager server to automatically withdraw information so that deploying a new client or replacing an existing client is quite straightforward. Every PC is virtualized with VMware and there is an ESXi post maintaining all machines, including the HMIs.
In addition to the Standard FTBatch reports, ECS also collaborated with the pharmaceutical company to produce custom reports. The reports show operations that were triggered, and whether said operation passed or failed. The reason for a failure is also recorded. The major components of the reports are a recipe information section and a unit procedure summary. These sections capture details of interest regarding specific operations. The reports are created in Infobatch, a format used by the company.
ECS’ role in the project changed as time progressed. Initially, the aim was to put an FTBatch wrapper on the existing code and not modify it, i.e., the original agreement with the company. However, subsequently, it was decided to make changes to improve the system, due to the lack of required functionality and today the company is able to make extended batches, considered to be active production batches while the product awaits FDA approval.
FactoryTalk Batch has also enabled a wealth of historical data to be logged in the form of batch records and reports, including electronic signatures. The modification has allowed the company to maximize the utilization of its equipment. Using eProcedure, electronic prompts allow operators to review and respond, including provision of detailed photographs of equipment setup for operators’ absolute activity clarity. Furthermore, using FactoryTalk Batch recipes to drive CIP processes not only simplified the use of the CIP system but also established repeatable consistent cleaning cycles that provide as much insight as product batches. This has significantly accelerated the commissioning and validation of those systems.
The migration to Rockwell’s standard PlantPAx solution provided the consistency and reliability needed to provide the plant with the necessary functionality and achieve qualification and validation. The addition of FactoryTalk Batch to the control system has allowed the pharmaceutical company to author version-controlled, electronic recipes that execute consistently from batch to batch.