Reducing the risk of extended downtimes

10 February 2020

While migrating away from an ageing SCADA system, Yeo Valley also established a secure system backup to protect data and avoid any extended production downtimes.

Yeo Valley is an organic food producer ¬– supplying yogurt, butter, cream, ice cream and fruit compote to UK retailers.

Because dairy produce has such a short shelf life, so production and delivery times are strict, with producers facing financial penalties from buyers if they fail to meet them.

At its Blagdon site, Yeo Valley was operating an aging SCADA application which was running on legacy hardware. Engineers were concerned that a problem with any one computer would risk bringing the entire production site to a halt.

While Yeo Valley had a disaster resilience solution in place, the ‘redundant’ back-up server had to be booted up before systems could be restored. Vessels emptied of product as a result of production downtime then had to be re-primed – all this would take up valuable time and would result in a recovery time objective (RTO) that could take several hours.

The impact of such downtimes would be damaging to the company’s reputation and profitability. With around seven million pots of yogurt coming off Yeo Valley production lines every week and production running 24/7, the potential implications, in terms of operational downtime, cost of wasted product and supply chain disruption were critical, and the need to ensure uninterrupted operations a key business priority.

Yeo Valley made a decision to migrate away from its aging hardware and embark on a full upgrade of its SCADA technology. A key objective was to establish a secure systems back-up provision, to protect data in the event of an issue arising with any platform.

An integrated digital solution

Industrial IT software company, SolutionsPT, devised an integrated digital solution to meet Yeo Valley’s requirement for best-in-class uptime and data protection, while also simplifying day-to-day operations.

This comprised a fault tolerant Stratus ftServer to host virtualised production machines. The solution is said to be easy to deploy, manage and service, while offering 99.999% availability.

The centralised management solution was deployed to deliver applications to the plant floor via Thin Client architectures, reducing maintenance and energy costs and delivering a reduction in total cost of ownership (TCO).

To protect data, a managed disaster resilience service was also installed. Called Proteus, this solution provides hourly back-up of critical applications and systems, as well as providing early ransomware detection protection and the ability to run critical applications in the event of a disaster, where the normal server would not be available.

Commenting on the new system, Ian Govier, Yeo Valley’s head of IT and business systems, said: “The digital solution means the risk that one single point of failure could halt production across a site has been eliminated. Virtualisation also offers us far greater control in terms of production. We can spot and deal with potential failures before they happen. And in the event of a true disaster, our data is protected.

“Essentially, we have a whole virtual factory on a different site, so systems could be restored within minutes.”

The digital solution has allowed Yeo Valley to eliminate the risk of extended periods of downtime which might have proved costly both to company finances and its reputation.

Potential risks to operational efficiency are quickly signposted so prompt action can be taken in the event of any system failure, with no reinstallation required. As these are real time copies of the VM (virtual machine) they can be simply fired up and get full production back up and running within minutes, ensuring product supply remains unaffected.