Routine Maintenance

Pree

Member
Join Date
Mar 2015
Location
new zealand
Posts
16
Hi Guys,
What are the usual routine activities you would carry out if you are looking after site with rockwell PLC and wonderware HMI.
 
So are we to guess whether the “site” of which you speak is processing something, or baking cakes or building cars or what? You might want to supply enough information to elicit a reasonable response.
 
You need to open the tray at the bottom of the PLC and empty out all the old broken CIP connections. Line the bottom of the tray with a fresh Technote. You can use folded up money if your Techconnect is not up to date.
 
Code backups and tests on the test hardware / spare PCs and PLCs. This does three things:
1. Make sure you have the backups.
2. Keep your upload download muscle memory in shape.
3. Test your spares so you don't install them and be like "what, this one doesn't work either!" Most awkward situation ever.

Torque all screw terminals to the torque specified on each components data sheet.

Calibrate instruments.

Record all downtime and perform root cause analysis on the failure. Sort your failure reasons by total downtime and another time by total number of stops, and assign some sort of weight to those two figures to identify your biggest problems, or as I call them "opportunity for productivity improvement" because I know that sort of language wets the budget person's whistle. Sit down with your ranked list of opportunities and work out which ones can be solved with routine maintenance and which can be solved with a small to medium sized project. During big projects, drag out your list and see if there is some design changes that can save you big down the track.

Oh, and document your process and hardware and code, and all that other stuff. Honestly, if your maintenance team aren't dedicating a few hours a week to this, you are about to make some serious coin!
 
1. Make sure you have the backups.
2. Keep your upload download muscle memory in shape.
3. Test your spares so you don't install them and be like "what, this one doesn't work either!" Most awkward situation ever.

This is a good one actually... in my facility before I arrived IT came in for an audit and raised the issue that the backups were not robust (I doubt they existed).

The guy in charge made a procedure to create backups, but not to restore them. IT never bothered to read in detail the closing action of the points raised.

If using RAID one server/computer, I would routinely test the backup hard drives that were stored. As I saw it, reimaging a RAID1 server took a lot of time. Popping in a new hard drive? 15 minutes. RAID will eventually rebuild itself and you're back to being covered fully. One thing I did learn is that when a motherboard fails both RAID disks get screwed... so if you are doing this, never ever put the whole array in. Just one disk and a blank.
 
let me be more sepcific.
As an Automation Engineer What are the Routine Maintenance Activity you would carry out ( Dairy Plant) with more than 30 PLC and 20plus HMI cleint, with 2 centralised IO server and 5-6 Individual server, Historian.
 
Looks like some good stuff has been covered already but.

1. Program backups of PLC's and HMI's and on the HMI's depending on the system you are using make sure you have the run-time program and the development program.

2. For servers and such you want backups of the applications and everything you need to reinstall to new hardware including install media and serial numbers and activation keys.

3. Also for servers it would be good to have an image backup and if you want to get fancy use something like Storage Craft Shadow Protect and you can save those images local and also to the cloud and if you server dies and you do not have replacement hardware you can spin that image up in azure cloud service and run your plant from a cloud server over VPN. I saved a great many of my customers arse with this

4. Also for servers and PC's it's good idea to have full spare units in addition to hard drives and power supplies and it's good to have a RAID setup on those servers if possible.

5. Make sure you ahve spare hardware for you PLC's and HMI's also and make sure they are the right hardware revision and software revision and if they are not the same make sure you research and have an understanding of what's involved in stepping up to the new revisions and make sure you have all the tools and documentation and knowledge on hand to facilitate doing that at 3:00 AM.

6. As someone said earlier it's a good idea to test your spares and know without doubt what they will do when possible.

7. Move to migrate things that are no longer supported or make plans to migrate hardware and software nearing its end of life. Also when making these migration plans try to make things as common as you can. Do you have 1 of every PLC a specific vendor makes? Then look to find a powerful unit that may be overkill on some of your applications but could handle any of your applications and this makes keep spares more economical and much cheaper.

8. Have a spec for all new equipment and for upgrades and migrations to keep things common and simple to manage as possible.

9. Make sure you have up to date drawings of all systems no matter how simple and make sure you have CAD original drawings and PDF electronic drawings for everyday use as well as a printed set in your shop / Office and inside the control panel as well but you must keep up with the revisions and plot new paper copies when changes are made and make sure the old paper is destroyed to prevent mistakes and confusion.

10. Make sure you have a good revision control, system for all your PLC, HMI code and applications and versions as well as good revision controls for your prints and drawings.

11. A knowledge base is a good idea to keep documents you have had to research to fix something and knowledge base articles you create yourself with step by step on how to calibrate things and how to repair things with pictures and well written. I have been in this business for 20 years and I can safely tell you that you can't have too much documentation.

These are things that make you a very trusted and valuable employee.
 
To add to the list the others have provided, I always check the backup batteries on the PLC's as well. Last thing you want is someone to cycle the power with a dead battery on the PLC, especially if you do not have the latest copy of the project saved somewhere.
 
To add to the list the others have provided, I always check the backup batteries on the PLC's as well. Last thing you want is someone to cycle the power with a dead battery on the PLC, especially if you do not have the latest copy of the project saved somewhere.

Very good catch. I totally missed that one but of course we could add to this list all day.

Depending on the cost of your downtime you may want to implement as many of these suggestions as you can.
 
let me be more sepcific.
As an Automation Engineer What are the Routine Maintenance Activity you would carry out ( Dairy Plant) with more than 30 PLC and 20plus HMI cleint, with 2 centralised IO server and 5-6 Individual server, Historian.

Apart from all the backup stuff, I would:

- ensure there was an engineering alarm area, and monitor those alarms. This would be on a daily basis, say beginning of the day.
- monitor the hard drive space on all computer/servers.
- monitor the network speeds (if available).
- if a proper tool is available, trend the communication counters on the switches for malformed packets or errors on the ports. Normally you'll have a baseline measurement and if one starts deviating from it, you can do something to avoid going into fault.
- depending on the type of historian, move the data from the historian to external storage.
- clean the computers/servers, no matter how clean the room is or how many filters you have, the servers will gather some dust inside. It's worth keeping tabs on the accumulation rate and act accordingly. Before doing this, though, be sure to get pictures of every little setting of the BIOS in the servers/clients.
- This is part of the process engineers duties, but I would also launch a alarm review meeting on a weekly basis to work out nuisance alarms and push forward important alarms during processing hours to be analysed. Operators tend to forget about these things...

Nowadays you can get applications like Nagios or Zabbix to monitor the network devices via SNMP (and there are also comms drivers for SNMP to interface with SCADA).
 
As someone who came in to a site 5 years ago, that had and still has a good number of Rockwell PLCs of all vintages, and Wonderware HMI, I can add my penny's worth..

1) create an install base of your PLC hardware parts, quantities, revisions and locations, and match them to the PLC systems.
2) Create a spreadsheet for each PLC, showing the rack layout and details of each module, with details of scaling for analogs
3) detail all your software licences for PLC and HMI and gather the certificates etc in one place
4) backup PLC software/network configurations, HMI applications and config setup (depending on Wonderware versions)
5) create a backup VM for Wonderware that can run your application (s), keep it up to date with any HMI changes, as a cold backup for quick changeover
6) Think about service contract with Rockwell for spares and software, depending on age of systems, else get some spare hardware in.
7) Create a wee test room, where you can try things on a real PLC before going on plant.
8) backups, clean and tighten terminals, change batteries as said before.
 
6) Think about service contract with Rockwell for spares and software, depending on age of systems, else get some spare hardware in.
7) Create a wee test room, where you can try things on a real PLC before going on plant.

It is worth noting that some devices from Rockwell are called consumables by them and repairs or spare parts will not be covered by their service contract. The entire FlexEx range falls under this category of consumables.

I second the test room and potentially if you need a buy in from management, look into how you can turn it into a process simulator. Depends on the process, but it's always a possibility.

One thing I remembered that is quite useful is to on a yearly basis take the list created on point 1 and yearly run it through the lifecycle tools of each manufacturer so you know what is still made and what's not.
 

Similar Topics

Hi All, I am looking to copy and paste a routine. I know this has to be done offline. My question is, when I go back online, these tags are...
Replies
6
Views
480
Dear all, I want to use the interrupt service routine in FATEK PLC using WinproLadder Software. I had configured the interrupt in software as...
Replies
17
Views
1,512
I intend to pass a BOOL array to an ST routine (for loop) to count the true bits. (should never have used a Bool array, lesson learned) The ST...
Replies
10
Views
823
Hi all, I have a project which utilizes an L-33ER CompactLogix (v32), which will communicate with a RIO rack with a 1769-AENTR card in its Slot...
Replies
4
Views
747
I have a fault routine set up in my 1769-L36ERM. Is this routine called repeatedly when there's a fault, or is it only called once? It seems that...
Replies
7
Views
1,538
Back
Top Bottom