Active Logic

What can be done to prevent equipment from shuting down when I loose comunication or a PLC fails?

I guess I would like my equipment to remain in the "last state" condition if it was running -keep running, if it was off stay off.

Recently we experienced a PLC failure where all my chillers stopped, cooling water valves slamed shut, water pumps stopped, etc.
Which made us loose production.

You've identified two different scenarios with widely different outcomes.

When you 'loose communication', what do you mean? Communication between a operator panel and the controller doesn't necessarily have to mean that you shut down the controller, so long as the system can be maintained in a controlled state without the operator having access to the panel.

If your PLC fails, or drops out of run mode, then your process is no longer being controlled. The controller can't respond to changes in the inputs, so you can't say for sure that holding the ouputs in their last state is the correct thing to do. If the the controller shuts down, unless you can switch to a hot backup controller, the outputs need to be turned off or returned to their default state.

You have to ask yourself; "If the PLC fails, and the system can no longer be reliably controlled, would I rather loose (A) production or (B) people's lives?"

I want the equipment to stay running, I guess you can call it "uncontrolled". I understand where you coming from. In a lot of automation applications the desired thing to do is shut down the equipment so you do not risk lives. We run into that in our manufacturing side of the business, and their philosophy is just that, and I agree with that.

In a chiller building where you are running chillers, valves, pumps, fans. The meaning of running "uncontrolled" means that evarything remains in the last position that it was, if the chiller was running I want it to stay running, same case for cooling water pumps, and cooling tower fans, and for supply valves if they were open keep them in that position, I don't want them to close. The philosophy changes, there is hardly any risk of keeping everything the way it was before the PLC failed. You just won't be able to do some things and you will notice those. Then you will have to do something to fix the real problem but in the mean time you didn't interrupt production.

So I guess my question is, is this an easy thing to program, or what do I need to do to the programming to allow me to keep my equipment running in case of a PLC failure.

PLC or Communications Failure

With most of the applications I work with I incoporate what I call a "Life Relay". As long as this signal is true the devices in this control system are working correctly. If this signal (or set of signals) is missing a piece of equipment is no longer responding and usually sets an alarm and/or halts the system and succeeding steps.

With communications I establish a "Watch Dog" signal. Since most of the communications is event driven in many applications, if a communications update has not been sent in a given amount of time, then a message is simply sent as if asking are you there and a reply yes I am. If this communications is not completed then an alarm is generated and/or the system is halted or the succeeding steps are not continued.

As Mr. Bailey explained in the above post I prefer to set the PLC in the Outputs Off State whenever possible due to a PLC failure. Safety for the Human Beings around the equipment is always more important than production of product. There have been some projects though where it was safest to halt the equipment in the last state and sound the alarms. This may require wiring in some external hardware to monitor equipment operation.

I hope this helps your situation...

Thanx Dan

Hey Lupasi,

From what you are describing, I guess you may mean the human to machine connection. There are several ways to address this in the programming, to where if an operator puts in an invalid entry, the machine will remain to where it was and reject the input, letting the operator know they made a boo boo. There are several ways and philosophies that will create this type of fix for you... It does take time to implement, since you essentially have to think of every thing that can go wrong. Murphy's Law is still alive and kicking

If you mean something like a power failure, there are ways you can fix it so the machine keeps running. Battery backup for the PLC, for example, or for the voltage in the controls. As the case with most safety circuits, etc... you can setup a backup power source to keep what you need running, but most companies do not view them worth the cost. Again, for safety circuits, etc... you can wire the switches so that they work with either power source, main or backup.

If you could give an example of what exactly happened with your chiller, we may be able to come up with a more concrete answer. Could be programming problems, could be electrical issues. Can go either way...

Hoot

So I guess my question is, is this an easy thing to program, or what do I need to do to the programming to allow me to keep my equipment running in case of a PLC failure.

Click to expand...

To rephrase your question, it almost seems like you're asking "what can I do to the PLC program to keep it running when it stops"? I know you're really not asking that.

Probably what you'd need to do would be to put the ouputs in the chiller building (pumps, valves, etc) on some sort of remote I/O or fieldbus network. On those systems, there is usually a configurable parameter that tells each node what to do when it loses communications with the master controller. The options are usually 'all outputs off' or 'hold last state'.

Tell us what type of PLC you're using and we might be able to point you in the right direction.

RE: PLC or communications Failure

I had meant to add this to my previous post, but was on the phone at the same time I was replying. I'm afraid I'm getting to old to do more than one thing at a time.

Some PLC systems allow you to setup a PLC CPU to run it's own program while being a Remote I/O node to a master PLC. In this case if your main PLC system locks up or crashes this CPU will continue to operate it own program. But if the PLC CPU in the Chiller room is the one that fails then your choices appear to be the same "hold last state" or "all outputs off".

Thanx Dan

I have done similar things in the past where the PLC interface to a motor starter is two interposing relays - one to close the start circuit and one to interrupt the stop circuit. This was in applications where it was important that the pumps continue to run. Use a relay with a NO contact in the start circuit. Turn the output to this relay on to start the device until the feedback tells you that the device is on. Use a relay with a NC contact in series with the stop circuit. Turn that output on until feedback tells you that the device has stopped. If the PLC fails the device will continue in the last state. Also, this allows manual control when necessary. I usually take an input from the HOR (hand-off-remote) selector to disable PLC logic when in hand operating mode. This will allow the SCADA system to inform the operator when the system is not capable of auto oeration.

Several PLC manufacturers offer "process control" analog modules that retain their last state. Some even have digital/discrete offerings.

For example, the 170ADM54080 discrete I/O module, from Schneider Electric, has a control word bit that can be set to define each output's shut down state (on, off, or last state). This line, Momentum, of I/O modules has communication adapters for use with Modbus, Devicenet, LON Works, and Profibus DP.

With Allen-Bradley enhanced processors you can select your outputs to remain in their last state when a fault occurs.
You can also run a fault routine prior to a shutdown.

Roger

There are a number of techniques that can be combined to achieve your result. You can set up your run command to pump starters as three wire control, using a relay with a holding contact to hold the run relay and keep the pump on. Use another relay with a NC contact in series and use a "pump stop" output from the PLC to trip the relay for normal stopping. Then, if the PLC dies, the pump will stay on indefinitely. If the pump is off when the PLC dies, it will stay off.

Most valves with positioners can be configured to "hold last position on loss of signal" so that if the analog signal goes to 0 mA the valve won't move. If you are using motor operated valves for on off service, they will naturally stay in the last position.

Most VFDs will also let you configure them to go to a set speed on loss of the speed command.

Before implementing this make sure you have clearly analyzed the results of the failure mode operation on safety.

Thanks to all of you. I will have the people in the plant try some of these and will get back with you to let you know what happened.

Thanks again!

One other thing you want to add is a watchdog relay or timer so you know the PLC is brain dead. There are a lot of ways to do this, but the simplest is to use on PLC output that is always on to hold on the coil of a "PLC OK" relay. When the PLC drops out, the relay drops out, the contacts change state, and you can use them with a light, or horn, or annunciator so you know the PLC is not controlling.

I have a plant where we had 60 machines to be put under PLC control. Each machine requires 5 or 6 inputs and outputs. It was first posed to use a remote IO setup at each machine and run everything off of one PLC. When I presented that this could cause all machines to quit at one time and loose 60 batches of production at one time, I was asked to come up with a different scheme. What I presented as solution and put into motion was to use small micrologix controllers 1 each to control 4 machines. I used DH485 to connect all of these to a single Panelview 550 controller (Allen Bradley was required by my company). This allows setup of each machine seperatly and data aquisition at the panel for the whole area. This way a loss of PLC would only shut down it's local area or loss of data highway does not prevent everything else from working. If you have to add a lot of latch / unlatch relays and stuff, you might consider this sort of scheme or something similar where each area has it's own controller (with all of it's safetys working) independant of all the other systems. Then use a supervisory PLC or HMI to run the whole area. I know chillers and pumps etc have pressure switches and freeze alarms and such that may need attention, a small plc could attend to these while you fix another part of the system that's bad.