SLC 5/05 Ethernet Locking Up Problems

Hi Guys
I could use some help please with an SLC 5\05 problem that we are having.

One of the sites that we maintain has 12 SLC 5\05 processors which are all connected via a fibre optic network over a distance of approximately 5 kilometers. The network was installed 9 years ago and has been functioning trouble free until a couple of months ago. The fibre converters are Hirschmann RS2-3TX\2FX and the fibre is multimode 62.5\125. There is no redundancy and the network is daisy chained straight through from the first to the last processor.

Also on the network are 10 PC's running Wonderware Intouch SCADA, these communicate through two I\O ABTCP I\O servers. There are also 8 off PanelView Plus HMI terminals at various points throughout the network.

We have recently had a strange occurrence where a number of PLC Ethernet channels appear to freeze up at exactly the same point in time. The green Ethernet LED goes solid and the processor loses all Ethernet communications to the SCADA PC's and to our maintenance laptop running RSLinx. The fault can occur on one or a number of PLC's at exactly the same point in time and each processor has to have its power cycled in order to recover from the fault and to re-establish comms.

The control supply to the processors is derived from a number of separate High Voltage transformers and in some cases totally seperate electric board supplies as they are located at another site. This would appear to rule out a mains power quality problem. There appears to be no pattern to the problem.

Any advice would be greatly appreciated.

regards
Steve

woodward_uk,

the first thing I would do is meet with all departments in a meeting and start explaining what is going on.

since networks just don't normally give problems without a cause, I would ask the department heads if any work has been done around your network?
has power lines been added to the transformers, has the fiber lines been disturbed, has someone been in the routers / switches, things of that nature.

you may have to trace the fibers thru the plant and in the conduits, look at each router / switch, reconnect each end - misalignment or dirt.

fiber connections do go bad now and then and have to be redone. the fiber may be getting old and need replacing - have someone test the fiber.

have you added / deleted something? another ip address or device? if so, your network may be getting bogged down.
upgraded something, a pc or application?
someone may have connected to your network with a duplicate ip address.

hope this helps,
james

What topology is your fiber network configured with?

Coming from the PC/Network world, When I have seen actions like these it tends to be the switch (central point all connections come to in a star topology network) causing the problem. You can try rebooting it next time this problem hits. Usually they come up in seconds. If rebooting the switch clears it, do you have a spare you can put in place to see if the problem goes away. If it does, faulty switch.

Part of it sounds like a ring network (The PLCs being daisy chained).

Bump, this sounds suspiciously like an issue we are having. Any suggestions?

I'm in the process of getting the firmware revision from the plant.

I would check for duplicate IP addresses assigned. Especially if a new device was recently connected to the network.

Otherwise try rebooting the main network switch, or replacing them one at a time.