E-net problems

I wanted to get some ideas about how to troubleshoot something here at work. For the last week or two(details are sketchy) we have been getting communication failure messages on all our HMI's that are on Ethernet(4 hmi's: 3 rsview32, 1 wonderware). On monday, one of our plc5's (ethernet) faulted out with a comm failure message. This suggests to me a problem with the ethernet, but I am a little vague on where to start. Any suggestions?

If it used to work, and now it doesn't, then something has changed.

1. Track down who has had any access to the network configuration at any point recently, in the HMI's, the PLC's and the network switches?

2. Buy an Ethernet media tester, and test all the cables.

3. Analyse any physical changes that have occured locally, power supplies, RF noise sources, environmental changes, other equipment failures.

4. Start substituting, ONE item at a time. If you don't have substitutes, MOVE things around and see if the fault moves with them.

4. If the simple steps above don't help, then consider hiring a network specialist for half a day. These guys have traffic analysers, and other neat goodies. If he is any good he should be able to nail it within a few hours.

5. Log each step. If it all turns to custard you will need it to CYA.

I started out by hooking up a fluke network assistant to our ethernet network to monitor the systems. I left that hooked up for a couple of days, and monitored the data recorded. Everything looked fine, except for the fact that every 4-8 hours the network reset itself. I dug through the rockwell forum and discoverred a "storm counter" for the plc5's that can be setup up to detect ethernet storms. Nothing there. While I was disconnecting the fluke, one of our lab techs said "oh ya. That thing under the counter is still beeping. Are you guys ever going to fix that?" I then replaced the battery for the ups powering our ethernet switches, and the problem went away. <SIGH>..... At least now I know what the network looks like when everything is working fine, so when an actual problem pops up I'll have a baseline to compare it with.
Thanks for your help!

Yes!!!!

"oh ya. That thing under the counter is still beeping. Are you guys ever going to fix that?"

Click to expand...

some days we win, in spite of the folks we are trying to help!