DLR Ethernet troubleshooting help

I've been trying to troubleshoot a DLR network for the past month with some forward progress, however I've exhausted all of the things to try that dont require downtime. This is a new installation down in mexico and here is some "useful" information that I can supply:

We are getting ring faults about once or twice a day, which isnt localized. Many times the faults are shutting the line down because it is dropping out a section of the ring, not just a single point ring fault.

The ring consists mostly of 294e armorstarts and a few Point IO with a safety module in it.

There is a group of 3 armorstarts at the far end of the ring all within 1-2ft from eachother and the middle drive is receiving most of the media errors (FCS, alignment, mac transmit, and mac receive errors), the drives next to it are getting a few errors but not to the extent the middle drive gets.
The motors are about 5ft away from the drives with shielded Belden VFD cable with the shields terminated on both sides.

I hooked up a scope from the ethernet shield to the conveyor structure to see if there was any noise being induced into the ethernet cables and found that there is one drive that has lots of noise, but it is the drive next to the one with all the errors.(it was about 20v peak, before we shielded both ends of the cable, then it dropped to 10v. The Armorstart housing ground terminal is at the same potential as the ethernet shields.)

Before I start going off on all the details of **** I tried Im wondering if you get media errors on a certain node does that mean the data is getting corrupted inbetween that node and the next? or could the data be getting corrupted somewhere else on the ring and the other nodes before it just pass it along if its not their IP address? I havent found any useful information on the method on how data gets transfered throughout the network at the most basic level and how/when it determines if there are errors in the packets?

Any information would be great! There are still many "bad practices" left from the install but nothing sticks out as a potential problem.

Jebel said:

This is a new installation down in mexico and here is some "useful" information that I can supply:

Click to expand...

Jebel, I got a tell ya. I was really disappointed in one of the pieces of Allen Bradley hardware we recently used in a LARGE network we recently started up. The network consisted of 3 DLR's connected via Fiber. We kept having MAJOR portions of the ring faulting the network with several devices "dissapearing".

After some troubleshooting, we found that the new PanelView 7 - 6" model with the plastic housing had very weak ethernet jacks and the cables would not seat correctly. Everything would act fine but anytime the machine started up and started vibrating, the connectors would work their way loose and fault the network.

Not saying you have the same problem, but make sure all your cables are seated well. Give'em a light tug to make sure there not shacking loose.

What media are you using for the DLR connections? Copper or fibre?

If copper, is it shielded correctly? Shielding should be done at one end only to prevent ground loops.

I am actually going to re terminate the shielding on all the cables. I spoke with an AB rep on the phone and he insisted that you can shield both sides of the cable due to the Resistive/capacitive circuit coupling the shield to ground on each node. however it still makes me wonder if that isnt the case.

I hate to think that its some AB firmware/hardware issue and would hate to go down that path, but I have seen some weird **** come from AB and it would be a mistake to assume anything. And if lets say it was a bad com module, how would one diagnose it? I mean if it spits out **** across the network causing communication issues somewhere else on the ring it would be difficult to pinpoint the problem, considering there are probably 25-30 armorstarts on the ring.

First things first, double check all the physical media connections and shielding. The media errors you are seeing points to some sort of data corruption.

Double check all your VSD cables shielding and earthing as well. Switch to EMC glands if not already being used.

Also is it possible to re-route the Ethernet cabling away from VSD cabling? Is there adequate segregation already?

Another thing to check is the duplex settings of all devices, make sure there are no mismatches. If it's set to auto make sure everything is set to auto, if it's set to full make sure everything is set to full. Definitely no half-duplex.

Ring Supervisor(s)

We assume that you are using a Control Logix with a single DLR scanner in the rack or a Compact Logix with DLR capability.

DLR scanners can be deployed in a pure RING topology, a LINEAR topology, or combination of ring, linear, and star. We assume you have a single RING with possibly a few E-taps to support single-port devices to be on the single ring.

If you are using oscilloscopes, and contemplating shielded re-strategies, then we assume that you have diligently, physically confirmed, your actual point-to-point Ethernet cabling matches your documented design.

Being that your DLR scanner is the SINGLE point of data connectivity between your PLC and your I/O, this device, by common sense, should be configured as “The” ring supervisor. Also, this scanner can provide you the diagnostics of your ring to the PLC via some basic message reads.

BUT. There are other devices on the ring that have the capability of being Ring Supervisors as well. (I have no idea the benefit of multiple ring supervisors).

BUT, if you do have more than one ring supervisor, you must establish a hierarchy while configuring each device. (See Rockwell Knowledge base).
If you have inadvertently made the check-box in one or more downstream devices on your ring also as ring supervisors, without also assigning a unique priority order to each, you may experience symptoms as you describe.

Note: This information is directly related to a two-week hunt of the same symptoms on a new machine we deployed. We found this by fluke, as we were going to configure a new DLR scanner, and try a hardware swap, and when looking at the DLR status in the scanner, we saw the actual supervisor IP address was a downstream device. Once we turned off all downstream supervisors, our problem was resolved.

Hope this helps.