Allen Bradley PLC and VFD network issues.

Has anyone ever encountered an issue where PLC's seem to lose connectivity momentarily, as well as all ethernet VFD's in different areas across the entire network seem to fail with an ethernet fault? The newer powerflex 525's have an ethernet fault of some sort which I haven't seen myself yet.. and the older style I have that use the 22-COMM-E cards also fault with a "Port 5 DPI Loss" at the same time.

My first thoughts were that the main network switch may be going bad... however the strange thing is that I cannot recreate this problem by removing the ethernet connections, or powering off the switches, or anything like that... I can't recreate this network wide faulting issue at all on purpose.

So it almost seems like there's something or some signal on the network that's knocking these out somehow every so often .. (like maybe a few times a week).

Just wondering if by chance someone else has this same issue and has idea what it is, or if its allen bradley related or not.

Thanks!

I I have experienced this sort of thing many times. More often than not, baud rate settings tend to be the cause. It is possible that the switch is the issue, I've had failing switches behave like this, but I'd let some statistics do the talking:

The first place for troubleshooting is in the web page of the PLC. There will be a bridged connections and/or application connections tab, look for missed/rejected packets. This value should only be 0, anything other than 0 means trouble is brewing. Also look for FSC and alignment errors on the general statistics tab. If you can post screenshots of these diagnostic pages, that would be helpful.

I have added a few screenshots below from one of the PLC ethernet cards. I didn't see any errors in the general section, however i do see errors in both application and bridge connections.. please see attached. I'm not familiar with what any of these mean.

Hmm, what are those devices that are accumulating the Missed Rx Packets? From my experience, the only acceptable number in that column is 0.

Interesting connection serial numbers, are they something home grown? The serial numbers are duplicated for more than one IP address.

10.200.20.42 - Flex IO, 1794-AENT
10.200.20.62 - Comms to other plc, 1756-ENBT/A
10.200.20.40 - Flex IO, 1794-AENTR
10.200.20.43 - Remote I/O Rack In panel several feet away, 156-ENBT/A
10.200.20.39 - Allen Bradley Powerflex525 VFD

I'm not sure about those connection serial #'s

Can you check and see what the baud rate of those devices are? Obviously don't change anything unless it is safe to do so, but they should be set to Autobaud in almost all cases. I've seen this sort of issue when you have a bunch of devices plugged into a switch and one is set to fixed 100MB Full Duplex while the others are Autobaud, for example.

Edit, there have been rare situations where a device would mis-negotiate when set to auto and would end up at half duplex or something like that, but that is rare.

10.200.20.41 PLC (100mbps, full duplex, auto speed and duplex)
10.200.20.42 - Flex IO, 1794-AENT (100 mbps, full duplex, auto speed and duplex)
10.200.20.62 - Comms to other plc, 1756-ENBT/A (100 mbps, full duplex, auto speed and duplex)
10.200.20.40 - FLEX IO 1794-AENTR (100mbps, full duplex, auto media speed and duplex found)
10.200.20.43 - Remote I/O Rack In panel several feet away. (156-ENBT/A) (100 mbps, full duplex, auto speed and duplex)
10.200.20.39 - Allen Bradley Powerflex525 VFD


They seem to be all set the same. These devices are all in the same area connected with a small managed hirschmann switch. I'm trying to install some software to access that switch.

I thought those hirshmann switches had a web interface.

they do... java based though. the software has it built in. HiView.

I got it installed but don't really see anything of interest... all ports set for automatic config of 100 mb/Full Duplex.

75 vs. 85

A little perspective on "Port 5 DPI Loss" on the PowerFlex 7-series drives.

Drive Peripheral Interface (DPI) is a CAN-based onboard mini network used by PowerFlex 7-series drives. It can be thought of as an extension of SCANPort, used back into the 1990's by the 1336+/1305 family of AC drives.

Because it's very small, and CAN is very noise-resistant, it is very reliable. When you get a DPI fault, it's usually because of a badly damaged cable or connector, or a very strong EMF interference, or an actually disconnected or failed peripheral like a HIM module or 20-COMM-x.

DPI Port 5 is the connector on the main control board that hosts the 22-COMM-x modules.

A "DPI Loss" fault means that the connection on DPI between the module itself and the main control board has failed. The fault code itself is "85".

A "Net Loss" fault means that the connection over the network (Ethernet, ControlNet, DeviceNet, Profibus, etc) has failed, but that the DPI connection is still up and running. The fault code is typically "75".

It is not impossible for a DPI fault to be related to a firmware bug, but it is usually caused by a very strong EMF interference, or poor grounding, or a damaged ribbon cable.

A source of poor grounding that has surprised me in the past is the actual screws that mount the 20-COMM-E. You'll see that they pass through tinned contact pads on the printed circuit board (which are connected to the ground plane/bus on the board), and the machine screws thread into nuts molded into the plastic of the drive enclosure.

If you look closely, you'll see that three of those are brass and one is plated steel, on the lower right side. That steel one is connected to the drive's chassis ground, and is required to provide noise immunity for the 20-COMM-x board. For some reason, that's the screw that some installers drop, forget, or leave loose.

The lost packets also suggest cable and noise problems elsewhere in the system, or broadly distributed in the facility. But I do recommend making sure that your drives are at least grounded and bonded correctly, including the network interface modules themselves.

I have seen this before some of the CompactLogix processors will only allow a connection with (I think it’s) 5 Ethernet addresses, when you add the 6th it will start randomly dropping one of the addresses.
Rslinx will usually show it a yellow flag
The only fix for this is to upgrade the processor to one that will support more IP addresses.
The way to test this would be to disconnect one and if the remaining work ok then you found it.
Contact Rockwell support they can explain it better

I have had this exact same thing happen not 5 months ago. Ethernet card connected to 16 port switch. Processor would momentarily lose comms with everything on the switch. Eventually found the switch to be defective. Replaced the switch and it never happened again.

The Hirshmann switches with the old Java web interface , in particular the MICE variety, did not play well with Autonegotiate on older devices. Just food for thought.

There are a dozen Hirschmann RS20 RailSwitches in our surplus shelves and I wonder if their Autonegotiate issues are why.

It would be very interesting to set up HiSense or an SNMP based monitoring system to watch for lost packets on a particular port, and/or link re-negotiation events.

I don't know how to monitor Rockwell devices themselves for link up/down events that don't result in a CIP connection loss.

If you hard set the speed and duplex it works. Of course the Java interface is enough of a problem to make work.. like an Altivar Web Interface.

BTW, the latest version of Hirschmann Firmware (9.0.04) removed 10HDX from even being an option. If you have a PLC-5 with a AUI adapter, you have to have that ability.