Powerflex not reporting to HMI or PLC

This is a troubleshooting problem. I have two AB Powerflex 700 VFD drives that report status on the HIM but do not always report thru RSLinx to the PLC & HMI. They used to work properly but I am not sure what may be going on. One of them usually reports but not always the same one. AB has a report but it is locked. can anyone help?
27318 - Unable To Communicate With A 20-COMM-E Through RSLinx Or Drive Viewing Software.
When having problems connecting via Ethernet using Drive Executive software and RSlinx (either drive does not show up in RSlinx, or drive has a red X through it in RSlinx), but can PING the drive’s...
Access Level: TechConnect

Thanks!

Here you are:

Does the I/O connection between the drive and the ControlLogix actually break ?

This could be an RSLinx or PC operating system problem, or it could be a network or 20-COMM-E problem. If the I/O connection is breaking, then it's probably not RSLinx or the PC.

Do you have a lot of EtherNet/IP I/O connections running on this network ? Are you using properly configured managed switches ? Have you checked the diagnostic counters on the embedded web pages of the ControlLogix Ethernet ports, or on the 20-COMM-E, or in the 20-COMM-E's Diagnostic Object ?

This system is about 8yrs old using redundant SLC 5/05s. This error occurred about 1 mo. ago. It has two Ntron unmanaged 900B switches. 1 has the PLCs and drives, the other has the HMI (RSView32) (Win2000)and remote connections in which two additional HMI's (XP & Win2000) are running. Getting similar event log errors in each RS Linx driver diagnostics. For the drive that is currently not responding I get 'Connection Refused' locally and remotely 'Connection timed out'. I can ping and get to the web page but the web page does give me some timeouts. On the web page under tcpconfig it reports EN Rx Overruns =75, EN Rx Errors =1. I currently cannot bring up the webpage for the drive that is currently has a working connection. 3 days it was not working properly and reporting very similar reports in RSLinx but also reports some 'no routes to host. I am not having any PLC issues on the same switch. This is a remote system and I don't currently have anyone onsite. I have access to the network remotely and also local PCs.

As long as the drive shows up with a connection in RSLinx I can connect with drive executive. Either one will sometimes show up. RsLinx log shows 62 opened, 1 lost, 23 refused, 39 timed out for the drive currently not connected.

@Widelto: Thanks for the info! unfortunately it doesn't help much. Best regards,

Are these SLC-5/05 controllers sending MSG instructions to the drives to perform control functions, or are the drives hard-wired to the controllers ?

I am a prominent advocate *against* that method

Those "Connection Refused" counters are very interesting, as are the RX Overruns.

Is there any fast multicast or broadcast traffic on this network that you know about, like maybe a ControlLogix in the mix, or a video system ?

I'd also check any RSLinx instances that can connect to the network and make sure that the RSWho isn't set for Auto-Browse. That can fill up the CIP Connection buffers of low-capacity devices like drives (and infamously, Net-ENI's) quickly.

The SLCs are sending MSG instructions to the drive but only for reading status feedback. The control is hardwired and the drives are working as commanded. The PLC reads the MSG and the HMI reads the tag from the PLC. So I suppose the RSLinx is just an indicator that there is a communication problem between the PLC and the Drive. The PLC MSG reports connection broken error. I unchecked auto-browse in all related RSLinx. There are many other devices on the network including an audio/video system(analog cameras connected to a DVR) that is viewed remotely 24/7. I know that is not ideal but it has worked for years. There is a 1755-L1 GuardLogix PLC also.

Wireshark Session shows interesting activity beginning around Item 534 but I am not sure what it means. 10.80.26.195 is a SLC5/05 that is redundant with .196 which currently is primary. It looks like alot of multicasts from the NTron which is unmanaged. I can't seem to get the OPC data from the Ntrons using their OPC tool either. I am thinking I should go ahead and replace these switches with managed ones and also at least one of the 20-Comm-E.

Thanks for getting more information on the topic !

Posting native *.PCAP files is OK; anyone on the Forum who's going to analyze Ethernet is going to have Wireshark installed. It's also easier to see the whole data payload and sort and search.

The N-Tron traffic is usually harmless; this is the broadcast traffic that their "-N" monitored switches send out with the port status information, which is picked up by the NView OPC software.

In your case I'm seeing the N-Tron switch sending out these broadcasts every 1/2 second, but in groups of two only a few milliseconds apart. I've only ever seen an NTron switch send out one at a time.

To narrow down the N-Tron traffic, use the Wireshark display filter "eth.type == 0x8874".

I don't know if that "double tap" of N-Tron switch broadcast tells you something about the N-Tron itself, or something about the network at large. Could it indicate a Layer 2 loop ?

The fact that you can't get that data into NView suggests something about the operating system firewall on your PC.

The traffic from the SLC-5/05 at 10.80.26.195 to the drive at 10.80.26.197 consists of a [SYN] packet from the SLC to attempt to open a TCP connection, and the drive responding with [RST,ACK] indicating that TCP port is closed. The destination port in your capture shows as "EtherNet/IP-1" which should be TCP Port 44818.

This suggests that the drive is not accepting any more TCP connections, which is consistent with the behavior that you can't browse it with RSLinx Classic.

I don't see any completed messages between the SLC and the drive in your capture, but you stated earlier that the MSG instructions were working.

Which exact port was this capture done on ? Did you use a hub, or a mirrored switch port ?

To troubleshoot this system further, I would set up a capture at the drive's Ethernet port itself. I would do a full capture for a few minutes, then use a Capture Filter "host 10.80.26.197" to capture only the data addressed to the drive, and run that capture for several minutes. If you can perform a reset of the 20-COMM-E and capture the traffic before and after that, it would also be useful.

I realize you're not onsite, so I'm impressed you've been able to do so much already.

I'm not sure if the 20-COMM-E shows you how many TCP connections or how many CIP connections are currently active, and don't have one handy to test.

If you're going to capture the embedded Web Pages of the 20-COMM-E (or any other embedded device) I strongly recommend HTTRACK software. It's a lot simpler and more efficient than screen captures.

I realize you're just trying to troubleshoot a drive, but once we get you a little taste of network analysis we'll try hard to get you hooked.

Because you're not onsite, I can see you're using VNC. There are another couple of ways to do this with even less bandwidth.

Check out the weirdly-named Wireshark fan site LoveMyTool:

http://www.lovemytool.com/

Search for "remote capture" and you'll find Tony Fortunato's lecture from two years ago about using rpcapd.

If you find yourself buying old thin terminals on eBay and fitting them with Linux to ship off as network probes... well, welcome to the club.