L33ER CompactLogix V31 comms Issue

Hello All,

I'm stuck and have contacted AB, and still no help.

I have recently upgraded 2 of 4 L32E V16 controllers to L33ER V31.011 and everything seemed to be fine. However we randomly will lose DCS (yokogawa centum VP R6) comms to the L33ER controller which communicates via an ALE111 card from the DCS over ethernet/ip comms to the PLC controller.

99% of the time there is no issue with the data coming back to the DCS however, every few hours we see a fault on the DCS (A320 - 'connection shutdown when transmitting to the connected socket') which quickly recovers (as per the retry time set in the ALE card - 30 seconds). this communication link between PLC to DCS has been fine for YEARS and since changing from L32 to L33 we just started to see this issue, and only on the updated (L33ER) controllers, there are 4 PLC controllers communicating to the ALE card of the DCS and only the 2 x L33ER controller are bringing up this bad data issue.

One thing to note is that when these events happen and we loose PLC-DCS comms, I can continue to ping the PLC and ALE cards fine with no lost pings, also Studio5000 does not disconnect, nor does the controller count any faults, nor does it show any minor/major faults. Looking at the diagnostics in the controller via studio5000 can see no error counters counting (FCS error = 0, so no issue with duplex that I know of..)

Diving deeper, the only thing I can see that looks of interest is when I log onto the webpage diagnostic and view the advanced diagnostics. I was expecting to see the fault counters register some issues however nothing sticks out, the only thing I notice is under the 'EVENT LOG' i constantly get 2 messages up:

1.) CipStatus: Generated on device
2.) Connection time out notification

These two messages come up every 52 minutes and dont specifically coincide with the comms fault seen on the DCS (A320).

I seem to think the constant spamming of those 2 events is a clue however, they also could be normal. I tried upgrading the FW to the latest version V33 and that still sees the same spamming of those 2 events, so i suspect maybe it is normal? I havnt waited long enough yet to see if I still get the comms fault on the DCS with the FW33 installed on the controller but I am going to assume I will. Also the L32E webpage interface does not offer the same table, so I cant make a direct comparison.

To complicate things, I can get the 2 events to stop spamming by downloading the properties of the ALE111 card to the DCS again and for some reason the PLC displays an event log message - 'CloseNode() occured, followed by a connection timeout notification. and then problem solved, no more spamming in the event log nor any comms drop out on the DCS. This stayed true for days...

I thought that was the fix, until i went to clear the eventlog by cycling the power to the PLC, which did clear the event log and history, but also reintroduced the spamming event log messages as mentioned above again, and also I started to see random DCS comms loss again (back to square one)...re downloading the ALE111 card properties stopped the spamming but thats not the fix as I now know, cycling the power to the PLC will bring this issue back.

any idea if the event log spamming is normal?

Cheers,
SJ

This isn't a fix, but any reason you can't try/use the latest version, 33?

Hello, thanks for the reply. I have tried V33 in the controller and a brand new ALE111 card in the DCS as well and still no luck. Actually generating a different DCS fault but regardless still seeing the same event log messages in the PLC via webbrowser.

Do you have an Ethernet TAP so you can get some Wireshark traces, to try to figure out the problem?

I have not yet, I am off site at the moment but will work through to that point. Was hoping someone knewhad seen this before. Specifically the messages in the event log of the L33ER controller.

The log simply says that the connection is timing out. There's many reasons why this could be the case. Not possible to tell much more than speculations.