Wonderware ControlNet slow after PLC upgrade

[lostcontrol]

Posts #1 and #11.

.

You only mentioned Ethernet, nothing about DASBACIP.

 

[phuz]

You only mentioned Ethernet, nothing about DASBACIP.

My apologies.

DASABCIP for CLX Ethernet.

 

[phuz]

Still being plagued by this. The customer verified that there are no extra T-taps on the network. There was an error shown in RSNetworx diagnostics relating to one of the 1756-CNB cards, although all lights were solid green. I'll post it shortly, although I believe it to be irrelevant.

 

[phuz]

Here's the error from diagnostics:

 

[phuz]

I'm reviving this thread, because I am spending a few days with this customer next week to investigate these lag issues further. They have verified all termination resistors and coax terminations are proper in functional. This was an unlikely fix anyway as the system worked prior to the conversion from PLC5 to Logix.

Below are two graphics which they sent me. The PanelView Plus talks Ethernet/IP to the remaining PLC5 (ControlNet) via a ControlLogix gateway.

The ControlNet network is still extremely lagged, and they are not even able to perform uploads from the PLC5 through the gateway. Additionally, the PanelView plus loses communication with the PLC5 while the upload is being attempted.

 

[rdrast]

Time to get out the ControlNet diagnostics tool (1788-CNCHKR) and start checking segment by segment.

Move both terminators to the very first drop segment, with an extra T, and use the diagnostic tool, then add back in the next drop, and check again... Not fun, but it will find bad connections, noise, and other network problems.

 

[phuz]

Thank you, but we're currently exploring options that don't involve a couple thousand dollars of hardware.

As I said, this worked fine before the PLC upgrade.

System ran great --> PLC5 removed --> CLX added --> System not worky so great.

 

[nhatsen]

I think the problem is something related to software/drivers/configuration and not hardware. Can you upload the RSNetwork configuration file? Is it possible to shutdown the ethernet comms and test just the controlnet?

 

[phuz]

I plan on unplugging the Wonderware for some tests.
I will have to get the customer's permission to post the XC file on here. I have uploaded it, though, without issue.

 

[phuz]

I will also post a network map drawing that I asked them to create.

 

[phuz]

I ran diagnostics. Only 3 warnings, which can be seen below.
The PLC-5 in node 1 is the active online keeper but it keeps losing keeper connection (see top log message).

 

[phuz]

Also unable to change active keeper, despite the two other CNB cards showing as Valid. It wants to default to the PLC-5, which is Node 1, but I thought you could change it.

 

[phuz]

I bet this doesn't help.
I have no idea how to reduce this either.

 

[phuz]

We had a 16-slot PLC-5 chassis full of analog IO that had RPIs set to 40ms. I changed them to 90ms or 100ms and set the NUT to 10ms.
The Average Scheduled Usage and Peak Scheduled Usage are now 31% and 43%, respectively, so MUCH better. However, I am still seeing the PLC-5 losing its keeper status periodically:

"1/31/2017 13:17:52","CNET:80B4","No device is responding as the online active keeper on this network. The online active keeper status will be checked every 10 seconds and a message will be logged when this condition has changed."

 

[phuz]

After a day of playing with this again, I made some observations as well as improvements.

I was able to reduce the scheduled bandwidth to 9%, so that is no longer an issue. By doing so, we took the 5-6 second update delay on the Wonderware to 2.5-3 seconds. Much better but still not the 1 second we have specified in the WWRSLINX driver.

As a test, I set up a DDE topic to look at a PLC, but by going through the ControlNet network to get there. ENBT --> backplane to CNB #1 --> CNB #2 --> backplane to CLX processor. So if the ControlNet network were the issue, I would expect this communication to be slow, but it wasn't. These updates were right on the 1sec mark, as set in the DDE topic. Only when polling data from the PLC-5 is the data slow. There is also a DHRIO card in one of the gateways, which happens to be connected to the PLC-5. I re-arranged the topic to have all the WW tags go through that topic, and it was just as slow, so I do not believe the ControlNet is an issue at this point, but rather the PLC 5/80c processor's ability to handle any communications.

Today I learned that there are a total of 8 PanelView Plus terminals hitting this PLC-5 in addition to the Wonderware station, so I think we are just taxed. In the morning, we are going to swap in spare ControlNet PLC-5 just to eliminate the possibility of a bad processor, but I think the comms will be just as slow.

Thoughts?