[Logix] Intermittent comms issue (ethernet/ip) between Control and Compact Logix PLCs

[defcon.klaxon]

For the record, "Implicit" is a Produce/Consume (technically known as "Ethernet/IP UDP") and "Explicit" is message based connection (technically known as "Ethernet/IP TCP”). As the technical names imply Produce/Consume is a UDP connection and Explicit is a TCP connection. One of the big differences is that a TCP connection is a connection that is opened, data is transferred and then the connection is closed. UDP is a connection that once opened remains that way continuously for “real time” data exchange. Produce/Consume was created for “real time I/O” and it is much faster at reading/writing I/O. The problem with it with regards to wireless is that it uses packet count information to determine the health of the connection specifically the number of packets lost in a given time frame (calculated via “4xRPI”. There’s more to it than that but it’s over my head). If you drop more than 4 packets in a specific time it will decide that the connection is broken and reset it. No matter what kind of wireless you’re using you will always lose packets which is why Rockwell says not to use Produce/Consume wirelessly. However if you can keep the number of tags low and tolerate high RPI times, depending on the radios speed, bandwidth and quality of connection you can make it work.

Thank you SO much for that explanation, it really makes things so clear. That is such valuable info to understand.

It sounds like you’ve got a good handle on where to go from here. Please do let us know what you come up with and good luck.

So I figured out a few things that are relating to my problem. First, the flickering faultcode was entirely spurious; the problem was that I fat-fingered some copy and pasting from that rung of ladder logic for other sites, and didn't change the destination tag on one instruction. Thus, why the faultcode (from a site that is not connected) would show up. So THAT is the first part of the problem.

The second part is, why is my data taking so long to show up? I think I figured that out too. I used the MSG Manager example code that I mentioned earlier; HOWEVER, I put in changes to speed up the scans and I think that's what broke things. I have code that checks the faultcode and if comms are lost, I bypassed the next group of MSGs...otherwise it sits there for several seconds before the MSG timeouts. But the problem seems to be that if the other sites are all down, the scan doesn't have time to unlatch the coils that control which group of MSGs to send, so it never resets. When I put a few timers set to a couple milliseconds in each group just to slow things down a bit, it worked perfectly. So, the root cause has been found and eliminated. The question now is, how can I reliably keep my MSG manager running without getting bogged down waiting for MSGs to time out.

WOW. What a great example of learning about code, but damn was that a big chunk of time. At least, thank goodness, I am tackling this here in the office and not at the water treatment plant!!

 

[ASF]

I generally have the manager cycle onto the next one at preset intervals, even if the last one is still "stuck" waiting for a response. When things are going well, I have only one MSG active at any given time; if a couple of sites go down you might have two or three (still not a big drama), and if all of the sites go down, well, you've got bigger s*** to deal with than how many messages you're trying to send at once.

FWIW, you can change the settings for timeout and retry on the MSG instruction, make it a bit quicker to raise the white flag when things go wrong. You have to change some of the values in the tag browser for the MSG instruction - I can't remember which ones off the top of my head but google can probably find out for you

 

[defcon.klaxon]

I generally have the manager cycle onto the next one at preset intervals, even if the last one is still "stuck" waiting for a response. When things are going well, I have only one MSG active at any given time; if a couple of sites go down you might have two or three (still not a big drama), and if all of the sites go down, well, you've got bigger s*** to deal with than how many messages you're trying to send at once.

That's an interesting idea; you could just have it advance to the next MSG after a time delay (and I'm sure you could fine tune that once you're in the field to not have it excessively long, or conversely too short). That's a great idea, thanks!

FWIW, you can change the settings for timeout and retry on the MSG instruction, make it a bit quicker to raise the white flag when things go wrong. You have to change some of the values in the tag browser for the MSG instruction - I can't remember which ones off the top of my head but google can probably find out for you

Sometimes the obvious escapes me. I didn't see any settings in the MSG configuration popup, but it sounds like I can go to the tag broswer and change them there. A bit frustrating that they'd not have that changeable in the popup, but lesson learned; check the tag browser for settings that may not be found in config popups. Thanks!!

 

[Phil Buchanan]

That's an interesting idea; you could just have it advance to the next MSG after a time delay (and I'm sure you could fine tune that once you're in the field to not have it excessively long, or conversely too short). That's a great idea, thanks!

What ASF gave you is a very common way of doing this so comms don't stop or get stuck from a single error. I think it's the best way to handle the setup of multiple messages.

I have also added counters for each to count the errors and setup logic to clear the count once a week or once a month. If you start to troubleshoot you can see your culprit very easy and it's good info to help determine the problem.

 

[ASF]

Sometimes the obvious escapes me. I didn't see any settings in the MSG configuration popup, but it sounds like I can go to the tag broswer and change them there. A bit frustrating that they'd not have that changeable in the popup, but lesson learned; check the tag browser for settings that may not be found in config popups. Thanks!!

Yeah, I can't work out why it's not in the MSG configuration popup either, apart from maybe to stop Bubba playing with it. But yeah, it's available. I think that there are two settings you have to change - the retry interval and the timeout, I think from memory. I think they may also be in different units, so just double check before you dive in