MSG timing

Logix 1756-L82E processor.

I am using the .ST bit to time how long a MSG takes to complete. This works fine except for every 10th MSG cycle takes significantly longer to complete than the other 9 . Normal ST duration is around 270ms and this 10th one takes around 1.5 seconds.

I have 4 MSG's going over ethernet to 2 serial Gateways to 2 different serial radios on 2 different hilltops to 2 different PLC types.

I fire the first MSG after a 10 second rest and the other three fire when the previous one goes DN or ER.

If I decrease the rest time from 10 to 5 seconds this symptom reduces in frequency but doesn't stop completely.

I can only envision that I am seeing some sort of race condition where the CPU can't deal with the MSG flags until it completes something else and changing the order of something will correct this. PLC load is less than 1% , there are no continious tasks and I am monitoring the .EW bit to see if the Queue is filling up. I don't see that happening either.

Its not really a problem yet, but I just want to be able to explain it so I can anticipate it. I need to add a whole lot more to this sequence.

You might get better results if you have one delay between each message. I typically poll radios with logic that waits for a DN or ER bit after each message, then an "inter-message delay" then fires the next one. I time the whole cycle too, and use that for crude diagnostics along with a "data age" which is the number of seconds from each remote site since the last successful read. For the serial radios we mostly use, an inter-message delay of 100ms is sufficient except for one site polling oddball RTU slaves that need a full 650ms to keep them from locking up.

The polling logic is just a state engine where I can turn off sites and skip those states, and the screenshot has my inter-message delay labeled as "POLLING INTERVAL". This is a Red Lion HMI and the PLC behind it is a Micrologix, so there may be some differences in how you'd have to code it for a different PLC.

I used to do that inter-msg delay as well but I am sequencing them now using the EW and ST bits so I'm fairly certain none are being queued.
In addition one of the MSG's in question is the first one after the 10 second rest period so for that MSG at least that delay is already there.

I even thought there may have been a setup period after a 10 second rest but the other MSG that does this is the 3rd in the sequence and it doesn't do it at the same cycle as the first, just at the same interval between occurrences.

I have since changed up some rungs adding traps etc and changed its interval to a more random pattern. but its still there.

More interesting than troublesome at this moment, I have concerns how this may escalate once there are 100 of these.

What is the value of the .EN_CC bit?

Maybe the network is causing delays; can you do a tcpdump or wireshark to see if any packets are being missed, not-acked, and/or re-sent

If its a network issue I wouldn't think it would be repeatable and regular like it is. And bear in mind I have an identical MSG to a different location across both hilltops and its just these 2 messages that occasionally do this.
The connected flag is the only thing I'm not certain of on the others but a while ago when I sent a connected msg to a 5380 the MSG will simply fail due, I am told, to an incompatibility with the serial module on a 5380, so I am certain that MSG at least is not a connected MSG.