Hi all,
Got a customer with a control system architecture like this:
The L71 uses explicit messaging to read and write to and from the ML1400. It's been in place and working fine for about 4 years. The L71 also talks to one other ML1400 with more or less identical architecture, and about 5-6 CompactLogix/ControlLogix PLC's using produced/consumed tags, again with similar physical infrastructure. The ML1400 doesn't talk to anything else.
About 4 months ago, they started getting sporadic comms dropouts between the L71 and the ML1400. There seemed to be the occasional dropout to the other ML1400 (not shown) as well, but none to the PLC's on produced/consumed, and the ML1400 in the picture was definitely the main culprit. It got worse and worse until I eventually had them disconnect the uplink between the Stratix 5700 and the unmanaged switch in the ML1400 cabinet, and string a patch cable directly between the two unmanaged switches, just to get them out of immediate trouble. That worked, and they've been running like that for about a month.
I finally got an opportunity to go out there and do some proper diagnostics. I re-patched the network as drawn above, and everything worked (as I mentioned, it was an intermittent problem). I connected my laptop to the stratix as shown, and set it to port mirror the port going to the ML1400, and then the port going to the L71. I took a wireshark capture of each port.
The port going to the ML1400 seemed normal. I'm very inexperienced with wireshark, so I'm definitely not able to be certain, but everything looked OK.
The port going to the L71 looked OK until I filtered it for traffic to/from the problem ML1400 only. When I did that, every single entry was followed by a retransmission. Every single one. This whole time, the comms was working just fine, but retransmissions abound!
Here's what I don't get. If we had a cabling problem from the stratix to the micrologix, I'd expect to see that sort of symptom - try to transmit, receive no response, try again. But I'd expect to see that symptom on the port going to the micrologix, not the port going to the L71.
If we had a cabling problem on the port going to the L71, I'd expect to see that sort of symptom - but I'd expect to see it on all devices, or at least, more than one. Filtering for all other devices shows no such thing.
Is there anyone more knowledgeable than I able to shed some light on what I'm seeing?
The L71 is 192.168.96.11, and the ML1400 is 192.168.96.70. Here's the capture from the port going to the ML1400:
Here's the capture from the port going to the L71:
Got a customer with a control system architecture like this:
The L71 uses explicit messaging to read and write to and from the ML1400. It's been in place and working fine for about 4 years. The L71 also talks to one other ML1400 with more or less identical architecture, and about 5-6 CompactLogix/ControlLogix PLC's using produced/consumed tags, again with similar physical infrastructure. The ML1400 doesn't talk to anything else.
About 4 months ago, they started getting sporadic comms dropouts between the L71 and the ML1400. There seemed to be the occasional dropout to the other ML1400 (not shown) as well, but none to the PLC's on produced/consumed, and the ML1400 in the picture was definitely the main culprit. It got worse and worse until I eventually had them disconnect the uplink between the Stratix 5700 and the unmanaged switch in the ML1400 cabinet, and string a patch cable directly between the two unmanaged switches, just to get them out of immediate trouble. That worked, and they've been running like that for about a month.
I finally got an opportunity to go out there and do some proper diagnostics. I re-patched the network as drawn above, and everything worked (as I mentioned, it was an intermittent problem). I connected my laptop to the stratix as shown, and set it to port mirror the port going to the ML1400, and then the port going to the L71. I took a wireshark capture of each port.
The port going to the ML1400 seemed normal. I'm very inexperienced with wireshark, so I'm definitely not able to be certain, but everything looked OK.
The port going to the L71 looked OK until I filtered it for traffic to/from the problem ML1400 only. When I did that, every single entry was followed by a retransmission. Every single one. This whole time, the comms was working just fine, but retransmissions abound!
Here's what I don't get. If we had a cabling problem from the stratix to the micrologix, I'd expect to see that sort of symptom - try to transmit, receive no response, try again. But I'd expect to see that symptom on the port going to the micrologix, not the port going to the L71.
If we had a cabling problem on the port going to the L71, I'd expect to see that sort of symptom - but I'd expect to see it on all devices, or at least, more than one. Filtering for all other devices shows no such thing.
Is there anyone more knowledgeable than I able to shed some light on what I'm seeing?
The L71 is 192.168.96.11, and the ML1400 is 192.168.96.70. Here's the capture from the port going to the ML1400:
Here's the capture from the port going to the L71:
Last edited: