AB ethernet network crashing

Hello, I am having Ethernet network crashes on an automation line that have got me puzzled. The system is divided into two parts, each has a PLC panel with the following:

Panel 1
CompactLogix L35E with an ENBT and a SERCOS module on the 1768 bus.
2 MOXA managed switches
18 Ethernet modules mapped in the PLC, and 2 HMIs (FTViewME on a VersaView)

Panel 2
CompactLogix L43
2 MOXA managed switches
12 Ethernet modules mapped in the PLC, and 1 HMI (FTViewME on a VersaView)

Network IP addresses are all 131.9.10.XX, and the two panels are linked via one cable. Each MOXA switch has IGMP snooping enabled, with the last port having priority. The last port on switch 1 connects to the PLC, and the last port on switch 2 connects to switch 1. Ethernet modules range from Fanuc robots, Banner safety controllers, weidmuller IO blocks, Numatics manifolds, CNC lathes, and PLC to PLC comm. All devices have the PLC as their comm target.

Here's the problem: Network is stable up and running. If you connect your laptop to the network and the third octet of the subnet mask is not 255 (ie your subnet is not 255.255.255.0) the entire network will crash and all devices lose comm to the PLC. I poll enet module status via GSV with diagnostic code that runs periodically, and see every device drop out. (Disabling the diagnostic code does not alleviate the problem.)

None of the ENBT processors are pegged in % usage. The online diagnostics of the MOXA show no excessive traffic or comm errors on any ports, and keep the multicast traffic in check. No connection limits are exceeded. I'm totally clueless as to what is wrong. Does anyone have any advice that might help? I'd greatly appreciate it.

probably way off here, but did you try using a different pc on the network to see if the problem is some way releated to the pc u are using.

No you are not way off at all. We can duplicate this behavior with pretty much any laptop we've tried to connect to the system. The reason we discovered what caused the problem was my laptop had 255.255.254.0 as the subnet mask, and one that didn't crash the network had 255.255.255.0.

Also, if you have your IP set to auto assign with a manual setting backup, the auto assign portion will crash the network as well.

Does the network have multiple dhcp servers? Possible that an additional device is getting the plc's ip address?

We saw this behavior while the system was still being debugged on our shop floor, no dhcp servers, and no external connections what so ever. I am not certain if it is connected to the plant network now, but I know the problem was present before.

Why do you use the different mask? Doesn't this invite traffic? Is it possible there's another subnet existing in the 255.255.255.0 mask? If so, your PC would show up on two different subnet with a 255.255.254.0 mask. Right?

Also, 131.9.10.XX is not a recongized private IP address that I know of. Why did they use a public address for control network?

So the network as a whole would really have no idea what your subnet mask is. It doesn't care. The use of the subnet mask only helps determine the routing table of the laptop, deciding what is local (i.e. use arp to connect to a remote host...) or not (i.e. use arp to the gateway, then send the traffic to the gateway as required by the destination). If all the devices are on the same subnet, there should be no reason to use a gateway at all. If the problem occurs due to a broadening of the mask, I wouldn't think it would matter for local traffic.

What's the gateway set to on the laptop? If you are just doing troubleshooting locally, you can leave it blank.
Can you get a packet capture from the PC? Start the sniffer then plug in. Disconnect, then change the subnet mask to cause a fault then plug in again. We can compare the traces and see if anything is obvious.

merc, seeing the issue you have, I assume your from MPG and your talking about a line at FPC?

Sounds like you may be running out of CIP connections. Use this EIP capacity tool to see waht you have vs what the controller supports.

Also make sure only 1 switch is the IGMP Querier and not both set to querior as this will case major problems.

Most switchs handle this automagically for you by setting the switch with the lowest ip addy to the querier but check to be sure as I am not familiar with how moxa does it.

Check for poor connections as they will cause dropped packets and retrys on a poor connection.

You mention ethernet modules of 12 I assume you mean point or flex IO?

12 modules or 12 adapters? If adapters you would have a IP adress for each. Can you clarify?

Can you post a drawing of the system network topology.

I think you are exceeding CIP connections on your L35. It only supports 32, analog cards take 1 connection each, so it is easy to forget those. Go online with the processor and you Exceed it. Check the ethernet Webpage it will show max connections seen. To test inhibit some ethernet modules and see if you online experience improves. If this is the problem you have to get rid of IO connections or upgrade the processor to an L4x series.

Hi guys,
All great thoughts but none are correct. First, there are 3 plc's on the line not one.
I think the overall point is being missed. The line runs fine as long as your laptop subnet mask is 255.255.255.0 or if there is no laptop plugged in.

CIP message limits are fine. One switch is the querier. If ANY of these issues were the case, the line wouldnt run normally. The line ONLY has an issue if the laptops you connect with doesnt have the mentioned subnet mask. If you change your mask to 255.255.252 or ANYTHING other than 255.255.255.0, the line crashes. This defies all logic for Ethernet or Ethernet/IP rules. The IP addy and subnet mask of a computer should not matter period. You should be able to set those two things to anything. No, you might not see devices if your not in the right subnet or if your not masked into the subnet by the mask, but either way, it should NOT cause an ethernet network to crash simply by having a subnet mask other than 255.255.255.0

mmcdermott1 said:

Hi guys,
...
I think the overall point is being missed. The line runs fine as long as your laptop subnet mask is 255.255.255.0 or if there is no laptop plugged in....

Click to expand...

I disagree, the overall point is not being missed at all. The OP has a problem, and we are trying to help resolve it.

You say yourself the IP and subnet mask should not matter, and I think most would agree so of course we are going to point out the "obvious". Especially when we have no further information of the system architecture. Many questions from members have been asked, yet feedback information is lacking. The OP says this behavior was seen during debug on the shop floor. How did it leave the shop floor if it had a known problem?

We don't have a crystal ball into your network and system and what you've already done for trouble shooting.