AB IP conflict Compactlogix l32e

Hola Everyone,

I am facing a weird situation here. Please help me out. I will try to put it as detailed as possible.

I have a 1769-L32E CompactLogix processor (IP : 10.65.18.29) connected to just one AB HMI (IP: 10.65.18.30) over Ethernet via a switch. This small network is isolated to the rest of plant Network. The client wanted to include this small network as well.
When they connected the switches with E-net cable, the whole plant system went down. Quickly they removed the cable and isolated it again to revive the plant back.

Guessing it as an IP conflict, I pinged both the plc's and hmi's ip in the plant/SCADA network, to find whether the address already existed. After I could not get reply from PLC's IP, I later found the HMI's IP to be the cause, I changed it to [10.65.18.74].

To avoid the risk of line going down again we did this on shutdown and connected both the networks without any trouble. Everything was perfect until the shutdown was over. The controller in question again faulted and they had to remove it again from the network for everything to be normal.

Suspecting the PLC now. I did the following:

1) Connected my PC to the SCADA network and pinged my PLC's IP (10.65.18.29) which is not connected to this network for which I was getting reply surprisingly as I was not able to ping it before the shutdown. So I decided to browse through the Rs-Linx and am getting a weird status. (SCREENSHOT 1)

2)I connected directly to this PLC and it pings as well as browses on the linx (SCREENSHOT 2)

3) A bit confused, again connected to the SCADA network from a different access point, I was able to ping the ip address again but without it browsing in the linx.(SCREENSHOT 3)

So I am thinking, if it pings then there must be a device with that add on the n/w but why would it not show on the linx.

I don't know what I am missing here, but I am sure it is a silly one.

Isolated !! It seems to be that you have a big plc network

I'd guess the IP is already being used by something else.

Try using something like Angry IP Scanner or ColaSoft Mac Scanner Free to find the MAC address of the device on the plant network. If it doesn't match the PLC's MAC address, then you've found your problem...

If you are using the EthernetIP driver in rslinx it can sometimes 'remember' a device at a specific address if you had a connection to it in the past but show the x's to indicate it is not properly communication with that device which was there in the past. The next time you start the EthernetIP driver with and the device is missing it will not show up in the list.

Again, if you are using the ethernetIP driver linx won't show you a device at 10.65.18.29 if it's not using the IP protocol so if it's another workstation, laptop, datalogger, router being used as a switch hidden in a panel. Jordan's suggestion of an more advanced port scanner will help you find a MAC address and i've had some success with this link trying to identify the the device associated with a specific MAC address could be.

https://www.wireshark.org/tools/oui-lookup.html

You might also check to make sure that you aren’t creating a network loop. If two (or more) switches are connected together and aren’t able setup a spanning tree then they create a loop that can wreak havoc and create very strange problems (and bring the network down).

Firejo said:

You might also check to make sure that you aren’t creating a network loop. If two (or more) switches are connected together and aren’t able setup a spanning tree then they create a loop that can wreak havoc and create very strange problems (and bring the network down).

Click to expand...

Scanning the MAC addresses should tell him that as well.
If you think it's not connected to the network but you can still find it by MAC...

JordanCClark said:

I'd guess the IP is already being used by something else.

Try using something like Angry IP Scanner or ColaSoft Mac Scanner Free to find the MAC address of the device on the plant network. If it doesn't match the PLC's MAC address, then you've found your problem...

Click to expand...

Thanks for the reply,

I was reckoning the same thing about another device on the network with same IP but was not able to identify it. Completely drew a blank about the MAC address thing. Thanks for the link to the other scanners. This should work.

Also, If it turns out to be what we are expecting, would you have me change the IP of the PLC or the absconding device ??

Thanks again

I may have an answer for you as we recently experienced the same issue.
Suddenly a few weeks our ControlLogix devices started scrolling IP address conflicts. In every case, they were devices on one particular side of a new Cisco 3650 switch. The MAC they were displaying was matched up to the fiber trunk port on the switch. On the other side of the switch, Panelviews were throwing up IP address conflicts followed by a hexadecimal number. It was very puzzling. While working on it with our network engineer I noticed that the first part of the hex numbers converted into the first octet of our IP address. I translated them out and saw it was the IP of one of our ControlLogix devices. We finally figured it out after our current network engineer, one of our former network engineers, and one of our network engineers at our company HQ were working on it. New Cisco hardware has a feature that is enabled by default called "IP Device Tracking" or "IPDT" it helps maintain the ARP table (he list the switch keeps of what mac goes to what IP) by sending out ARP requests that look like "Who has xx:xx:xx:xx:xx, tell 0.0.0.0" . That in itself is not an irregular thing. Many devices will ask in that way, but not with the frequency that the switch does it. That will confuse the hell out of the newer PLCs and cause them to report an address conflict, not with themselves but the fact they can see two different devices acting like they are on the same IP. It probably does that for safety purposes.

This issue was isolated to one particular VLAN on our network, which is the one that the PLCs use. So the question to ask is have your network guys installed any new hardware recently. If so, have them research IP Device tracking. Unfortunately you can't turn it off as of yet, BUT you can dial down the request frequency to a point that the PLCs ignore it.

Also you may notice if you have any Windows 7 computers on that network, they may start popping up the duplicate IP errors in the event logs.

There is a very brief mention of it on page 9 of this document: http://literature.rockwellautomation.com/idc/groups/literature/documents/rn/1756-rn591_-en-p.pdf

http://networkengineering.stackexch...-mac-reported-with-same-ip-as-other-equipment

http://www.cisco.com/c/en/us/suppor...ion-protocol-arp/118630-technote-ipdt-00.html

http://community.spiceworks.com/topic/623283-cisco-ipdt-and-automation-devices

Stonent.
That one got me to. I had no idea IT was even working on my switch over holidays. One big firmware install and First thing the next day IP Conflicts on L43 coming from one confused VLAN.
It was a learning experience.

I have 4 switches in my own home network, its not the number of switches that will do it, its the physical wiring. You're right in what your calling it, but 50 switches won't do it. You need a physical wire loop. I worked at a local tech college in their IT dept on the pc side, not the network side. However I knew more on networks than most of the people on the network side short of the department head.. anyhow one of the others in that dept installed a new switch to add more physical clients to the network and crashed it. She couldn't figure it out, she came to my boss who sent her to me. I went down there with her and in about a minute of looking at what she did fixed the problem.. two wires from one switch to another on non-managed switches crashed it. A managed switch can have this and be fine if the second is a fail-over connection, the switch won't turn on the second link unless the first dies.

IP Conflicts can be a PITA to track down. I dislike a truly fixed IP setting on a device. I would always prefer DHCP assigned reserved/static addresses. The DHCP server has a block of IPs to hand out, usually it gives out the next one available in its list. But a reserved is held for a specific MAC address, it will always be given that same address but no conflict can occur as long as no device actually gets a static IP set in its hardware.