Duplicat IP

PLC Pie Guy

Member
P
Join Date
Jun 2013
Location
Halifax
Posts
1,144
Hello there.
I just came across something that I wanted to ask a question about. I have an L43 Processor that has 2 ENBT/A modules connected to it. One is my Industrial Ethernet connection, its address is 10.202.221.56. The other one is connected to six Powerflex 40 drives and is IP 100.100.100.1. Our plant is down for maintenance and has been for 6 days now. Everything was fine when we shut down. This morning I opened the cabinet to have a look at the inside and I noticed that both of the ENBT modules we flashing a message stating that there is a duplicate IP address and sure enough looking at LINX I can see my processor with a big red X on it. Also both cards had the Link led flashing green but both of the other led on each card were flashing red. The first thing I did was one by one I disconnected the Ethernet cables from the ENBT modules then re-connected them. Instantly the problem went away and all is healthy again.
I am the only person in this plant that touches the PLC's or networks so I know nothing else got plugged in. Also the fact that both ENBT modules were in this state makes me wonder if the problem didn't originate in one of the modules itself.

Any thought on this?

Thanks
 
You might think you are the only person who touches and plugs stuff in - but you can't be certain on that.

Can you see the MAC address of the modules? Take note of that.

On a laptop that's plugged into the same network do the following

ping ip.address.of.module

then

arp -a

then look for IP address and make sure it is resolving to the proper MAC.

then unplug the module from the network (keep your laptop/computer plugged in)

then arp -d

then ping ip.address.of.module - see if it resolved. if it DOES then you do have a duplicate

arp -a

look for MAC corresponding to that ip address

post MAC here, or look up first 6 characters online to tip you off to hardware identifier.

good luck!
 
Are both 1768-ENBT modules connected to the same switch, or to the same type of switch ?

I encountered a problem a few years ago where I would get Duplicate IP errors when the whole system was power cycled because of some issue with a Hirschmann RS20 switch. The 1756-ENBT modules would send their Duplicate IP detection packets at just the right time in the RS20's boot sequence that it would echo them back.

The problem didn't happen if you power cycled just the controller, or just the switch.
 
So this happened again.
And once again the resolution was to briefly unplug my Ethernet from both modules and re-connect.
If there was a duplicate IP on my Network side. the 10.202. side, then would that really fault the ENBT on the drive network side? it is on 100.100....

When it happened this time I unplugged both ENBTs and pinged both address with no response.


Any thoughts?
 
One of the guys here had this issue with a 1756-ENBT. He resolved his by bumping to the latest ENBT firmware. I have no explanation as to why this helped in his case but it did. I'm not sure what firmware level the card was at initially.

I understand that you are on the 1769 platform but I would find it hard to believe that Rockwell would follow parallel Ethernet/IP development paths for the two platforms. I suspect they share the Ethernet/IP portion of their firmware.

Keith
 
I ran into it when we did an IOS upgrade on our Core Switches. There was some feature in the 15.x branch enabled by default that was screwing w/ the ENBTs we had.

(Note, these were my test PLCs, hence why they were on the overall network)
 
So this issue is now constant. I can re-set the ENBT by removing the Ethernet connection then replacing it. It will go for a max of 20 mins before it fails. Sometimes it just faults the PLC network ENBT and sometimes it faults both the PLC network and the drives network ENBT.
The other thing I noticed is that a scale module that I have in the project as a generic Ethernet module is no longer seen by the processor even after the re-set. I can ping it and see it in the CMD prompt but not on the project tree. It shows as a yellow triangle. I thought this might be the source of my trouble so I unplugged it but the ENBT faulted anyway.
The PLC Network, the drives network and two scale modules (on the PLC network side) are all connected through a managed switch. Could this be causing me grief.
This issue is a little over my head, I think. I have set this stuff all up before and I can troubleshoot network issues to an extent but I really don't know where to go from here. I have called my Rockwell rep to try and get a new ENBT coming my way but I fear that may take a while.

Any ideas on how to proceed with this?

Thanks
 
I should mention that when the module LEDs go red it is flashing a MAC address. 50:06:04:55:33:B1
Would that be the MAC address that is causing me trouble?

How would I find this particular devices IP?

Thanks
 
Last edited:
If you are running a managed switch, check the ARP tables on it for the IP.

If you can reliably fail the modules in 20 mins, I'd setup wireshark on a machine, plug it into the switch using a port setup as a mirror(or monitor) port and wait for the failure. The packet capture will go a long way to telling you what is misbehaving.
 
PLC Pie Guy said:
I should mention that when the module LEDs go red it is flashing a MAC address. 50:06:04:55:33:B1
Would that be the MAC address that is causing me trouble?

How would I find this particular devices IP?

Thanks
Click to expand...

Linux would be useful.

http://www.cyberciti.biz/faq/linux-duplicate-address-detection-with-arping/

Else, as mentioned, run arp -a from CMD on a windows machine and it will list IP addresses and their associated MACs. Else, look at managed switch tables as suggested by others.
 
I did run ARP -A after pinging the address of the ENBT/A. It only returned the IP and MAC of the ENBT itself and the computer I am using. Only two responses.
 
UPDATE:

So I goggled the offending MAC address. It came back to be a cisco switch.
The only cisco switch here is the big 48 port momma that runs my industrial network and some ports dedicated to office network.
This switch is ruled by IT.
I called them and indeed they see a duplicate IP issue but it is somewhere else on their network. They are working on resolving this now.
Is it common to have your Industrial network ruled by IT? This seems kind of crazy when they are half a world away from our plant. Any time I have worked on networks they tend to be smaller with only the essential equipment connected to it so this seems a little risky to me.

I have not gotten wire shark yet but I think I may look into it now. This is the first network issue I have had that PINGing and using a cable verifying tool couldn't fix.

Thanks for all who offered suggestions.
 
Yes, wireshark, out of the box is a handy tool and easy to figure out for basic monitoring. I felt silly for waiting so long to try it out. It quickly became useful for many things.

The line between IT and automation is blurry. You should have total control of I/O networks at the very least, but for plant network infrastructure, it is often the IT guys who own that hardware.

When they say that a dupicate IP can cause unexpected behavior, that is exactly my experience. I accidentally did that to myself going off documentation for our controls lan, without a preemptive ping...

All seemed okay sort of, then not, power cycling made it better, but then new different symptoms, then the third time, I think I faulted a running PLC (during a down day, no harm done).
 
Last edited:
Well I'm glad to say this is no fault of mine.
I.T. did some firmware updates to the server and all the connected switches. All happen to be Cisco. This is what caused all the mayhem. I did not know anything was done which kind of angers me as I am responsible for maintain the networks in the plant and I am the first person called when anything hits the fan. It really messes with my head when I see issues like this and had no idea that anybody was even in our network. They did this from a room half a world away with no follow up to see any resulting consequences.
They are still doing battle with this and are not getting very far. Now they are logged in to our system through one of my NON I.T. owned laptops as it seems to be the only way they can get in right now. They are concerned about not being able to restore the previous firmware that they know worked. To bad for them as now it close to quitting time for me on Friday and the company said they don't want to offer me time and a half to be here this weekend.
Oh well!
 
Back
Top Bottom