1734-PointIO, RPI??

[lostcontrol]

We are having some issues with a PointIO module (& maybe more) going into the shutdown state momentarily.
This is connected to a 1756-ENBT that is also servicing ~8 VSD's, 4x other point IO & CIP comms to 3x PLC's, not too loaded in my opinion.
Main CPU is a L55.
I have put a GSV trap on the module, & it is only dropping off for < 3/4 sec max.
RPI is set at 71ms for this module, there are 2x IE4C, 3x OB8 & 2x IB8, so only small.
Where should I be looking?
Firmware
RPI
CPU
ENBT
cabling (not convinced on this one tho)
Switches?
Port speeds (ENBT is fixed)



Cheers

 

[Eddie Willers]

The 1756-ENBT built-in diagnostic web pages (and the ones on the 1734-AENT as well) are the first place to look.

The -ENBT will give you an idea of the packets/second and the CPU loading. That will let you put some quantitative measurements on the loading of the module.

Both web pages will also let you check for media error counters that might be increasing.

A fixed speed/duplex port is always a big red flag for me... I've seen a lot of duplex mismatches or autonegotiation failures that cause collisions or errors on the port. Why did you use fixed speed and duplex ?

Are you using managed switches ?

Are you using unicast or multicast connections ? If multicast, have you made provisions to handle the IGMP group snooping and querying functions the switches might have ?

Remember that there will be at least 3 different RPIs for the 1734-AENT module; one for the Adapter itself and the Rack Optimized connections to the discrete modules, and one each for the Analog modules.

 

[lostcontrol]

Sorry for the late reply..

The 1756-ENBT built-in diagnostic web pages (and the ones on the 1734-AENT as well) are the first place to look.

The -ENBT will give you an idea of the packets/second and the CPU loading. That will let you put some quantitative measurements on the loading of the module.

CPU loading is low (< 35%). Packets/second is 888 with 90 rejections (should I be concerned about the rejections)

Both web pages will also let you check for media error counters that might be increasing.

There are a lot of FCS & MAC transmit Errors

A fixed speed/duplex port is always a big red flag for me... I've seen a lot of duplex mismatches or autonegotiation failures that cause collisions or errors on the port. Why did you use fixed speed and duplex ?

Based on internal discussions & some reading. Mainly to prevent the autonegotiation going into an endless battle & not recovering.

Are you using managed switches ?

We are not using Cisco 37xx series in multi-stack config. Previous was 3com stack. Interesting here on the 3com, I could ping across the entire stack, but the ENBT could not see any IO/drives plugged into different stack modules. Therefore they had to be all on the same module. This drove itself to the replacement with the Cisco equipment.

Are you using unicast or multicast connections ? If multicast, have you made provisions to handle the IGMP group snooping and querying functions the switches might have ? I am pretty sure the Cisco gear has it on by default, but will double check.

Multicast as far as I know, think Unicast is firmware/version/model dependant??

Remember that there will be at least 3 different RPIs for the 1734-AENT module; one for the Adapter itself and the Rack Optimized connections to the discrete modules, and one each for the Analog modules.

ahh, wasn't as aware of that as I should of been.
But it did lead me to something.
For some reason (& again I think this was based on internal discussion), each module had a different RPI. There was a reason, but with some GSV traps that I put in place, I was getting a lot of re-connections (> 8/hr).
Except for 1 module that was at 50mS. So I have them all set to 50mS, & for the first 48 hr, there were no re-connections that I captured.
Now, 5 days later, the reconnections are only 1/day, at random times from each other.
We are also going to change the CPU from the L55 to a L61 soon, this has been on the cards for some time just have not had a break in production to do so.

 

[Eddie Willers]

Thanks for the detailed update !

Solve the FCS and Sequence errors first. They're either due to bad cabling, induced noise, or duplex mismatches.

Any money or time you spend on configuration, controllers, or firmware are going to be wasted unless you get a clean Ethernet hardware layer.

When autonegotiation is available on both sides of an Ethernet link, you should enable it and let it function. I've seen a lot of problems where one side was fixed and the other left to Autonegotiate, and I can't tell for sure from your post exactly how the Cisco switch ports are set up.

 

[lostcontrol]

Thanks for the detailed update !

No problem, thought I had better get something back to you!!

Solve the FCS and Sequence errors first. They're either due to bad cabling, induced noise, or duplex mismatches.

Ok, will set the duplex of the ENBT first. I hope it is not cabling or induced noise
Is there anyway I can reset the counters?

Any money or time you spend on configuration, controllers, or firmware are going to be wasted unless you get a clean Ethernet hardware layer.

Agree, I am also thinking about getting a analyser to check the hardware layer.

When autonegotiation is available on both sides of an Ethernet link, you should enable it and let it function. I've seen a lot of problems where one side was fixed and the other left to Autonegotiate, and I can't tell for sure from your post exactly how the Cisco switch ports are set up.

Interesting, but understanding.
Will keep you posted.

Any thoughts on the RPI setup though, should all modules be the same rather than offset??

 

[lostcontrol]

Is there anyway I can reset the counters?

Doh, it can be done via the ENBT GUI in Logix5000.
So have just managed to set the ENBT back to AutoNegotiate & have reset the FCS & MAC error counters.
Will monitor overnight to see what results I get.

 

[JDCROCKETT724]

I think I recall something similiar happening to me with an AENT. It was a while back, but I think it was a problem with another device having the same IP address. Or I might have update the firmware on it. Either way it gives you a couple other things to check/play with. Disconnect the Ethernet cable from the device and try pinging it. If you get a ping back then you know there is a duplicate address out on the network.

 

[lostcontrol]

I think I recall something similiar happening to me with an AENT. It was a while back, but I think it was a problem with another device having the same IP address. Or I might have update the firmware on it. Either way it gives you a couple other things to check/play with. Disconnect the Ethernet cable from the device and try pinging it. If you get a ping back then you know there is a duplicate address out on the network.

I very much doubt that it is a duplicate IP problem, have had the management of this network under control since it was first installed 6+ years ago, but thanks for the suggestion!

Firmware is one thing I would like to explore, but as Eddie point out, it will do nothing into the hardware layer is sorted. As of this morning (~8hours later), the FCS & MAC error counters are still zero, so is looking like the Auto-Negotiate may of been the trick...

Now, here's a bug-bear for me at the moment.
I can access the Web Server of the ENBT, but anything else on the network does not allow me, saying that my security settings are blocking access. I am picking this is due to the java version, & that perhaps the devices I cannot access are older versions that are not compatible.
Any ideas, all of this used to work for me which is frustrating!!

UPDATE:
Found out how to add the specific sites into the Java exception list, so can now view what I was wanting to.
Will disable it altogether on the VM I normally use.