CLogix v15.6 redundancy issue

Hello all-

We have a CLogix redundant system (v15.6) with each of the processor racks using a 1756-CNBR card for comms to two remote I/O racks.

The CNBR card were both rev E's, fw 11.3.7, and all was working fine up until the other day--

I happened to go online to the processor to check some tuning changes that had been made in my absence, and noticed the processor redundancy status indicator was RED, and the system had switched to back-up.

In looking at the racks I found the CNBR card on the Primary rack was X'ed out in RSLinx. I took a new fw 11.3.7 CNBR card up to the rack, set the address dials, and hot-swapped it for the bad one, which had a RED "ok" LED lit, and both A and B channel LED's were OFF.

The new card booted up ok, and I used RSNetworx to update its keeper status.

PROBLEM: The SRM module on that rack is still reading "DISQ" (disqualified), am presently unable to resynchronize it using the SRM tool in RSLinx, and under the SRM status tab it is showing both CNBR cards as "incompatible".

Again, the firmware (11.3.7) is identical in both cards, the replacement being a NIB spare.

The only odd thing I have found is under "signatures", the new CNBR card is showing all 0's, whereas the working card is showing a HEX value.

???

What are the LED’s doing on the new CNBR card? Are both still off? Ideally they should both be solid green. Did updating the secondary keeper do anything? Also, when you go online in RSNetworx, are you able to see all the drops on the controlnet network? I am sorry to tell you but worst case you may need to re-schedule the network. And you know what that means. I have gotten to work in the past by just cycling power to the secondary. This shouldn’t affect your primary chassis.

the saga continues

Both A and B LED's were solid GREEN, and the OK LED was flashing GREEN.

I ended up rebooting the (secondary) rack and the signature error cleared itself, but then the processor faulted out. Major fault, "loss of program".

I went on-line to the processor and cleared the fault. The status of the processor then changed to PROG mode, and the SRM's apparently engaged at that point and did their job and reloaded the program. The redundancy light in RSLogix then turned GREEN.

All seems to be ok now.

POST SAGA NOTE--In checking out the bad CNBR card I found out that RA released a Product Advisory covering most of the Controllogix line from 10/06-3/07, and the bad CNBR, as well as six other modules in the racks, all need to be replaced. Bad lot of Amtel IC's--

Appreciate the follow-up--thanks.

Thanks for letting me know. Its crazy how the processor lost the program! Did you have to re-schedule the network? Usually if the light is flashing green then it’s something with the media or the network scheduling. Sometimes I’ve had major issues with crappy crimping of the coax connectors as well. They seem to go in and out.
On my last project I used the Enhanced Redundancy Firmware 16. It was solid for the most part. Good thing you found out about the product advisory.

No, no rescheduling. There were also some tech notes I found about corrected FW anomalies in the CNBR's--I'm thinking I may go to 11.4 to fix an issue I've seen where our CNET comms go crazy if I remove one of the terminating resistors. It's a bullet-proof system when it runs, and hell to fix when it doesn't. I am told the new redundant LAN system works great.

I have used the redundant EN2T with Stratix8000 switches. It runs pretty solid. You need to ensure the comm settings are set to 100mbps and full duplex and you are good to go. The only issue that occurs is when a switch over occurs. Your HMI will loose comms for almost 2 mins, depending on how big the system is. It`s a bit freaky when it happens.

I have a V16 redundanct Cllgx system - It sometimes takes a long time for the 'backup' controller to update itself if you have to install a new one - I guess the messaging rate is pretty slow.
...And while the backup is updating, it'll say "DISQ" the whole time, making you think that there is more going on.

NB- this only takes a while when it is sync'ing a new/blank CPU, regular online updates are seamless.

-John

Right, I've seen what you're describing. They are a bit slow to re-synch--maybe 1-2min. I did have to cycle power though in this case for the keeper signature file to update.

Yesterday a coworker and I ended up replacing the CNBR in the other redundant rack (Rockwell PSA) and went through it all again. This time, after updating keeper info, I also noticed the CNBR flashing "!Cpt" in its string of messages, meaning there was a compatibility error. We waited several minutes but it never cleared, and ultimately had to cycle power.

Other than some concern about this PSA that our RA Vendor never told us about, everything worked out ok. We were fortunate the failure occured on one of our systems that had redundant backup, as many do not.