Controlnet redundancy problem.

I’ve mentioned bit of this before,



http://www.plctalk.net/qanda/showthread.php?t=68356



Its a bit long


Last year I inherited a system that has two Controlnets, a lot of work was done to update system but the hardware for the Controlnets was retained. However all the copper media was changed including the taps as damage had been identified in some panels, the taps have a fine solid drawn centre core and this had broken where bending radius had not been maintained(too much use of cable ties).


The system is a pair racks each with a 1756 L73, a enhanced RM card other network cards and two Controlnet CN2R/B cards(these were updated from CNBR/Ds) for Control net there is one network with 23 connections and a second with 14 including the two in the main racks. The rest of the nodes are Flex I/O adaptors and a couple of DNET adaptors. The configuration is as follows, there are 15 remote panels and in each panel there may be 1 to 6 flex adaptors. Within each panel the media is RJ6 and copper taps, between each panel is a redundant fiber ring connecting to Coax/Fiber converters (1786-/RPFM)


OK, the system works well, redundancy also works well and scan times are fast and very reliable.


In the original scheme the ”OLD SCADA” had a screen that showed the Controlnet network and was able to indicate if a there was a fault, but it was error free for year apparently. We have replicated the same screen (in our own way and improved it J) when looking at the original code it is clear that it never worked, it was based on reading the Led status of the Controlnet card using a user –defined data type “Code_83”, however the data type order was incorrect and so the Led status was always overwritten. I fixed this in my code and now it works OK


Here the problem, because it now the LED code works, the customer notices that occasionally part of the network will change from green to flashing green or flashing red and he wants it sorted out. We have monitored the network and can tell that the channel that flashes is always the redundant channel (know that that would probably be the case anyway but if I disconnect a channel the other channel that was reporting an error now stays solid green. I can’t seem to track down the exact cause. Whats interesting is that the fault doesn’t propagate across the fibre only in the local copper unless it is in the main PLC area. The system won’t actually alarm unless redundancy is lost for five minutes and this has never happened.


I have purchased a license for MD for RSnetWorx for Controlnet (it only took Routeco four weeks to acknowledge that it actually existed) but it would seem that it doesn’t cover redundancy, (not on my test setup anyway, anyone have experience with this?)


I’m getting exasperated, all the copper media is new and tested including terminating resistors and couplers between taps, taps are now rail mounted rather than screwed to the backplate to reduce any stress on the couplers. The cable end were made off using the AB kit and the cable is the correct type all tested with the AB media checker. The fiber has been tested point to point, all ends cleaned.


In most of the panels there is no AC power, most of the events happen when there is not actual operations taking place, so no relays opening or closing etc.


Naturally I’ve been to Tech Support, (we do have a contract) they have been helpful but are now suggesting replacing everything to see if it helps (that’s not going to happen on a live building)

If you got this far, well done. Thank you.

So any ideas? How do I automatically scan the adaptors to find why some of them report an error and then identify the cause of those errors? Oh and I’m not allowed to make any software changes in the processors.



Chris

Could it be that the redundant channel of the CN2R's are going "stale"? As in it hasn't been used and is attempting a "ping" of sorts, and you are detecting it inadvertently?

Oakley said:

Could it be that the redundant channel of the CN2R's are going "stale"? As in it hasn't been used and is attempting a "ping" of sorts, and you are detecting it inadvertently?

Click to expand...

I really don't know, what overhead is sent on the redundant channel to see if its OK?

Chris

Could be a wiring issue on the redundant channel. Can you check the BNC connectors to see if their loose? Go online with RSNetWorx and scan using the MD and see if that finds anything.

jkerekes

We have replaced and tested all ends, teminators and couplers, all OK.

If, for example channel A is flashing, I can disconnect B to force data on to A, A then stays solid green untill B is availbe and only then when it becomes the primary channel.

There is somthing less then perfect somewhere.

Within Rsnetworx for controlnet there is a "media configuration" tab, if this is not filled in exacly right could this cause problems, by exactly right I mean exact cable and fibre lenghts?

Chris

ControlNet is time based. The media config. tab just ensures that the time delays in the system are accounted for. For simple coax systems, the default 1000 M. is OK. But if you have repeaters and a longer coax run for example, these have to be entered here.

Jkerekes

That should be fine then, coax and fibre are both less then 1000 and thats what they are set for. Repeaters are accounted for

Chris

Controlnet redundancy problem

Hi,
I am also working on similar kind of system. Instead CNBR i am using CN2R and i have 5 controlnet networks in redundant chasis. I am encountering same problem. Both chan A and B are working fine individually. But any one flash red. Now it was happening in other network also which is now working fine. I noted that there has to be a sequence to power up the chassis for RIO and both PLCs. It worked somehow so what i understood is when you have successfully assigned active keeper in RIO You have to power of primary and secondary plc. then power on primary and after sometime power up seconday. it should work. At least in one network it worked. But i am trying to figure out what is going on exactly in the system. If you find anything let me know.

We have had our share of Controlnet issues. Mostly due to installation problems. We were advised by a Rockwell Field Engineer to buy a controlnet checker and use it to observe noise on the cables or nodes. That might be a good next step. It seemed to help us isolate what nodes or segments of cables that were problematic. Good luck!

ControlNet modules have built-in media statistics counters you can access using RSLinx Classic, RSLogix 5000, or MSG instructions from the controller. Start by monitoring those to see which ControlNet segments have media problems.

If the only tool you're using to diagnose the ControlNet signal is the channel LED, you're not taking the problem seriously.

The "ControlNet Checker" that Mark refers to is a specialty device from Rockwell Automation that can examine the ControlNet signal at a very low level. It's part number 1788-CNCHKR, and all RA field service engineers have one or have access to one.

Morning all

I thought some of you might like an update to this thread. This system is now sorted out. It was the old chestnut of assuming that the original designers had designed the system correctly. (My father once told me that ASSUME is an *** of U and ME, but at the time I knew more than he did so I didn't listen).

I read some tech notes about Packet Skew (16518) and realised that both channels transmit the same data (under the heading “Not in the Manual”) at the same time, I thought that if we had packet skew (our MD module in rsnetworx for controlnet had reported some modules sending data at the same time) that perhaps this was something to do with the actual ring topology. Further reading leads me to tech note 33364, “Redundant Media Not Permitted when using Fiber Optic Ring Repeater” and then 32215 “Allowable configurations when using repeaters in a ring topology on ControlNet”.

So you can have a ring or you can have channel redundancy, but not both. As a test we broke both rings on one system, re optimised and now have a solid system.



I can’t believe I missed this, I can’t believe how may others missed it as well, it’s been in service since 2004.


Chris