PLC Output Dropouts during troubleshooting efforts

I'm new to Logix 5000. We have a slave rack of 1756 relay output cards driving our motors (120VAC motor control). I was trying to troubleshoot a completely unrelated problem, to see if a level transmitter had power, and simply inserted our digital voltmeter between two 24 VDC terminals (DVM set to DC voltage), one terminal being DIN rail (ground to cabinet) the other being 24 V, and all the outputs cycled out when I contacted probe to terminals, then back on. This caused ALL our motors to shut down while plant was running! This happened to our electrician a week before on the other side of the cabinet, he did NOT short anything. What might be a plausible explanation for this? I thought then it might be a loose wire but he was 4 feet away & it happened right when he made contact w/terminal, any chance alternately that voltmeter has something wrong with it?

Also, not sure if it's related, but nobody was in the cabinet a couple days ago, and the slave rack just dropped out. That time was different. The slave rack I/O cards, all displayed blinking red OK. The Ethernet card said good comms. I know very little about this system and none of my coworkers are too familiar with it either. I finally figured out I needed to reset the power supply on the slave rack. Then the cards communicated. I called the rack without the CPU the 'slave rack', not sure if that's how Rockwell terms it. Is this problem related? Note when it has the 'DVM trip', the OK lamps didn't blink red, or if they did, they didn't stay blinking. This is a running plant but I need to service the instruments & understand this.

How is your slave/remote I/O powered [The power Supply for the rack]? Is it 24VDC or 120VAC? If it’s 24VDC, what type of power supply?

Follow-up

Thanks much for responding! It is powered with 120 VAC coming off an APC UPS. UPS is at 10% of capacity and battery reads good; batteries were changed about 1.5 years ago. CPU rack is powered off of the same UPS. I don't think CPU rack dropped out at same time but wasn't watching; is there any way to tell in log(s) what happened? We have a 2006 version of RSVIEW and RSLOGIX. I have DeltaV and Honeywell Hybrid Controller experience but not much A/B experience yet. I'd like to address this problem so I can service some instruments in this cabinet but right now, I don't want anybody in there for fear of shutting plant down again.

I'd say loose wire. Loose connections are bugger sometimes, and once you find one loose you think you've got it licked, but then it happens again. Where there's one loose, there's probably more. Anymore, if I suspect a loose wire, or if I happen to stumble upon a loose wire while troubleshooting, I tighten every connection in the panel, and tug each wire after tightening. It can't hurt anything (unless your cabinet is 30+ years old and terminals are brittle) and going through and tightening all connections is actually a PM for some organizations (incl. the military).

strantor said:

I'd say loose wire. Loose connections are bugger sometimes, and once you find one loose you think you've got it licked, but then it happens again. Where there's one loose, there's probably more. Anymore, if I suspect a loose wire, or if I happen to stumble upon a loose wire while troubleshooting, I tighten every connection in the panel, and tug each wire after tightening. It can't hurt anything (unless your cabinet is 30+ years old and terminals are brittle) and going through and tightening all connections is actually a PM for some organizations (incl. the military).

Click to expand...

An excellent idea. I seem to remember reading somewhere that the stranded wire connections on compression screw type terminal strips and devices should be rechecked six months after installation. They do loosen over time - temperature cycling, contactors chunking in and out, etc.

I've done panels where the vast majority of connections were tight but a small percentage would take half a turn to a whole turn. An even smaller number were loose to the point they'd have fallen out if it weren't for the wire stiffness.

Doug-P said:

An excellent idea. I seem to remember reading somewhere that the stranded wire connections on compression screw type terminal strips and devices should be rechecked six months after installation. They do loosen over time - temperature cycling, contactors chunking in and out, etc.

Click to expand...

Wago loves to get on the soap box about this when marketing their "Cage Clamp" terminals; they aren't wrong, screw terminals do get loose over time, whereas the Cage Clamps actually get tighter over time.

A common link being, the panel door was open both times.

check the wiring on the door or doors first would be my starting point

Sounds like you merely lost output power when you checked your voltage with the DVM, but encountered a fault in the other instance. You said that you had to reset the power supply to re-enable the chassis after the incidental fault. Have you checked your power supply grounding?

I will check resistance to ground. Anything else I can check on 1756 power supply? There is no wiring to the door except for 3 Rice Lake load cell transmitters/indicating displays. I thought loose wire at first but problem seems to coincide exactly with when DVM probes contacted DIN rail screw & live wire, I don't think we bumped anything. This same event happened on left side and right side of cabinet, first time we definitely didn't touch a wire as DVM probes were simply against terminal strip screws. The other failure occurred when cabinet was closed, I was told nothing like that ever happened but that PLC had crashed maybe 2 times over last few years. I have no idea if the chassis reset issue is related, I am new to these cards & controllers. Panel itself was done by PESCO, it's lame. Lots of wire nuts, no cabinet lights, handwritten numbers on terminal strip (nearly worn off), nothing individually fused. I think this entire cabinet has maybe 12 fuses in it. I'm going to upgrade to redundant controllers at some point but want to fix the basic reliability/stability issues first. The computers also tend to freeze when plotting (RSView32). No idea why.

I would agree with Bit Bucket to check the eathing on the PLC power supply, and also check that the PLC chassis has a large sized earth wire going directly to earth.

What version of RSView 32 are you using, I take it that RSView freezes when plotting a Trend
Are you using native RSView trending or TrendX

strantor said:

Wago loves to get on the soap box about this when marketing their "Cage Clamp" terminals; they aren't wrong, screw terminals do get loose over time, whereas the Cage Clamps actually get tighter over time.

Click to expand...

^^^This is actually quite amusing, considering the infamous WagoWiggle issue from years ago.....