compact logix - guardlogix safety fault

[James Mcquade]

Hi all,

we have a 1768 compactlogix 5345s safety controller firmware 20.14.
it is randomly giving a major fault and a minor fault. the minor fault is a math overflow. the major fault is in our safety routine. time out error.
my question is this, can the math overflow fault cause the major safety fault.
i am thinking yes because it should fault out the processor. i would like your opinion on this. the machine runs 24/7 and i never get the chance to track down the issue because it's a critical machine and they won't let me have it long enough to look into the issue. they reset it while i am looking at the logic so they can run.
thanks in advance,
james

 

[ASF]

No, a minor fault does not cause the PLC to halt. A significant percentage of the PLC's I see are continuously generating minor faults - more often than not math overflow is the primary cause - and they run just fine. I saw one 1768 Compact Logix generating 10 minor faults per second, every moment it was in operation. It ran fine.

The math overflow can be discounted - what's the error code and the exact error message of the major fault? If you can take a screenshot of the major fault before you reset it, that will make it a lot easier to diagnose.

 

[JeremyM]

Hi all,

we have a 1768 compactlogix 5345s safety controller firmware 20.14.
it is randomly giving a major fault and a minor fault. the minor fault is a math overflow. the major fault is in our safety routine. time out error.
my question is this, can the math overflow fault cause the major safety fault.
i am thinking yes because it should fault out the processor. i would like your opinion on this. the machine runs 24/7 and i never get the chance to track down the issue because it's a critical machine and they won't let me have it long enough to look into the issue. they reset it while i am looking at the logic so they can run.
thanks in advance,
james

A minor math fault like an overflow could indirectly lead to a watchdog timeout, especially so if it's something like a loop's index never meeting the exit condition.

 

[ASF]

That is true, I didn't consider that. Although if someone is using loops inside a safety routine then my solution would start with a blowtorch.

 

[James Mcquade]

thanks for the replies,
what you are describing is exactly what i am seeing.
we have what looks like a divide by 0 creating the math fault, and a safety routine that is timing out.

james

 

[danno4200]

I have a question. I work in a very large plant and I have one (of many hundreds of control cabinets) that has one 5580 (1756-L83E), two 1756-L73 and seven 1756-EN2T in the 16 slot rack. The one in question here is the 5580 (1756-L83E). It is having thousands of minor faults per second (it appears). I know what the cause of the repetitive (T:4 C:51) fault is. It is due to some data being pushed into a specific index of an array that is a few characters over sized. I will be fixing it, but I'm doing a little RCA here and trying to determine if it may be reasonable to assume those thousands of faults could be responsible for the primary EN2T module (acting as the host connection to a cloud service) randomly losing communications to said cloud service? This is a 24/7 operation and it has happened about 5 times (at least) over the last 5 months causing significant downtime each time (hour, to a few hours). Only recently did I notice all the minor faults, but I traced it back to have started happening about 6 months ago when someone made some logic changes that began overflowing the array index. I know how the logic functions would obviously play a role in this, but I'm just wondering if anyone might agree that these thousands of faults occurring steadily 24/7 could possibly, randomly cause the network issues we're experiencing. Like in combination with allll the other processing this controller is constantly doing. I will be fixing the logic, but someone else wants me to replace the EN2T module tomorrow morning, so I am going to wait a while to fix the logic to determine if it is the module or the faults. Just wondering if anyone might think my theory could be very likely. Also, the network cables and switches and all other devices from inside the panel and out appear to be just fine and IT ran traces on the lines with their fancy little network analyzer meter tool and determined they see no issues (between both the current line/port being used as well as the spare unused emergency line/port right next to it).

 

[joseph_e2]

The minor faults wouldn't be my first suspect, but I wouldn't be at all surprised to find that they're contributing.
Also, don't discount the network cabling. One time, I had a PLC that would reply to ping but I could not connect with Logix 5000 no matter what I tried. Cable "checked out ok" so I spent an hour or 2 (or more?) banging on it until I finally, out of desperation, replaced the patch cable. And it just started working. I put the bad cable back in just to verify and promptly cut the ends off.

 

[JLand]

Check what the safety task watchdog is set to. It may be inappropriately low.

Additionally, 9 times out of 10 safety code is pretty simple (which safety code should be). If the watchdog is set appropriately, look through the safety code. Check if there is anything that could get the code stuck (like a non-terminating loop) or anything that will take a long time to execute (a massive loop with complex instructions being executed many times).

Any time I see such a timeout error, it's one of those two issues.