mylespetro
Member
Hey everyone,
Got called out tonight on an issue with a ControlLogix L62 processor, curious if anyone has seen anything like this before. I've posted a couple threads about this particular system before, but it's a central data collector (Integrator) for a power plant, acting essentially as a multiplexer to read/write between a bunch of field systems (mix of PLC-5, CompactLogix, and ControlLogix PLCs in the field) and the plant DCS (Emerson Ovation).
They were having issues with some (but not all, this is key) I/O points in their sootblower system, so I came out to take a look at the Integrator program. They had the sootblower system locked out under permit earlier in the day (or maybe yesterday), and part of this procedure was powering down the sootblower system PLC. At first we noticed one of eleven messages going to the sootblower PLC-5 (DH+) was erroring out, with the error message "16#0200 - An unknown error has occurred." Then after a couple of minutes we realized that actually three of the 11 messages were erroring out (messages 1, 5 and 10 in the sequence), however, the rest of the messages were executing without issue. We know this to be the case because the I/O contained in the addresses in those messages was updating in the DCS.
When I investigated further, message 1 had the error "16#00fe - Message timeout," while both messages 5 and 10 had the previous 16#0200 error code. First we tried a power cycle on the sootblower PLC-5, which didn't work. I then tried disabling the messages from executing and then re-enabling them, but this didn't solve the issue either.
What eventually solved it was pulling the Integrator ControlLogix processor and reseating it, and the message sequence started back up again with no issues, and they were able to read all of the I/O in the sootblower again, as well as start the sootblowers as required.
Has anyone ever run into this issue where some (but not all) messages going to the same processor are erroring out? If I recall correctly, they were all PLC-5 Word Range Read MSG instructions.
Just to reiterate the sequence of events as I understand them:
Again, just curious if anyone has run into this before or may have any insight as to why this would happen. Not sure if the PLC-5 power cycling caused it or not.
Thanks in advance!
Got called out tonight on an issue with a ControlLogix L62 processor, curious if anyone has seen anything like this before. I've posted a couple threads about this particular system before, but it's a central data collector (Integrator) for a power plant, acting essentially as a multiplexer to read/write between a bunch of field systems (mix of PLC-5, CompactLogix, and ControlLogix PLCs in the field) and the plant DCS (Emerson Ovation).
They were having issues with some (but not all, this is key) I/O points in their sootblower system, so I came out to take a look at the Integrator program. They had the sootblower system locked out under permit earlier in the day (or maybe yesterday), and part of this procedure was powering down the sootblower system PLC. At first we noticed one of eleven messages going to the sootblower PLC-5 (DH+) was erroring out, with the error message "16#0200 - An unknown error has occurred." Then after a couple of minutes we realized that actually three of the 11 messages were erroring out (messages 1, 5 and 10 in the sequence), however, the rest of the messages were executing without issue. We know this to be the case because the I/O contained in the addresses in those messages was updating in the DCS.
When I investigated further, message 1 had the error "16#00fe - Message timeout," while both messages 5 and 10 had the previous 16#0200 error code. First we tried a power cycle on the sootblower PLC-5, which didn't work. I then tried disabling the messages from executing and then re-enabling them, but this didn't solve the issue either.
What eventually solved it was pulling the Integrator ControlLogix processor and reseating it, and the message sequence started back up again with no issues, and they were able to read all of the I/O in the sootblower again, as well as start the sootblowers as required.
Has anyone ever run into this issue where some (but not all) messages going to the same processor are erroring out? If I recall correctly, they were all PLC-5 Word Range Read MSG instructions.
Just to reiterate the sequence of events as I understand them:
- System was running without issue
- Sootblower system was taken out of service for mechanical maintenance, PLC-5 powered down
- Sootblower system and PLC were powered back up to return to operation
- Of eleven (11) sequential messages from CLX to PLC-5, messages 1, 5 and 10 were erroring out but all others were executing without issue
- Had to pull CLX processor to re-establish message function
Again, just curious if anyone has run into this before or may have any insight as to why this would happen. Not sure if the PLC-5 power cycling caused it or not.
Thanks in advance!