Hi all,
I have recently upgraded a PLC 5 to a 1769-L33ER and 1769-AENTR. It is two panels, back to back, with the L33ER in one, and the AENTR in the other, communicating on a larger PLC network. None of the existing panel wiring was changed or swapped out, strictly the PLC hardware.
Since installation, we have had a constant, intermittent Type 3 (I/O fault), Code 16 (I/O module connection failed on required module). And this fault code references module AENTR:7. On the AENTR rack, it goes AENTR processor, IA16, OW16, IA16, PA4, OW16, IA16, OW16, IA16, OW16, end cap. The PLC is for a refrigeration system, and the AENTR is controlling the solenoids (directly from the relay outputs, no interposing relays), as well as contactor coils for fans. The fault occurs randomly, but is consistently the exact same AENTR:7 module fault. It could happen 6 times in a 12 hour period, or it could happen once, or not at all. It doesn't appear at the same time, and there are no other visible patterns we can find for when it occurs. The only way to reset and clear this fault is to power cycle the PLC's and then clear the fault and put back into run mode, which is a pain for the operators as the panel is across the plant from their control room. I've chatted with my local Rockwell rep numerous times, as well as calling Rockwell tech support, and nothing we've tried has stopped it. I will try to lay out all of the things we have tried, and I am hoping someone has a solution they can think of or found!
- We thought at first there was a communication issue between the main processor and the AENTR, we've tried connecting them directly, to different switches, changed settings on the switches, all to no success or effect. As far as I can tell, it is not a network issue causing the fault.
- We then thought we had a bad card/bad connection. I swapped all of the IA16 and OW16 cards around randomly and made sure all connection tabs were secured correctly. We swapped end caps from the main rack and this rack, and swapped PA4 cards from the main rack to this rack. All of them never stopped the issue, and the error code module never changed, still AENTR:7.
- We looked into maybe the RPI being too fast, so we maxed it at 750ms, which can only be changed on the cards, not the AENTER module itself. We thought maybe the problem was happening quick enough that we could basically ignore it for 750ms, and it would avoid the fault. No such luck. We tried turning off the Major Fault on Connection loss as well, but all it did was still fault (connection loss), and we just wouldn't know it was faulted instead.
- The 120VAC supply for the PA4 on the AENTR rack actually came from the other panel, so we thought there could be noise or EMF of on that line, so we installed a 120VAC filter right next to the AENTR rack. After this change, we did not see the error for a whole week, the longest it had gone before that was 2 days. We thought we solved it, but it popped back up again, and is happening every day again. There is a 480-120VAC transformer in the AENTR panel, so we tried taking the 120VAC off that transformer to the filter instead of from the other panel, and still did not stop the error.
- Rockwell recommended that for inductive loads, such as solenoids, on a relay output card, should be surge suppressed. They had no suppression from the PLC5, and I've rarely encountered needing surge suppression on AC stuff, but we decided to go down this road. We chose RC suppressors as the best method to try to quiet any noise from the turning on or off of a solenoid. These aren't exactly cheap, and we have 48 solenoids total, so we decided to order one cards worth, and try it on the card we think is the issue. We believe since the code is always on module 7, that OW16 in slot 6 must have the noise on it, which is then dropping the connection to card 7. So we installed surge suppressors on all 12 solenoids on slot 6. Again to no avail.
Where I'm currently at is swapping all the cards around one more time, seeing if maybe the fault moves to another module and we just didn't catch it moving (which I'm 99% positive we didn't see it move the first time), as well as putting in interposing relays for all of the output cards to ensure that no external noise can get on the backplane. I'm running into desperation mode, and running out of straws to grasp at. Hoping someone here has encountered this issue and can put me on the right track! Sorry for the long read, but thank you!
I have recently upgraded a PLC 5 to a 1769-L33ER and 1769-AENTR. It is two panels, back to back, with the L33ER in one, and the AENTR in the other, communicating on a larger PLC network. None of the existing panel wiring was changed or swapped out, strictly the PLC hardware.
Since installation, we have had a constant, intermittent Type 3 (I/O fault), Code 16 (I/O module connection failed on required module). And this fault code references module AENTR:7. On the AENTR rack, it goes AENTR processor, IA16, OW16, IA16, PA4, OW16, IA16, OW16, IA16, OW16, end cap. The PLC is for a refrigeration system, and the AENTR is controlling the solenoids (directly from the relay outputs, no interposing relays), as well as contactor coils for fans. The fault occurs randomly, but is consistently the exact same AENTR:7 module fault. It could happen 6 times in a 12 hour period, or it could happen once, or not at all. It doesn't appear at the same time, and there are no other visible patterns we can find for when it occurs. The only way to reset and clear this fault is to power cycle the PLC's and then clear the fault and put back into run mode, which is a pain for the operators as the panel is across the plant from their control room. I've chatted with my local Rockwell rep numerous times, as well as calling Rockwell tech support, and nothing we've tried has stopped it. I will try to lay out all of the things we have tried, and I am hoping someone has a solution they can think of or found!
- We thought at first there was a communication issue between the main processor and the AENTR, we've tried connecting them directly, to different switches, changed settings on the switches, all to no success or effect. As far as I can tell, it is not a network issue causing the fault.
- We then thought we had a bad card/bad connection. I swapped all of the IA16 and OW16 cards around randomly and made sure all connection tabs were secured correctly. We swapped end caps from the main rack and this rack, and swapped PA4 cards from the main rack to this rack. All of them never stopped the issue, and the error code module never changed, still AENTR:7.
- We looked into maybe the RPI being too fast, so we maxed it at 750ms, which can only be changed on the cards, not the AENTER module itself. We thought maybe the problem was happening quick enough that we could basically ignore it for 750ms, and it would avoid the fault. No such luck. We tried turning off the Major Fault on Connection loss as well, but all it did was still fault (connection loss), and we just wouldn't know it was faulted instead.
- The 120VAC supply for the PA4 on the AENTR rack actually came from the other panel, so we thought there could be noise or EMF of on that line, so we installed a 120VAC filter right next to the AENTR rack. After this change, we did not see the error for a whole week, the longest it had gone before that was 2 days. We thought we solved it, but it popped back up again, and is happening every day again. There is a 480-120VAC transformer in the AENTR panel, so we tried taking the 120VAC off that transformer to the filter instead of from the other panel, and still did not stop the error.
- Rockwell recommended that for inductive loads, such as solenoids, on a relay output card, should be surge suppressed. They had no suppression from the PLC5, and I've rarely encountered needing surge suppression on AC stuff, but we decided to go down this road. We chose RC suppressors as the best method to try to quiet any noise from the turning on or off of a solenoid. These aren't exactly cheap, and we have 48 solenoids total, so we decided to order one cards worth, and try it on the card we think is the issue. We believe since the code is always on module 7, that OW16 in slot 6 must have the noise on it, which is then dropping the connection to card 7. So we installed surge suppressors on all 12 solenoids on slot 6. Again to no avail.
Where I'm currently at is swapping all the cards around one more time, seeing if maybe the fault moves to another module and we just didn't catch it moving (which I'm 99% positive we didn't see it move the first time), as well as putting in interposing relays for all of the output cards to ensure that no external noise can get on the backplane. I'm running into desperation mode, and running out of straws to grasp at. Hoping someone here has encountered this issue and can put me on the right track! Sorry for the long read, but thank you!
