ControlLogix dropping inputs ??

I am having a strange intermittent problem with a ControlLogix system appearing to drop discrete inputs momentarily when we first go online with the processor. This has happened at least twice that we know of but is not consistent.
System is redundant ControlLogix L62 (1756-RM redundancy modules), firmware version 19.53, with Ethernet I/O via 1756-EN2TR modules operating in a ring.

Here is what we observe....
Several motor run command outputs are held up by a sealed in logic rung. The entier rung is conditioned by an "in remote" digital input that is always on. Below is a simplified schematic.

remote startpulse stop pulse run cmd
----||----------||------------|/|------------()--------
| |
| |
| run cmd |
|---||------|


Twice now, when we go on line, the motor run commands which were on are now off. There is no evidence of a rudundancy switchover. Equipment in Auto mode will go off momentarily then on again.

We have programmed several traps to try and catch the root of the problem and are waiting for it to occur again. I'm just wondering if anyone has experienced anything similar.

oops....looks like the seal-in part of my schematic did not appear correct in the posting. The run command XIC should be around the start pulse. Let me know if you don't understand from the description.

Do any of the inputs come from a remote rack or are they comming from an input card on the local rack?

Are these "dropping out" events correlated only with an RSLogix 5000 instance going online with the controller ?

Expand on your description of "first going online with the controller". What is "First": after a power cycle, after a download, first time that day, first time with that PC, or just "whenever we start an online connection" ?

Are the programming PCs on the same EtherNet/IP ring network as the I/O racks ?

Is the "Remote" tag tied to a real I/O point or to a memory tag ?

The default behavior of the ControlLogix is to leave the Tags associated with Input modules in their last state when a network connection is lost.

The default behavior of an A-B Output module is to turn its outputs OFF when a network connection is lost.

If you were to unplug a remote I/O rack from the network and then plug it back in without changing the physical state of any of the inputs, you would NOT see a change in the input tag values.

Your description of output devices "going off and then going back on again" sounds like the actual loss a physical or network connection to the output module.

One way I would troubleshoot this is to use the embedded Web page for the 1756-EN2R module. There will be a list of the I/O connections that has the uptime for the Connection. If any of the Uptime values are different from the others, that indicates a connection failure and re-establishment.

You can also do this programmatically by periodically reading the EntryStatus from the Module Object of your remote I/O adapters using a GSV instruction.

Some answers to Ken's questions.

As near as we can tell, going on-line is what causes it. This is an intermittent problem and it has happened only twice now. What I mean by "first going on line" is the first time since we were last on-site (usually several weeks or more between visits).

Programming is either done from RSLogix on a server via RDP session or from a laptop by opening the .ACD file from the network. In both cases, everything is confined to the local subnet.

The "Remote" tag is an I/O tag directly mapped (XIC-OTE) to a memory bit.

FYI...this is a redundant CLX system. Programming and HMI is done on one network. All I/O is remote and on a dedicated Ethernet ring using the 1756-EN2TR modules.

I am only guessing that the digital inputs are "going off and back on" because the only other tags involved in holding up the output are memory tags. I'm assuming if a memory tag were to go off it would stay that way....and that isn't what we observe.

I appreciate you taking the time to look at this. I realize it is a rather involved issue. Besides being intermittent, this is an almost impossible problem to troubleshoot on a production system. We will investigate your suggestions about monitoring the EN2TR modules.

My main reason for posting was to find out if anyone else had a similar experience...especially since this firmware version is the first to support Ethernet I/O in a redundancy system.

Thanks,
Warren