Ghost in the machines

See who can diagnose this one.

ControlLogix talking to a micro850 via ethernet (CIP table read/writes).
Logix is also talking to a compactlogix and FTVserver
Switch is a managed Ntron with IGMP with Query enabled
Cisco firewall/DHCP server also attached to network
All devices have static IP's (DHCP is for unconnected clients)

When firewall is connected messages to the micro850 initially work.
Messages work for about 30 minutes but then fail and do not recover
Error message 16#001f 16#0000_0204 Error processing connection related service

Pinging the Micro850 both before and after a failure lockup produces 1 valid response (1st ping) and the rest fails (timeout); a -t produces infinite timeouts. If a second ping command is made after the first set, it reacts the same way, 1st ping is good, the rest fail.

When firewall is removed pinging is normal and communication messages do not fail.

When firewall is connected all other devices communicate perfectly.

Any input at all would be helpful, I am at wits ends.

I suspect something in communications is triggering the firewall. It is protecting from a perceived attack. Maybe sending strange messages to the 850. Send error message to Cisco and AB. Cisco might have a software/flash update for the firewall, if they know what a micro 850 is.

If it works for a while then stops it seems more of a hardware issue than a software / firewall rule issue.

Can you use a different interface port on the firewall? Can you log into the firewall and do a packet capture? A packet capture will give you a better idea of where the problem is.

Cisco might be overzealous in their DDoS protection, perhaps. Could be a network utilization issue.

As suggested by The Plc Kid, a packet capture (Wireshark is awesome if you haven't used it) can certainly help diagnose this.

What model of Cisco firewall device are you using ?

Are all of the automation devices on the same side of the firewall ?

If traffic between the controllers is not passing *through* the firewall, I don't see how it could affect them.

It's possible that the Cisco device is doing some kind of device detection that is confusing the Micro 850.

I agree that Wireshark is the best tool for this. Mirror the Micro 850's port to an unused port on the switch, connect Wireshark, and see what's going on.

Some monitoring with WireShark would be great but I was primarily making reference to the internal packet capture tool built into the firewall.

This will detect traffic that is only inside the firewal like vlan routing ,ACL , Inboud /Outbound rules, Interface routing , etc.

@ the OP is this a new setup? Has it been working in the past with no issues? was something added,removed or changed?

SOLVED: Ended up being a firewall issue, (ARP issue I believe). The strange thing was that it only affected the micro850, the control logix, compact logix, FTV and a Panelview were unaffected. Thanks for the help folks!