SLC 5/04 DH-485 communication issues

One of my SLC 5/03 must be reset manually several times per day. It communicates ("MSG" instruction) with two similar CPUs during every scan. I completely rewired the network cabling because I thought noise may have been an issue and also replaced a P2 power supply that was noisy. The CPU ran OK for a day, but the issue is back again. I've attached a screenshot of the error.

Does anyone have any suggestions? It's driving me crazy. I've even tried changing CPUs - no difference.

I'm switching over to SLC5/05 processors very soon, but I just can't cope with this issue until then.

FYI, I have tried assigning sequential numbers to all three nodes. Didn't seem to help. I have a 1747-KE card in one node that has "30" assigned as its node number. I'd change it if I figured it would help.

The only other nodes are a SCADA - node 4 (used to be 31) and an HMI - node 5 (used to be 15).

I don't deal with message instructions too often, but I'm not sure that you want them triggered every scan. Does it really need to?

No, I was mistaken. Communication takes place once every 10 seconds, each time T4:19 is <DN>. Sorry for the confusion.

Can you post the RSS file (zip it first)?

The details of the logic that triggers the MSG and how errors are handled can be critical to reliability.

Paul

Here it is. The node address on this version may be "11", but none of the logic is different than what's currently loaded.

Look in rungs 614 & 622.

Okay. The way your message control block bits are being manipulated, the MSG instruction will be re-triggered during the whole ten seconds it is active.

The timer2 T4:56 & T4:57 regulate how often the messages occurs (1.5 seconds). Also, there is no interlocking between the two messages you have both going to the same node. So, you are most likely overflowing the MSG buffer hence causing the timeouts.

If I were you, I would change the logic so that you trigger each message once every five seconds...That way, neither of them is triggered at the exact same time. This will leave your DH485 buffers more free to deal with other things (like having a PC with RSLogix500 connected).

On rung 616, the DN bit or the ER bit will unlatch the EN bit. That will cause the next scan of the MSG instruction to set the EN bit again.

Same thing for the other MSG. On rung 624, when the ten second timer ends, the MSG is DoNe or ERrored, then you clear the DN, ER, EN, and the 1.5 seconds timer trigger bit which causes another message to occur.

I will edit your file and post a revision you can try...

Also, having all the nodes sequential would help, but since you are using Node 0 and Node 31, then it is pointless. 6 nodes on one DH485 network is about as many as you are likely to get away with. Changing all of them to sequential numbers would probably help out though...

Attached is an edited version...take a look at rungs 610 through 617...

If you use this, be very careful if you decide to copy and paste these rungs as an online edit. Delete the old message instruction logic first, then paste in these new rungs.

WARNING, I take no responsibility for any unseen consequences...I have no idea how critical these messages are to your process!

Good Luck, and let us know if this helps.

Thanks for taking the time to help. I downloaded last night and everything seemed to be working OK, but I just dialed into the modem a few minutes ago and the processor isn't showing up on the network again. I also lost comms to a 1747-KE card in another processor.

Are you aware of any firmware versions that have bugs with the MSG function?

I'd be happy to send you the .rss for the other processors, if you like.

Hmmm. Well, sure, zip up all the files and post them. I have no experience with the KE card though. Hopefully one of the others here (who are true experts) will have something to offer also.

Paul

EDIT: A drawing of your communications network might be very helpful too.

EDIT: Here is a technote from ab.com lit library with throughput calculation examples.
http://literature.rockwellautomation.com/idc/groups/literature/documents/ap/1747-ap001_-en-p.pdf

Here they are.

I don't have a diagram on my laptop, but I can explain it:

VALVNORT and NORTCIP are within 3' of each other. Each are connected to their own RS-485 link coupler

MIX_HTST is about 12' away. It has its own link coupler, too.

All CPUs are connected to link couplers via RJ-45 x AMP cables.

All three link couplers are interconnected with Belden cable.

I have a UniOP ECT16 HMI connected to MIX_HTST through DF-1 port. A UniOP ETOP11 HMI is connected to DH-485 through one of the ports on a link coupler.

A PC with Iconics SCADA is connected to another link coupler.

I should mention that VALVNORT has 4 slave racks with 1747-ASB cards.

Okay, now looking at the active node tables, you have six gaps in your network node number assignments. 5 gaps if you moved VALVNORT to node 2.

What do you have at (???) nodes?:
01 - NORTCIP
03 - MIXHTST
11 - VALVNORT (Moved to node 2?)
13 - ???
16 - ???
30 - 1747 - KE
31 - ???

One (or more) of those might be your programming PC(s) if you are connecting them directly to the DH485 network.

With each gap, you add 30ms to the token passing cycle, so it would definitely help to get them all sequential, so there is only one gap.

I think you can improve things by moving VALVNORT to node 2, Move whatever devices are at 13 and 16 to nodes 4 and 5, and move the KE to node 6. You'll also need to edit the MSG instructions in MIXHST and NORTCIP to node 2 if you do that.

If possible, I would highly recommed that you add a Diagnostic file for all of the PLC communication channels. Then you can get a LOT more informatino about what is going on with the network. The problem is that it appears two of your 5/03s are too old to support that feature. But, you could add those files for MIXHTST. To do that (offline) go to Channel Config, General tab and place a value in the Diagnostic File box for each channel. Use a number that is not already in use in your data tables, and RSLogix will create the data table for you. Then download it, and you'll be able to monitor the network at least from the perspective of that node.

Paul

I have not reviewed the existing or suggested MSG instruction logic, but based on the Error 37 "waiting for queue space" error code described in a separate thread, I wanted to point out RA Knowledgebase document #16446 that I refer to to refresh my memory on SLC messaging, along with Chapter 12 of the SLC Instruction Set Reference.

There are four transmit buffers that will accept a message and the corresponding data if it's a transmit message.

After all four transmit buffers are full, a message enters a "waiting queue" where the message waits, but doesn't have a copy of the data payload yet. This queue is 10 messages long.

If you trigger yet another message when four are in the buffer and ten are in the queue, you get the Error 37 error code.

Every time I have seen this error, it's been because the program logic triggered MSG instructions every scan or on a periodic basis without considering if the previous messages had been completed.

Thanks for you help, but I still have the problem. The MSG instruction runs one full cycle, then the MSG Transmit bit (ST) goes high and stays high. I added some logic to detect if N:10/14 (ER) is high. If it is, it unlatches N:10/8 Ignore if timeout (TO) bit, which clears the error. Testing it now - seems to be working OK

The new strategy didn't help. All it did was mask the communication issue by eliminating the possibility for the "Waiting for queue space" error.

DH-485 network is messed up. If I connect to a link coupler on one end of the chain, I can see all three PLCs, but from the other two link couplers I can only see PLCs intermittently. I tried moving the link couplers, but the same problem occurs. It's the strangest thing I've ever seen. Spent over 12 hours troubleshooting it yesterday. Changed power supplies, CPUs, etc. - no change. I'd almost say a bad card is causing the network issue.

What's really strange is that the system worked fine until recently, then it started causing intermittent problems.

I'm converting to SLC 5/05 tomorrow or Thursday.

Thanks, again, for everyones' help.