PLC Memory Usage

[raamakoti]

Is it possible to trend PLC memory usage in the historian? We are getting few faults on the compactlogix controllers. They are old running V17. I don't have the model number handy.
Comms loss happens and by the time we try to dig down the source everything runs normally again.
I am struggling what the best way to keep track of these sudden now warning loss of comms and miraculous recovery without doing anything.
Thanks

 

[Saffa]

What is communicating to the PLCs? Do they all fail at the same time?

Sounds to me more likely to be a networking issue?

 

[AlfredoQuintero]

Hi there.
In the post on this link there is a discussion on the use of Rockwell vendor specific class 72, user memory object, which can be obtained through service 2 of a generic CIP message. I am not familiar with Historian, but if you can program Historian to send periodically CIP generic service 2 to class 72, I think you can get information on the memory condition of your processors.

 

[raamakoti]

What is communicating to the PLCs? Do they all fail at the same time?
Sounds to me more likely to be a networking issue?

Over the years as the plant grew inter-connectivity between the PLCs through messaging grew as well. The existing system is not scalable going into the future. We have to clean the mess and simplify the structure. I think implementing cheapest and fastest solutions many times got us to this mess.
Last night one PLC communication error brought the entire production down. Took few hours to find the issue and without doing anything it started working again.

 

[Lare]

PLC programs don't use different amount memorys at different times. Sounds like that you have some component fault on PLC's circuit board.

 

[Saffa]

Over the years as the plant grew inter-connectivity between the PLCs through messaging grew as well. The existing system is not scalable going into the future. We have to clean the mess and simplify the structure. I think implementing cheapest and fastest solutions many times got us to this mess.
Last night one PLC communication error brought the entire production down. Took few hours to find the issue and without doing anything it started working again.

Ah I see. So there's a mish mash of message blocks, possibly not always executing every scan? So you end up with some conditional stuff, that works fine until it just happens to execute at the same time as something else?

Or are you using produced / consumed tags?

Definitely sounds like some careful documentation, followed by optimizing down to one or two read / writes if possible, is the way to go.

Maybe once you've documented the existing messaging schema, come back with a drawing and we can help with some suggestions

 

[Lare]

You should count how many simultaneus communications plc is sending same time and compare it to max alloweded amount on com module's manual.

If PLC is sending more messages than module is capable to handle then part of messages is rejected and never sended.

 

[Ken Roach]

In my experience, power loss and network infrastructure problems are a far more likely reason for "the PLC went down and then came back up and nobody knows why" sorts of failures than a memory management function.

It sounds like you have some sort of historian in your system; exactly what is it ?

RSLinx Classic and FactoryTalk Linx have some limited "pre-defined items" that you could log via OPC. @IsPresent tells you the basic connectivity status, and if you're really interested in memory, @FreeMem will tell you that.

Each communication driver also has a Diagnostics set of data that you can monitor, but it's not exposed to OPC or easy to log manually.

The CompactLogix communications daughtercard has Ethernet and CIP communications statistics in its various objects, which you could trend with some effort.

But often the most bang for your buck comes from network management software that's set up to monitor the SNMP objects inside the switches and routers that make up your infrastructure. Sometimes you get it from the vendor (Cisco, HP, etc) sometimes it's IT-industry stuff like SolarWinds or OpenNMS or Cacti.

The task of figuring out organically-expanded networks to find the vendors and capabilities of their switches also often lends itself to clean-up and better understanding of the network.

I wish the CompactLogix had an easily-accessible "Uptime" counter, but it's hidden in a little Java applet. Instead, put a simple one into your CompactLogix to grab the WallClockTime when the controller goes into RUN mode, so you can compare it to the current Wall Clock Time and figure out if the controller has recently been power cycled.

 

[Ken Roach]

If somebody told me "I think that inter-PLC messaging is causing our problems", my general process would be:

1. Get a physical and logical network map from the IT department.

If you discover that there isn't an IT department who ventures beyond the office spaces, then you're going to learn about network discovery tools. I've been tickled with the features that Slitheris provides, and of course you're going to use RSLinx Classic if you have A-B controllers.

2. Create a full program inventory, including uploads from each PLC you can find.

3. Open each PLC's embedded webpage, go to the two Connections tabs, and grab a screenshot.

4. In the same embedded webpage, go to the Ethernet port statistics, grab a screenshot, and reset the error counters.

5. In each RSLogix 5000 program, sort for MESSAGE data types and examine their sources and destinations and how they are triggered in logic.

You will inevitably find some MSG instructions that are self-retriggering as fast as possible. Slow them down with a Timer (and don't let the timer retrigger another message into the queue while they're still /EN) and write logic to capture at least the last .ERR value.

You might also find controllers whose Ethernet port statistics show significant actual physical errors and lost packets. At the very least examine the installation for damaged cables.