Datalog ControlLogix PLC to CSV file

thingstodo

Member
Join Date
Feb 2012
Location
Saskatoon
Posts
533
Overview

I have 3 registers, 32 bits each, to log to a CSV file. And I'd like 'THE PROGRAM' to read 4 registers and write the data (first 3) to CSV if the last two registers do not match. If the data is logged, I'd like to take register 3 and write it to the PLC at register 4 so that the PLC can index to the next set of data. I would like to create a new CSV file for each day, or if the program is stopped during the day, to start a new one each time it starts within the day.

I'd like to have 12 copies of this program running, since I have 12 PLCs to log data from.

I have a proof-of-concept 'sort of' working in excel with RSLinx gateway and DDE. I can't take datalogging with Excel seriously as a permanent solution. It's got to be reliable if it is to be useful. I can run one virtual machine (VM) for each instance if required.

Details

I have 12 Controllogix PLCs networked with ethernet across several buildings. The date and time are synchronized across the PLCs to plus or minus about 10 ms. I have data logged, complete with time and date stamps, in each PLC. Each of the PLCs has a circular buffer of 5000 status or alarm changes. The data is cycled through a single set of 4 contiguous 32 bit (double integers) as described above. Register 3 is the timestamp, register 4 is a read-back. In each PLC, when register 4 matches register 3 (readback), new data is moved to the 3 data registers if it there are more data changes in the queue. If not, nothing happens until a data change is detected.

Every solution that I have found to purchase so far timestamps the data when it arrives at the PC. This is not useful unless I can export that data to a CSV and process it to insert the REAL date and timestamp. The order that things happen is important to me, not just that they occurred.

Most of the DIY solutions involve Excel and DDE. I have had trouble keeping DDE and Excel running together, and figuring out when it stops working, for many years (Our lab guys love to use Labview and Excel. It works great in a lab, when supervised)

I was able to do this task with a distributed system of 186 Modicon A120 PLCs gathering data, reporting it via dial-up modem, to a custom modbus driver running Factorylink HMI software on a VAXstation (mainframe) in 1991. This task HAS TO BE POSSIBLE!

Available to help accomplish this task:
- RSLinx Gateway - allows communication to all of the PLCs if OPC or DDE are used
- Matrikon Datamanager (which does OPC server and client tasks)
- a VMWare ESX server - I can run several Virtual Machines, one per PLC if required. It has sufficient storage, memory, and horsepower
- a lightly loaded 100 MB ethernet network

All of the Rockwell (PLC vendor), Emerson (DeltaV Vendor), Matrikon (OPC vendor), OSI (Pi database vendor) solutions have issues. Most timestamp the data when it arrives at a PC. Those that don't still POLL the data and timestamp it locally instead of accepting the timestamp as PART of the data. Some have a limit (20 tags) to the number of tags that can be polled at a high data rate (50 ms). A number of them will not export to a simple CSV. They will tie to ODBC ... but putting this data into a SQL server or an Oracle database just so that I can get it into a file is a bit of overkill. The tie via ODBC preserves the useless timestamp when tied to a PI database.

Any suggestions would be appreciated. The perfect solution would allow me to try it before I buy it. I have no dedicated budget for this. If there is a supported and documented solution for less than $10K, I'm pretty sure that I can sell it to management. The data that I collect has proven very valuable for troubleshooting but right now I am the only one that can reasonably dig through logs in different PLC memory and correlate it.


If anyone is still reading

Each PLC is logging EVERYTHING that happens. Inputs and outputs going on or off, start and stop commands from the HMI, analog alarms, etc. Across the 12 PLCs there are 700 motors and 150 gates/valves. This includes the electrical substations and power monitoring.

When something strange happens (which seems to happen between once a month and twice a day) I need enough information to figure out what happened. Then I can figure out if it can be fixed, what it will cost. And then someone can decide if it is worth the money. It would be nice if more people had access to this information ... and if it took me less time to gather it and process it.

I would like to take the files from the PLCs in question (if I can remove any), append them together, sort them by date and time, and the select point numbers (every status and every alarm from every PLC has a unique number) to monitor. Use grep to pull these to a file. Use a utility (that I'll have to write later) to put the CSV data into the correct format for import to the DeltaV/PI historian and import it to the public debug terminal. Trend everything on as many charts as required, then figure out what happened and if more data is required.

The only solution that someone has come up with (it was online a while ago, not this site though?) was essentially a dedicated ethernet data logging network with a dedicated Rockwell Factoryview system. It sounded like his solution works ... but at a much higher price. This solution was estimated by one of our vendors at $130K.
 
I am currently using IBA PDA to log all of our controllogix/VME racks.

You can buy various licenses depending on the amount of signals you require.
 
Contacted IDA support

I am currently using IBA PDA to log all of our controllogix/VME racks.

You can buy various licenses depending on the amount of signals you require.

Thank you for the suggestion.

I scanned through the manuals (not an exhaustive search). It appears that this package polls data and timestamps when the data arrives at the PC, so I don't think that I can use the data with their trending package.

I could not locate any provision to write data to a text file for external processing, but I emailed tech support to check.
 
It seems that you are really stuck on the automatic timestamp at the PC end. Maybe it's this simple...

Let the standard timestamp do it's thing, but still log your timestamp into a separate timedate field. Now you have both timestamps in the database: when it actually happened, and when it got logged. I've done this with FT Transaction Manager into a MSSQL database. FT transaction manager isn't cheap, but your tag count should be 12 x 4 = 48 points, the smallest package should work. I haven't actually done the write back that you have, but I don't see why you can't implement that in transaction manager or as a stored procedure in SQL.
 
Factorytalk Transaction Manager

Thanks for the response.

It seems that you are really stuck on the automatic timestamp at the PC end. Maybe it's this simple...

Sorry - I am a bit single-minded at times. I've been asking vendors for a solution for several years now and I'm frustrated trying to explain what I want to non-technical sales-people. ;) I like simple ... everything that I touch seems to get complicated, but I LIKE simple

Let the standard timestamp do it's thing, but still log your timestamp into a separate timedate field. Now you have both timestamps in the database: when it actually happened, and when it got logged. I've done this with FT Transaction Manager into a MSSQL database. FT transaction manager isn't cheap, but your tag count should be 12 x 4 = 48 points, the smallest package should work. I haven't actually done the write back that you have, but I don't see why you can't implement that in transaction manager or as a stored procedure in SQL.
So use 4 tags from each PLC and log the data as well as the timestamp to SQL. That certainly seems like a step in the right direction. At least it's on a disk somewhere.

From there I can certainly sort by the time/datestamp data instead of the PC timestamp ... then I need to decode the 32 bit integer data into point numbers and values. I have not used SQL for a while but I don't remember strong math operators or string manipulation ... is that still the case? I'm not familiar with stored procedures in SQL. Maybe I have some reading to do.

I'll talk to our Rockwell guys and see what it will cost me to try out Factorytalk Transaction Manager

It would be nice to make some progress on this
 
The IBA time stamps it and saves it to a .dat this then can be exported to a .csv automatically using one of the IBAs tools.

I am currently away but can show you an example sometime tonight if you are interested
 
thingstodo,
if you want to keep it simple and cost effective not to metion robust try looking at ignition by inductive automation. I think there are many here on the fourm that would recomend it over transaction manager if not I will.
 
Contact

thingstodo,
if you want to keep it simple and cost effective not to metion robust try looking at ignition by inductive automation. I think there are many here on the fourm that would recomend it over transaction manager if not I will.

Hi bce123,

I contacted Inductive Controls today. It sounds like the system allows for some powerful scripting via Python.
 
Contact

The IBA time stamps it and saves it to a .dat this then can be exported to a .csv automatically using one of the IBAs tools.

I am currently away but can show you an example sometime tonight if you are interested

Hi nic00,

I received a contact email on the weekend. I replied today (not sure if that was you or someone you talked to) and we'll see what solutions are available.
 
Since you are running on a Logix platform, why not deploying a Rockwell historian, such as FactoryTalk Historian SE ( minumum 9518-HSE250 for 250 data points; 250 through 100K available).
http://www.rockwellautomation.com/rockwellsoftware/data/historian/
I have recently quoted 9518-HSE250 for $4900 and probably a server class PC with enough hard drive capacity will run you up another $5K.
FT Historian SE comes with one free license of FT VantagePoint which is the Rockwell dedicated "front-end" of the Historian database, however, there are available "connectors" for pretty much every database graphical interface applications/platforms out there.
I would let the CPUs do what they do the best (control systems) and log the data within a dedicated data collection system.
 
Rockwell solution

Since you are running on a Logix platform, why not deploying a Rockwell historian, such as FactoryTalk Historian SE ( minumum 9518-HSE250 for 250 data points; 250 through 100K available).
http://www.rockwellautomation.com/rockwellsoftware/data/historian/
I have recently quoted 9518-HSE250 for $4900 and probably a server class PC with enough hard drive capacity will run you up another $5K.
FT Historian SE comes with one free license of FT VantagePoint which is the Rockwell dedicated "front-end" of the Historian database, however, there are available "connectors" for pretty much every database graphical interface applications/platforms out there.
I would let the CPUs do what they do the best (control systems) and log the data within a dedicated data collection system.

I would LOVE to have a Rockwell solution. I've talked to over a dozen Rockwell reps at various levels - perhaps I'm really bad at describing what I want. They don't appear to understand that I want the timestamp (a timestamp that shows up on the historian trend) to reflect when the data actually changes, not when the PC gets around to polling the data.

Ensuring that the order of status changes is preserved using a simple, inefficient polled system does not appear possible.
 
I hear ya...We have been through this also...and the issue was exactly the one described within your first post...
 
You have a several Logix platforms and want to "keep an eye" on the systems...
The Logix CPUs are wonderful and powerful devices, however, they were not designed for data collection/logging implementing...
Of course they could do it very well, but there are dedicated database systems, designed and streamlined exclusively for one purpose, data collection, one of which is RA FT Historian SE...
It uses RSLinx Enterprise as the Logix dedicated data server (RSLinx Classic for legacy CPUs)and the FTLD (Factory Talk Live Data)"connector".
You install the system,RSLinx Enterprise browse to any tag,decide the "polling rate" (we went to some 0.01 seconds in some instances), the "compression percentage" and "Apply" it...the Historian system will record the values of the tag at your defined user rate when the changes exceed the "compression" settings and every record is time stamped.
It really is that simple...
Once the database is active you could access it by various means. Rockwell's dedicated "front-end" is FT VantagePoint
http://www.rockwellautomation.com/rockwellsoftware/performance/vantagepoint/
however, you could use any "querry" based system to retrieve the Historian database data and use it for any intended purposes.
There is an available FT Historian client utility, the PC-Datalink, which is an Excel Add-On dedicated to populate spreadsheets with database data.
As usual, Rockwell Support could be quite "unsupportive"...I really don't know what to tell you...;)
I think this is what you are looking for...the Historian SE "time stamp" relies on its own "clock", independent of the CPU's ones, however you could "make" the processors time stamp an independent tag and log it within the Historian at the same rate as the "master tag" and then trend both on your choice of graphical user interface...We've solved numerous years old issues after interpreting the data collected by the Historian system...
 
IBA solution

Hi nic00,

I received a contact email on the weekend. I replied today (not sure if that was you or someone you talked to) and we'll see what solutions are available.

The IBA solution appears to use some dedicated hardware (an EN2T for each ControlLogix, a specific ethernet card in a server class computer) to poll all the data to be logged very quickly - some systems deployed appear to poll 1000 registers at under 2 ms.

It still seems to be a bit of overkill since the PLC is detecting all of the changes already ... polling data at least twice during each PLC scan.
 
Inductive Controls - ignition

Hi bce123,

I contacted Inductive Controls today. It sounds like the system allows for some powerful scripting via Python.

A free download, about an hour on the phone with tech support over 3 days, and I have a proof of concept solution.

So far:
- no cash up front - free download
- no hardware has had to be upgraded
- excellent technical support via phone and webex

Trending as a timing diagram appears to be quite easy to do. Investigation continues.
 

Similar Topics

Context: PLC= S7-1212C, HMI=KTP1200 Basic. Hi, The operator has reported that, from time-to-time, when he presses the "Generate Report" button...
Replies
5
Views
409
I never really noticed this because I rarely do datalogging in a PV+, but it seems you can only start one datalog. Is this really true? My...
Replies
5
Views
759
Hi guys, Is it possible to automatically generate PDF Report of Trend and Datalog in FactoryTalk View SE v12? If YES, Please, how?
Replies
0
Views
638
Hi Guys, I'm using an ABL33ER with Panelview 600. I have datalogging running on the HMI using the view studio datalog model on startup of the...
Replies
4
Views
1,097
Hi all, I have a customer with an FTView SE system running some datalog files. They can see the trends on the client, but they now want to...
Replies
7
Views
2,601
Back
Top Bottom