PowerFlex 700 Vector losing link to PLC

[Clay B.]

Ebolbol,

You should have some lights directly below the HIM. These lights are for Comms. When you are unable to control the drives can you look at the states of these indicators. They are from top to bottom Port,Mod, Net A, Net B.

Also whe you get to the drive does the HIM have a Popup saying something like "Port 5 fault"?

Like a said earlier there is a work around until you solve the problem. But info on these lights could give us some insite to your issue.

Also when the drive is non responsive can you take a screen shot of what your sending the drive. The O's along with the I's

 

[ebolbol]

Clay, Thanks for your post & also patience... you can tell that I'm just lost! I don't think it's a Comm Fault. and I don't really think it's a serious issue... something stops the drive. Port 5 fault almost never pops up. In 5 yrs maybe we've had Port 5 fault pop up 2 or 3 times during power outages. The [ONLY] and truly only thing that indicates to us that the drive will not control is STS blinks and the HZ on the HIM has "0" And I have associated this condition with the "Active" & "At Speed" bit both being "0"in RSLogix Tag view as you will see in the first picture. All the rest of the led's you mentioned [NEVER]change behavior and they are always as follows: Thats why I don't think its a comm issue.

Port = Always solid green
Mod = Always solid green
Net A= Always solid green
Net B= Always Blinking green

The second picture demonstrates a drive that will almost certainly receive the command from the plc and control... notice the STS is solid and the HZ on the HIM has 20 and up Top it says "AT Speed" and not "Stopped"

so the two pics up top demonstrate (1st) a drive that will not control (unless the green button on keypad is pushed) (2nd) a drive that [will] control. and both above pic are when there are no engines running. Now below is a screen shot of the live values on an engine that [IS!]running and controlling with everything normal.

Ebolbol,

You should have some lights directly below the HIM. These lights are for Comms. When you are unable to control the drives can you look at the states of these indicators. They are from top to bottom Port,Mod, Net A, Net B.

Also whe you get to the drive does the HIM have a Popup saying something like "Port 5 fault"?

Like a said earlier there is a work around until you solve the problem. But info on these lights could give us some insite to your issue.

Also when the drive is non responsive can you take a screen shot of what your sending the drive. The O's along with the I's

 

[Ken Roach]

Mr. Bolbolian will be leading an advanced clinic this summer in "how to post detailed, clear and useful information about your automation questions".

A number of our newer members are strongly encouraged to attend.

The most interesting thing I see posted is the drive's Status LED flashing while the drive is stopped.

>>> Is it flashing yellow, or flashing green ? <<<<<

If a drive is just stopped, hanging around waiting for a Start command. the Status LED flashes green. Flashing yellow indicates that there is something preventing the drive from being started.

The device responsible for preventing the drive from being started is indicated by Parameter 214, "Start Inhibits". It's an array of bits that indicate different reasons that the drive cannot start.

The most common of these is that the Stop command is being held true on the network adapter. But that would also cause the text to read "Stop Asserted", and the drive stop logic shows that the ControlLogix only asserts the STOP bit for 5 seconds to cause the drive to stop.

Parameter 215 is also very handy, as it indicates the source of the most recent Stop command.

Follow the directions on HIM module navigation in the User Manual (20B-UM002, from literature.rockwellautomation.com) and read those two parameter values.

I have a feeling that the control program is the issue, not a configuration, wiring, or network problem with the drive.

 

[ebolbol]

Those are merely acts of desperation

but I do appreciate the kind words... and I do appreciate the [in-depth] expertise YOU, & the LIKES OF YOU provide just for the sake of helping people. It's priceless to us wanna-be's. Sharing knowledge shows selflessness and we thank you for that ... To answer your question on the STS led color... it blinks Green when it's in that non-cooperative state. I've actually never seen it yellow and didn't even know it could change colors like that. Every time it falls on it's a** like that, all it takes is the push of the green button on the keypad and things come alive and you hear a humming sound indicating that it's actually running. But what get's it to that state is beyond me... I think I share the same feeling with you on the last part of your post, I truly believe it's a logic issue my self... I think our programmers perhaps had limited knowledge on these drives and their configuration, so they gave us a bare-bone basic setup. It's been like this from day one. These things do great if you baby sit them to make sure they're always on. I wonder if I can set up trends on some of those output tags to see what they do in the event of an abrupt shut down? I hope I've answered all your questions and I will check out that manual you mentioned. Thanks again.

Mr. Bolbolian will be leading an advanced clinic this summer in "how to post detailed, clear and useful information about your automation questions".

A number of our newer members are strongly encouraged to attend.

The most interesting thing I see posted is the drive's Status LED flashing while the drive is stopped.

>>> Is it flashing yellow, or flashing green ? <<<<<

If a drive is just stopped, hanging around waiting for a Start command. the Status LED flashes green. Flashing yellow indicates that there is something preventing the drive from being started.

The device responsible for preventing the drive from being started is indicated by Parameter 214, "Start Inhibits". It's an array of bits that indicate different reasons that the drive cannot start.

The most common of these is that the Stop command is being held true on the network adapter. But that would also cause the text to read "Stop Asserted", and the drive stop logic shows that the ControlLogix only asserts the STOP bit for 5 seconds to cause the drive to stop.

Parameter 215 is also very handy, as it indicates the source of the most recent Stop command.

Follow the directions on HIM module navigation in the User Manual (20B-UM002, from literature.rockwellautomation.com) and read those two parameter values.

I have a feeling that the control program is the issue, not a configuration, wiring, or network problem with the drive.

 

[bguinn]

This sounds like a problem of the start being made before the stop has been released. In this case the drive will not start. You stated that this only happens when the drive is stopped unexpectedly, is it possible it is being started back before the 5 second stop that Ken referred to has completed. I often put a 1 second delay on the start after the stop has gone false.

I had a problem like this recently after I changed companies. The new company had a PF 700, they had replaced a 1336 Plus, because the drive kept overvolt faulting when the line would start. I changed the stop to a ramp to hold using the DC injection. The drive quit faulting, but would not start with the rest of the line. I then added a delay to the start after the stop was released and have not had any more problems.

 

[Clay B.]

This sounds like a problem of the start being made before the stop has been released. In this case the drive will not start. You stated that this only happens when the drive is stopped unexpectedly, is it possible it is being started back before the 5 second stop that Ken referred to has completed. I often put a 1 second delay on the start after the stop has gone false.

I had a problem like this recently after I changed companies. The new company had a PF 700, they had replaced a 1336 Plus, because the drive kept overvolt faulting when the line would start. I changed the stop to a ramp to hold using the DC injection. The drive quit faulting, but would not start with the rest of the line. I then added a delay to the start after the stop was released and have not had any more problems.

This is a very likely cause and I have also seen it before. One way to get around this without timer is to hold your Start bit high until you get your AtSpeed Bit from the drive.

If you Stop is high when Start goes high then it will not start Stop has to be off before Start can happen.

 

[ebolbol]

bguinn,
Clay B,

How do I verify that that's what's happening? one of the engines is running right now and there's no telling when they will shutdown... most of the time when they shut the engine down, there is no one here.

This sounds like a problem of the start being made before the stop has been released. In this case the drive will not start. You stated that this only happens when the drive is stopped unexpectedly, is it possible it is being started back before the 5 second stop that Ken referred to has completed. I often put a 1 second delay on the start after the stop has gone false.

I had a problem like this recently after I changed companies. The new company had a PF 700, they had replaced a 1336 Plus, because the drive kept overvolt faulting when the line would start. I changed the stop to a ramp to hold using the DC injection. The drive quit faulting, but would not start with the rest of the line. I then added a delay to the start after the stop was released and have not had any more problems.

 

[OkiePC]

Overlapping the start command with the running status of the drive can sometimes lead to this problem. Some drives will tolerate a limited overlap of the start command with the stop command, but I don't have the specs handy for this case, and this probably varies between hardwired and network control methods.

In your case, I believe that you have access to the drive running status bit which you could monitor, and/or incorporate into the logic to a) detect a successful stop. b) ensure a successful start.

Also, this is just something to research, but there may be drive parameters which may allow the overlap of these conditions.

And, I agree with Ken, that your thread is an outstanding example of how to do it up right.

Phil needs to make this a Sticky with the title "ATTENTION NEWBIES" and a paragraph written by Ken about how to get your problem solved in fine form and fashion as you are doing.



EDIT: Did you get the drive configuration set back to the correct rating in your Logix project?

 

[ebolbol]

Dear OkiePC,

Thanks for your encouraging comments... much appreciated. To answer your question, yes I matched the drive with the project and it seems to be happy. As far as all the rest of your comments, they make sense, but I wouldn't know how to go about putting logic in there to correct it. I did however, set up a datalog model to trend all the inputs, and all the outputs associated with one of the drives. The engine will be shutdown tonight and I should capture all the activity that takes place in the bits going out, as well as coming in from the drive. I may or may not catch what's happening, but I'm sure once I post it, one of you guys probably will. Hope it does some good.

Overlapping the start command with the running status of the drive can sometimes lead to this problem. Some drives will tolerate a limited overlap of the start command with the stop command, but I don't have the specs handy for this case, and this probably varies between hardwired and network control methods.

In your case, I believe that you have access to the drive running status bit which you could monitor, and/or incorporate into the logic to a) detect a successful stop. b) ensure a successful start.

Also, this is just something to research, but there may be drive parameters which may allow the overlap of these conditions.

And, I agree with Ken, that your thread is an outstanding example of how to do it up right.

Phil needs to make this a Sticky with the title "ATTENTION NEWBIES" and a paragraph written by Ken about how to get your problem solved in fine form and fashion as you are doing.



EDIT: Did you get the drive configuration set back to the correct rating in your Logix project?

 

[Ken Roach]

Three Tags with One Purpose

Open up RSLogix 5000 and go to the "Periodic PID" Task, then the "PID LOGIC" Program, then the "U8_CW_CNTRL", and scroll to Rungs 51 and 52.

You'll see that Rung 51 controls the Start command to the drive by setting two BOOL tags true:

CWF_04A1_START_CMD is set true and held true for 45 seconds.
CWF_04A1_START_ENABLE is set true and remains true for as long as the drive needs to run.

Earlier in the routine (Rung 31), CWF_04A1_START_CMD in turn sets the actual U8_470_04A1:O.Start command for the drive true.

On the next rung after the START logic (Rung 51) you'll see that when the CWF_04A1_START_ENABLE is false, the CWF_041_STOP_CMD is set true, and held true for 5 seconds.

Ignore the "Start Timer" and "Stop Timer" timers for now.. they are holdovers from old motor control methods.

Now, consider the way this logic is executed. It's in a subroutine that is called from the PID_MAIN routine every scan of the PERIODIC_PID Task.

Periodic Tasks are the way the ControlLogix executes time-dependent logic. Since PID instructions are dependent on being executed at a specific periodic rate, they can run most accurately when the ControlLogix operating system executes them at precise intervals.

The PID_MAIN Periodic Task is configured to execute every 100 milliseconds, and the PID instructions used in the routines in that Task are configured for a Loop Update Time of 0.1 Seconds... good !

The PowerFlex 700VC drives are also configured to exchange their I/O with the ControlLogix every 100 milliseconds.

While that might have been intentional by the original programmers, it's more likely that "100 milliseconds" sounded like a nice round number.

[Continued......]

 

[Ken Roach]

[......Continued]

The Requested Packet Interval (RPI) for the PowerFlex drives and the Periodic Task rate are by coincidence the same, 100 milliseconds.

But they are not synchronized in any way. This is a surprise to most new ControlLogix users because virtually all older PLC architectures did their I/O scanning in between program scans.

ControlLogix I/O updates are asynchronous. That 100 ms RPI could time out at any time and grab the Input and Output tag values from the Controller tag database and ship them off over EtherNet/IP to the drives.

I'm trying to figure out if there's a way for the RPI expiring in between scans of the PERIODIC_PID Task could cause the actual drive's O.Start and O.Stop bits to both be true at the same time, which would cause a "failed Start". I'm getting a little dizzy chasing the logic around in circles.

I know how I'd trap it on the wire (with Wireshark) but I'm not certain that's the problem. Hmm.

 

[ebolbol]

Ken,

I followed along with what you said and navigated around the program and it looked like everything you said is right on. Do you think it's the logic that is causing this? Incidentally, I tried to log ALL module defined tags associated with Unit 10's VFD which was running last night in hopes of capturing data before/during/& after a unit shut down. Well, they did shut the unit down at 11:30 last night, but Unfortunately my data log file grew so big that I could not plot it to see if I could see suspicious events. It generated errors trying to plot... So I have to try again but this time not log everything and only pertinent data. Any suggestions?

[......Continued]

The Requested Packet Interval (RPI) for the PowerFlex drives and the Periodic Task rate are by coincidence the same, 100 milliseconds.

But they are not synchronized in any way. This is a surprise to most new ControlLogix users because virtually all older PLC architectures did their I/O scanning in between program scans.

ControlLogix I/O updates are asynchronous. That 100 ms RPI could time out at any time and grab the Input and Output tag values from the Controller tag database and ship them off over EtherNet/IP to the drives.

I'm trying to figure out if there's a way for the RPI expiring in between scans of the PERIODIC_PID Task could cause the actual drive's O.Start and O.Stop bits to both be true at the same time, which would cause a "failed Start". I'm getting a little dizzy chasing the logic around in circles.

I know how I'd trap it on the wire (with Wireshark) but I'm not certain that's the problem. Hmm.

 

[Ken Roach]

Most data logging software is good for trends and things that happen slowly over time, but not very good at all for capturing short-duration Events.

What software tools do you have available; just RSLogix 5000 or do you have some HMI or SCADA software ?

For "logic trap" work like this, I prefer to build array tags to store data, use ladder logic to populate those arrays with useful information, and then grab it out and parse it later.

What I would do is add some logic at the end of the Main Routine of the PERIODIC_PID task to pack the internal bits and the I/O tags into a 32-bit DINT and store that DINT and the 7-DINT DateTime timestamp any time those values have changed. Make the array big enough to hold a few hundred events and test the logic carefully to make sure you don't index past the end of the array.

Then you'll have a set of Events you can review.

I'll see if I can find time to write an example... got some deadlines to meet.

 

[ebolbol]

Actually we have FactoryTalk View Studio SE Edition and I build Datalog Models in there and trigger it off of an event or start it on demand and it writes a *.DAT data log file to disk where you can grab it and plot it using RSView Enterprise File Viewer.exe It did me no good because the file grew to 600Mb and clogged up the viewer program due to its size. Dead end there.

However, I just finished putting the system through a vigorous testing (all in manual mode). The rungs below, as referenced in your last post 51 & 52, show the system in manual mode. As you stated... Start command latches in for 45 secs and the tmr .DN bit takes it out. Stop command latches in for 5 sec until the tmr .DN bit takes it out. And the drive goes in the Stopped mode with STS blinking. Here is the interesting part. In manual mode (which is what you see below) I started and stopped the fan numerous times with out having to go to the drive and push the green button!!! for some reason in manual mode, even though it drives the same exact bit as auto, it doesn't care whether the drive is stopped or not, it runs the fans no matter what... I don't understand if manual starts the drive regardless of Active or not, why can't Auto??? I did it 5 times, and each time the drive went into the "Stopped" mode with STS blinking. The next time I hit manual push button it just picked up and ran???

Most data logging software is good for trends and things that happen slowly over time, but not very good at all for capturing short-duration Events.

What software tools do you have available; just RSLogix 5000 or do you have some HMI or SCADA software ?

For "logic trap" work like this, I prefer to build array tags to store data, use ladder logic to populate those arrays with useful information, and then grab it out and parse it later.

What I would do is add some logic at the end of the Main Routine of the PERIODIC_PID task to pack the internal bits and the I/O tags into a 32-bit DINT and store that DINT and the 7-DINT DateTime timestamp any time those values have changed. Make the array big enough to hold a few hundred events and test the logic carefully to make sure you don't index past the end of the array.

Then you'll have a set of Events you can review.

I'll see if I can find time to write an example... got some deadlines to meet.