PowerFlex 700 Vector losing link to PLC

ebolbol · Jul 7, 2011

Hoping to catch a seasoned guru's attention on this thread who knows these drives.

We have 3 PowerFlex 700s controlling fan motors in a cooling tower that cool jacket water temps in big gas engines. We have ControLogix 5000 plc’s talking to a remote I/O rack which talks to the PF 700’s through a 20-COMM-E module. A PID logic written by the integrators who put this stuff in controls the VFD's which vary speed of the fans from 20 Hz to 60 Hz.

Works great, as long as the Ethernet link is not lost. Here is the problem from day one. Every time the engines shutdown unexpectedly or sometimes even brought down normally, you can be certain that the link between the Logix and VFD's has been compromised or lost. So the next time the engines start, if you don't manually hit the green button on the powerflex's, you pretty much don't have fan control and will overheat engines. Even though the Logic is executing locally in the plc, it is not being communicated to the powerflex modules. I’ve been asking management to send me to training on these stupid things, but I can see that’s not gonna happen

. I know zero about the 700’s, how they talk to local plc, the settings, or config.

Any information on why this is happening or anything in general on these would be deeply appreciated. When I drill down within rslogix, I can see the drives, but can not see them through rslinx. I think because they are talking through a remote rack with a different IP base.

Ken Roach · Jul 7, 2011

Post that screenshot at a larger size, or post the *.ACD file itself.

Your description raises the red flag of "more than one network bridge between the CPU and the network" but the screenshot seems OK, but I can't really see it well.

In your application are the engine shutdowns due to a communication loss from the controllers, are just part of ordinary operation sequences ?

Clay B. · Jul 7, 2011

First thing I would have is a reset on the Powerflex Drives that clears the com fault. As to the cause that can be several things, Ken pointed at one of the most common. If you look in your Global Tags file you will find a Tag related to the VFD that is a Fault Clear also ther is some monitoring bits you can use in your program. There is a watchdog that can be used on the Control Logix level but not on the Compact logic level that monitors the comm connections. Myself I like to back door it. I contunously check the drive speed I am sending the drive compared to where it is at. This little function checks all kinds of things including the condition of the Comms link.

Example: If I send a comand to go 30 Hz and my responce from the drive is it is still at 0 hz I know something aint right with the drive.

One thing that bites alot of people is the fact that they think data stops when comms goes down. This is not true. Your data does not "Update" IE. If the Comm link went down before another fault then the fault bit did not update so it is still saying everything is sitting pretty when in fact your hosed.

Dig around in your global tags and you will find a whole bunch of tags already setup for your drives.

widelto · Jul 7, 2011

I got the same problem and i fixed it when ethernet switch was changed.
Your attached fig is too small can you send a bigger one ??

Ken Roach · Jul 7, 2011

A very common programming error with AC drives is to not consider a network connection disruption in the Start/Stop logic.

For these drives, I would just use an ordinary GSV instruction to grab the EntryStatus value from the Module Object for each drive. This mirrors the "exclamation point" that shows up in RSLogix 5000 when there's a connection error, and you can use it in your logic to allow the controller to send a new Start command to the drive. You'll find lots of examples of this logic on this Forum and in the RA Knowledgebase.

A-B drives always use network control similar to a "three-wire" system where you must send a low-to-high transition of the Start bit to start the drive. Just having the Start bit held high when the drive is powered up or reconnected to the network won't do the trick.

Similarly, the STOP bit always trumps all other commands. I don't program the Stop bit as "if not Started, then assert Stop", but rather "if not Started, then assert Stop until Running status is false".

RA drives can be configured to hold their last state in the event of a communication loss, or to fault, or to stop, or even to go to a specific speed and direction.

Zip and post the *.ACD file and there will be folks who can help.

ebolbol · Jul 8, 2011

Hello Ken, Thanks tons for pickin this up. I will make an attempt to post a bigger picture. All I'm trying to show in the picture is, when I open the main project, (attached), I can see the drives on the left pain when I drill down. But cant see in RSLinx. I think because of the different IP base. Here goes my attempt to attach pic and *.ACD

here is where the ACD is:
https://docs.google.com/leaf?id=0Bx...YTktYjBhMGZkMTQ2Mzg2&hl=en_US&invite=CIPv1ZkB

here is the image:
https://lh6.googleusercontent.com/-oZllQIEr9v0/Thc2OGF295I/AAAAAAAAAIE/fUHhz-mJv0M/s800/New%252520Bitmap%252520Image.jpg

Ken Roach said:
Post that screenshot at a larger size, or post the *.ACD file itself.

Your description raises the red flag of "more than one network bridge between the CPU and the network" but the screenshot seems OK, but I can't really see it well.

In your application are the engine shutdowns due to a communication loss from the controllers, are just part of ordinary operation sequences ?

ebolbol · Jul 8, 2011

What you're saying makes sense. One thing I didn't mention in my first post is the VFD doesn't go into full fault mode... meaning it doesn't have the red led on and fault code on lcd screen. The green STS led blinks, the lcd display has 20 hz and all else seems normal. So every time, we have to push the green start button on the keypad then you see the hz and everything else come alive. Somebody once said when your engine dies unexpectedly, the VFD doesn't know how to decelerate the fan and it goes in this neutral state. However, they didn't elaborate on it and I didn't know what to do. I have no software to view the settings on the VFD and the "onboard" functions are too complicated for me to mess with. I’ve been living with this for 4 yrs

Clay B. said:
First thing I would have is a reset on the Powerflex Drives that clears the com fault. As to the cause that can be several things, Ken pointed at one of the most common. If you look in your Global Tags file you will find a Tag related to the VFD that is a Fault Clear also ther is some monitoring bits you can use in your program. There is a watchdog that can be used on the Control Logix level but not on the Compact logic level that monitors the comm connections. Myself I like to back door it. I contunously check the drive speed I am sending the drive compared to where it is at. This little function checks all kinds of things including the condition of the Comms link.

Example: If I send a comand to go 30 Hz and my responce from the drive is it is still at 0 hz I know something aint right with the drive.

One thing that bites alot of people is the fact that they think data stops when comms goes down. This is not true. Your data does not "Update" IE. If the Comm link went down before another fault then the fault bit did not update so it is still saying everything is sitting pretty when in fact your hosed.

Dig around in your global tags and you will find a whole bunch of tags already setup for your drives.

ebolbol · Jul 8, 2011

see if this helps:

it only seems to happen when there is a shutdown on the engine or when the engine has been brought down, it's never done it when the engines just run and run for weeks.

widelto said:
I got the same problem and i fixed it when ethernet switch was changed.
Your attached fig is too small can you send a bigger one ??

Ken Roach · Jul 9, 2011

The first thing we'll do is take advantage of a little trick for navigating across the ControlLogix backplane using RSLinx Classic.

Because you have two separate EtherNet/IP networks, you can't automatically browse the remote one. You have to configure RSLinx Classic to browse specific IP addresses on the Local_Control_Network.

Browse to the ControlLogix backplane in RSLinx, then to the 1756-ENBT in Slot 2 and through it to the Ethernet network icon (the Local_Control_Network).

Right-click and select Properties, then fill out the Browse Addresses table with the three addresses of the AC drives (128.3.3.8, 128.3.3.9, and 128.3.3.10).

Getting this browsing path working will allow you to browse to the drives so that you can upload their parameters into the embedded DriveTools applet inside RSLogix 5000.

Once we get the parameters for both the PowerFlex 700 drives and the 20-COMM-E modules uploaded and saved, post that version of the *.ACD file and we'll have a look through it.

I suspect that the drives are configured to run at 20 Hz in the event of a communication connection loss, rather than fault or stop. The 20-COMM-E parameters will tell us for sure.

Follow the Ethernet cable from the 20-COMM-E to the nearest Ethernet switch it connects to and figure out if that switch might be getting powered down or otherwise affected by the state of the engine whose cooling tower it is running.

While you're poking around with RSLogix and RSLinx, I also want you to right-click on every CPU and network module, select Properties, and write down the serial number. We're going to do a survey of this system for the infamous "Atmel 64" backplane chip issue from 2005. If you have potentially defective network modules installed then we have to eliminate them as a possible source of problems.

ebolbol · Jul 9, 2011

I can't wait to go in Monday morning and try everything you said first thing!!! I'm excited!!!

I get in at 6:00 am and that's the first thing I will do and send you results... I think I am safe on the Atmel thing.... I remember about 3 or 4 yrs ago having to go through each ENBT and looking at the brand names of the on-board chips and the serial numbers associated with them. But wont hurt to recheck... thanks for being thorough & covering all bases! Is the the below link what you are referring to?
https://docs.google.com/viewer?a=v&...ZDMtMmFjOTU3ODk3Nzg4&hl=en_US&invite=CPWDhY0D

ohhh were you able to open the ACD file link? Because thats how I will attach the newly modified one after I try all the things you suggested

Ken Roach said:
The first thing we'll do is take advantage of a little trick for navigating across the ControlLogix backplane using RSLinx Classic.

Because you have two separate EtherNet/IP networks, you can't automatically browse the remote one. You have to configure RSLinx Classic to browse specific IP addresses on the Local_Control_Network.

Browse to the ControlLogix backplane in RSLinx, then to the 1756-ENBT in Slot 2 and through it to the Ethernet network icon (the Local_Control_Network).

Right-click and select Properties, then fill out the Browse Addresses table with the three addresses of the AC drives (128.3.3.8, 128.3.3.9, and 128.3.3.10).

Getting this browsing path working will allow you to browse to the drives so that you can upload their parameters into the embedded DriveTools applet inside RSLogix 5000.

Once we get the parameters for both the PowerFlex 700 drives and the 20-COMM-E modules uploaded and saved, post that version of the *.ACD file and we'll have a look through it.

I suspect that the drives are configured to run at 20 Hz in the event of a communication connection loss, rather than fault or stop. The 20-COMM-E parameters will tell us for sure.

Follow the Ethernet cable from the 20-COMM-E to the nearest Ethernet switch it connects to and figure out if that switch might be getting powered down or otherwise affected by the state of the engine whose cooling tower it is running.

While you're poking around with RSLogix and RSLinx, I also want you to right-click on every CPU and network module, select Properties, and write down the serial number. We're going to do a survey of this system for the infamous "Atmel 64" backplane chip issue from 2005. If you have potentially defective network modules installed then we have to eliminate them as a possible source of problems.

Ken Roach · Jul 9, 2011

I was able to download the *.ACD file you first posted, but not that most recent file. I'm not familiar with the Google Docs file sharing mechanism but it looks promising.

Generally I just ZIP files and attach them natively to the Forum so they're stored on the PLCTalk server but it will save resources and space for the PLCTalk host to use DropBox or Google.

I had a look at the start/stop logic for the fan drives and it looks like there's logic to detect a "failure to start" but not a communication failure. We'll have a closer community look at that.

ebolbol · Jul 11, 2011

Hello again, hope you had a great weekend.
I did all the things you described… I think???
RSLinks Classic went off without a hitch
In RSLogix PoweFlex drive properties I hit “connect to drive” and it said there is a firmware mismatch, I went offline and changed the rev to 4 since that’s what the drive had. Re-tried, and it let me get passed but stopped me when it said there is a mismatch in drive rating. Drive is 480VAC 27a but who ever installed the drives initially had 400vac 1.5a, went off line again and changed rating to match and went back online. It actually let me connect to the drive!!! But before it does it gives me the following…

I hit upload and it connected to the drive. I did this for all three. But it does not stay connected, each time I go to the properties I get this

Didn’t know what to do

, so I clicked on the function below where it says create new database files from online drive then I saved the project…

I hope I didn’t screw anything up… I have back ups of the project just incase, but it was nerve racking going in there and changing things…

I hope I didn't screw anything up... The new *.ACD file is here:
https://docs.google.com/leaf?id=0Bx...N2I0Yi00NjI1LWEzYzItOTFlZmM2MjI4Y2E2&hl=en_US

Ken Roach said:
The first thing we'll do is take advantage of a little trick for navigating across the ControlLogix backplane using RSLinx Classic.

Because you have two separate EtherNet/IP networks, you can't automatically browse the remote one. You have to configure RSLinx Classic to browse specific IP addresses on the Local_Control_Network.

Browse to the ControlLogix backplane in RSLinx, then to the 1756-ENBT in Slot 2 and through it to the Ethernet network icon (the Local_Control_Network).

Right-click and select Properties, then fill out the Browse Addresses table with the three addresses of the AC drives (128.3.3.8, 128.3.3.9, and 128.3.3.10).

Getting this browsing path working will allow you to browse to the drives so that you can upload their parameters into the embedded DriveTools applet inside RSLogix 5000.

Once we get the parameters for both the PowerFlex 700 drives and the 20-COMM-E modules uploaded and saved, post that version of the *.ACD file and we'll have a look through it.

I suspect that the drives are configured to run at 20 Hz in the event of a communication connection loss, rather than fault or stop. The 20-COMM-E parameters will tell us for sure.

Follow the Ethernet cable from the 20-COMM-E to the nearest Ethernet switch it connects to and figure out if that switch might be getting powered down or otherwise affected by the state of the engine whose cooling tower it is running.

While you're poking around with RSLogix and RSLinx, I also want you to right-click on every CPU and network module, select Properties, and write down the serial number. We're going to do a survey of this system for the infamous "Atmel 64" backplane chip issue from 2005. If you have potentially defective network modules installed then we have to eliminate them as a possible source of problems.

Ken Roach · Jul 11, 2011

Great work !

I'll bet that the original programmers didn't ever use the embedded Drive Executive features, which would explain why the drive in the database was totally different from the actual drive.

I'm not sure that all the parameters got uploaded properly, though.

Select the PowerFlex and do an Upload, and then select the 20-COMM-E and do an Upload, then Save the project. Do this for all three drives.

What I was looking for was a "Comm Fault Action" (parameter 22 in the 20-COMM-E) that caused the drive to run at 20 Hz, but I didn't find it.

The other reason the drive might run at 20 Hz in the event of a communication failure is if there are local circuits that set it to a Preset Speed... the third Preset Speed is 20 Hz by default. There would have to be some sort of relay or other circuit near the drive that used the Discrete Inputs to do that.

So go check out the wiring near the drive and compare it to the discrete input wiring diagram in the user manual to see if there's something connected to the Preset Speed Select inputs, and then upload the parameters using RSLogix 5000, and we'll have another look.

ebolbol · Jul 11, 2011

I will definitely do that as soon as I finish up some work! Can't wait!! One thing that has me kinda worried is when I changed the Device rev and power rating offline, I did a download. By me changing those two items, does that change the configuration of the devices and how the programmers set them up? I Didn't change or delete the devices, just rev and rating which was mismatched! I wonder if there was a reason they had gone with 400 vac 1.5a? or was it just an honest mistake? I later found all three drives faulted red as a result of my offline downloads (i think?). I had to press the red button followed by the green to start them back up. I wonder if that was the reason you're not seeing what you're looking for? But it did say it was connected to the device

But apparently I was dealing with faulted devices... just confused

and it amazes me how much you know about these things...

Ken Roach said:
Great work !

I'll bet that the original programmers didn't ever use the embedded Drive Executive features, which would explain why the drive in the database was totally different from the actual drive.

I'm not sure that all the parameters got uploaded properly, though.

Select the PowerFlex and do an Upload, and then select the 20-COMM-E and do an Upload, then Save the project. Do this for all three drives.

What I was looking for was a "Comm Fault Action" (parameter 22 in the 20-COMM-E) that caused the drive to run at 20 Hz, but I didn't find it.

The other reason the drive might run at 20 Hz in the event of a communication failure is if there are local circuits that set it to a Preset Speed... the third Preset Speed is 20 Hz by default. There would have to be some sort of relay or other circuit near the drive that used the Discrete Inputs to do that.

So go check out the wiring near the drive and compare it to the discrete input wiring diagram in the user manual to see if there's something connected to the Preset Speed Select inputs, and then upload the parameters using RSLogix 5000, and we'll have another look.

ebolbol · Jul 12, 2011

Ken,

I led you down the wrong path, I said when the plc loses control..., the VFD is at 20 hz, but I just realized it's actually 0 when it's in that state with STS blinking. Without further due, I took some very good picturs of both controlling state and non-controlling state complete with associated input statuses to give you a contrast between good and bad. There is nothing running in the plant right now. I don't want to use the term (faulted state) because it is not, no fault indication or red led at all. Please see below. Something makes the VFD's go in the state you see in the second picture that makes STS blink and hz to drop to zero & not control. So sorry for mis speaking. p.s. the callout I have below that says "speed" should actually say "At Speed"...

Ken Roach said:
Great work !

I'll bet that the original programmers didn't ever use the embedded Drive Executive features, which would explain why the drive in the database was totally different from the actual drive.

I'm not sure that all the parameters got uploaded properly, though.

Select the PowerFlex and do an Upload, and then select the 20-COMM-E and do an Upload, then Save the project. Do this for all three drives.

What I was looking for was a "Comm Fault Action" (parameter 22 in the 20-COMM-E) that caused the drive to run at 20 Hz, but I didn't find it.

The other reason the drive might run at 20 Hz in the event of a communication failure is if there are local circuits that set it to a Preset Speed... the third Preset Speed is 20 Hz by default. There would have to be some sort of relay or other circuit near the drive that used the Discrete Inputs to do that.

So go check out the wiring near the drive and compare it to the discrete input wiring diagram in the user manual to see if there's something connected to the Preset Speed Select inputs, and then upload the parameters using RSLogix 5000, and we'll have another look.

PowerFlex 700 Vector losing link to PLC

ebolbol

Member

Ken Roach

Lifetime Supporting Member + Moderator

Clay B.

Lifetime Supporting Member

widelto

Member

Ken Roach

Lifetime Supporting Member + Moderator

ebolbol

Member

ebolbol

Member

ebolbol

Member

Ken Roach

Lifetime Supporting Member + Moderator

ebolbol

Member

Ken Roach

Lifetime Supporting Member + Moderator

ebolbol

Member

Ken Roach

Lifetime Supporting Member + Moderator

ebolbol

Member

ebolbol

Member

Similar Topics