ControlLogix 5000 EtherNet/IP Mystery

We have an Allen Bradley ControlLogix 5000 system operating for about eight months. It controls a stamping machine. anyway, it broke down on December 28, 2009. Then suddenly started to work from the next day morning. No surprise that it has broken down again two days ago, after running for about 12 days. I'll explain the symptom of the breakdown. But let me describe a strange thing that I noticed here.

Description of the PLC.

1. Power supply
2. CPU ControlLogix 5555 (slot 0, catalog 1756-L55 A)
3. 2-Axis servo controller (slot 1, catalog 1756-M02AE/A)
4. DC Input card (slot 5, catalog 1756-IB32)
5. DC Output card (slot 7, catalog 1756-OB32)
6. EtherNet/IP communication bridge (slot 8, catalog 1756-ENBT A)
7. DeviceNet card (slot 9, catalog 1756-DNB)

Now, from my limited IT knowledge I think the EtherNet/IP bridge should connect the PLC system with some external IP network. But strangely enough, there is absolutely no connection to this module, except the backplane.

There is an HMI connection in the system, but that is on the DeviceNet card, and through a CANbus cable.

Does anyone know why the EtherNet/IP card is there? Can it have any function I'm missing?


Now about the breakdown. The symptoms are . . .

• CPU starts up. "OK" and "RUN" LEDS glow right. "I/O" LED blinks.
• Servo controller - "OK" LED blinks. "FDBK" and "DRIVE" LEDs for both channels remain off.
• DC Input card - "OK" blinks.
• DC Output card - "OK" blinks.
• EtherNet/IP card - "OK" on. "NET" blinks. "Link" off. Scrolling display says "Link Lost".
• DeviceNet card - "OK" blinks, "I/O" blinks, "MOD/NET" on.
• HMI starts up, cannot read any param from the PLC

I need to fix the breakdown and get the production line back immediately. But I cannot find any clue. Is it possible that the EtherNet/IP card is looking for a link and finding none, it is stopping the CPU?

We have two other machines like this, and both have similar Ethernet card with NO external connections. None of them are reporting "Link Lost".

Thanks much for your time. I really need some insight into this mystery. Best regards.

I'll go out on a limb and say that the machine builder (or integrator) put it there for programming / monitoring. Shouldn't have any impact on your current issue.

About the issue, does the program remain in memory, or do you have to reload?

Thanks much Oakley. I'd like to believe that also.

The program remained in memory. I do not even have RSLogix and RSLinx for uploading/downloading :-(

Any thought?

This is really quick , try opening the program on line and look for a I/O warning triangle, this one shows where the I/O is missing or is at fault. It seems it has something to do with the ENBT board.

Can you give us the info from the hmi? brand /model #

If the hmi is on devicenet check all the devicenet gotchas such as resistors on each end of the run and only on the ends. The correct resistor type,Grounding,shielding,are they using a devicenet rated power supply? is it grounded properly?

If nothing is plugged into the ENBT then it is not being used. It is just there for programming i would assume.Does the processor have a ethernet port on it? If not the enbt is most likely just there to have a faster programming connection than serial.

Did they obey all the devicenet rules for cable type and length?

VoltSmith said:

Thanks much Oakley. I'd like to believe that also.

The program remained in memory. I do not even have RSLogix and RSLinx for uploading/downloading :-(

Any thought?

Click to expand...

Without the tools, about the only thing you can hope to do is power down the machine and reseat all cards and check all the connections.

I think the flashing I/O LEDs indicate that there is I/O that is missing, or not communicating...I have never heard of the "Link Lost" on the ENBT...

So, the OK LED is solid green on the CPU? That is a good sign that it is run mode...

The blinking I/O LED on the CPU could be pointing at the devicenet card which also has a blinking I/O led, so I would first suspect that a device on the devicenet network has a problem.

Are there other devices beside the Panelview?

With devicenet, check the power supply and cabling first, and look at the status of all devices...

To go much further, you will need some tools (RSLogix5000/RSNetworx for Devicenet)

Thanks everyone. I really appreciate the very quick responses.

@widelto, I could do that if we had RSLinx/RSLogix. Unfortunately we don't. We're in the process of ordering those (way expensive!).

@The Plc Kid, The Devicenet cable seems ok. It has resistors at both ends. Cable is screened and grounded. The power supply is good also. I don't know about rated power supply for DeviceNet. I'll check this. But that is probably ok. The supply comes from a Siemens SITOP PSU.

The HMI is PanelView. I'm away from the plant now. I'll update you about the brand/make ASAP.

@OkiePC, there is no other device on the DEviceNet card, only the HMI.

Looks like the common consensus is the EtherNet card was for program downloading/debugging only. That's a relief. But why does it suddenly saying "Link lost" in its status? Just after the breakdown?

I'll recheck the DeviceNet and get back to you. That's going to happen in seven hours from now. In the mean time, I'd be glad to receive any more tips that I can get. Thanks everyone again.

Rahul

Why not contact company that built the machine for troubleshooting? Clearly there is an issue so they would be the best to talk too.

Ethernet card "Link Lost" means no Ethernet connection exists, just confirmed it on my desk. I'll agree it's there for programming.

Is there any remote IO on the system??

Paully's5.0 said:

Ethernet card "Link Lost" means no Ethernet connection exists, just confirmed it on my desk.

Click to expand...

We must have been doing this at the same time!

No remote IO. The servo card controls a servo drive, and the CPU serial port controls a glue-sprayer controller. No remote IO is there, and the DeviceNet only connects to the HMI.

The OEM is also pointing to the DeviceNet card. We've checked the station id for their prescribed value. That was ok. We checked the cable also. That was found ok also.

I'll check the DeviceNet-HMI connection more rigorously tomorrow.

If the CPU is in run, this sounds like communications has been lost on the Backplane Control Bus. Since all of them are blinking, I would have to suspect a problem at the CPU end. Either with it's connectin to the Backplane, or the CPU itself.

The other possibility might be a bad card causing the communications to lock up, but you would think that the communications protocol is smarter than that.

If there is communications, the OK LEDs should be on solid. Might be worth a Rockwell Knowledgebase search, or at least a Google search.

The following is are a couple I found at Rockwell;

Quote/

A blinking I/O LED on the processor indicates it is not able to talk to one or more of it's scheduled connections to I/O modules listed in its I/O tree.
Make sure the I/O tree is properly filled in to reflect the I/O on the network. Next, check the first error shown in the treen. Open the module properties and go to the connections tab to view the error. The corrective action to take depends on a the error code. If the error code ends in 204, then it is a timeout, make sure the module is powered up and has a good network connection. An error ending in 317 means a connection not scheduled error. On controlnet, all connections should be scheduled with RSnetworx for Controlnet. There may by many other possible error codes. The main point is to get the error code first, then start troubleshooting from there. Once the first error is resolved, then check for the next remaining erro in the i/o tree and repeat this process as long as errors remain in the I/O tree.

/End Quote


Quote/

OK LED's blinking green I/O LED on processor solid green
If the rack with the blinking green ok is remote, remove power from the remote rack.
Does the IO LED on the processor start blinking ?
If this situation is in the local rack, does removing the one of the modules from the rack cause a yellow triangle for that module's slot ?

Chances are your rack is in program mode and the module with the blinking OK is an
output module. With the processor in program mode, the output modules are disabled.
If you put the processor in run, the OK LED's for these output devices will go solid green and the output modules will be enabled.


/End Quote

Stu......

You've got a whole lot of blinking OK LEDs there. That sounds like a backplane comms issue to me. That's not to say that it isn't one of the cards causing it, though.

CLX I/O modules are hot swappable. The next time this happens I would pull cards one at a time and see if you can get all the other cards happy. The PLC I/O indicator will never be happy is ANY of the cards are missing or unhappy.

If that doesn't help you may want to try a new backplane.

I wouldn't get too hung up on the DeviceNet thing just yet. What, if any, error code do you get on the DNet card?

Keith

Edit: sthompson beat me by a minute.

kamenges said:

You've got a whole lot of blinking OK LEDs there. That sounds like a backplane comms issue to me. That's not to say that it isn't one of the cards causing it, though.

CLX I/O modules are hot swappable. The next time this happens I would pull cards one at a time and see if you can get all the other cards happy. The PLC I/O indicator will never be happy is ANY of the cards are missing or unhappy.

If that doesn't help you may want to try a new backplane.

I wouldn't get too hung up on the DeviceNet thing just yet. What, if any, error code do you get on the DNet card?

Keith

Edit: sthompson beat me by a minute.

Click to expand...

I am sure it will be the other way around next time Keith.

Plus you have added some more valuable advice. Pulling the offending card, if it is a card problem, just might restore communications. It would be a hell of a lot easier if he had the correct tools to look for error messages.

Stu...

Try pulling out the ENBT card and reseating it. If the problem does correct itself after that or a power cycle look up Amtel.
Something sounds kind of off about the other 2 ENBTs don't say missing link and this one does. I had this problem on a system it was the ENBT card causing loss of comms on the backplane.