Worst thing to ever happen!!

@ ceilingwalker no problem anytime. I'm also impatiently waiting to know what the reason was for the two production lines to come to a stop from changing a light bulb, logically thinking i just don't see how he was able to do that or how that was possible, unless maybe the power supplying both the light bulb and PLC's were driven off of the same power supply and while changing the light bulb with power on could have created some undesired current to flow back in the circuit thus creating a short maybe?? but then again he did say it takes skill, i crack up every time i read that.

Talking of the 'light-bulb' thing.
I once had a machine that would blow the control fuse randomly.

I would go to the machine, replace the blown fuse and check the control wiring and find nothing. The machine would run happily for the next 3 hours/3days/3 weeks. Then pop, it would blow again.
It became my nemesis. Over time, I think I checked every wire on that machine.

Then one day I took the light stack apart and there it was!

A bulb holder was broken and the bulb had fallen out. Every now and then - with the vibration, the metal part of the bulb would momentarily short across the bulb pins.

I could see all the little 'short' marks on it.

I told them it was fixed for good but nobody believed me

Not the worst ever thing to happen, but boy did it take me a while to find.

we have a simple machine here which checks if kegs are upside down. If they are we send them to a machine which turns them the correct way up.

We have been having issues with the turner stopping occasionally, and for no apparent reason.

Speaking to the guys that run the line, they tell me that it only starts happening in the afternoon when it starts to get warmer outside.

So I start thinking maybe the motor is getting hot. It only runs at about half speed, so maybe the cooling fan is not turning fast enough to cool the motor.

Next time the turner stops I go down and the motor is cool as ice.

So I get the laptop out and hook it up, and start to look for the reason this thing keeps stopping. There are 2 lines of code that stop the machine, so as I am watching the 2 lines of code, the machine stops. Damn, didn't see anything. It has to be in one of these lines of code so I set up a trap to catch which bit is going true.

After an hour or so, they call me. The machine has stopped again. So off I go confident that I am going to find the problem. Sure enough I have caught the problem with my trap, and it turns out there is a sensor on the infeed that was triggered.

So I go to look at the sensor, and hanging off some of the guarding is the operators jacket.

Every now and then his zip would waft past the sensor stopping the machine.

Sure enough it only happened when it got warm because this was the only time the operator took his coat off.

Mark

He he, that's a funny one Mark.
I did a great project setting up a conveyor system. It was like a railway marshaling yard.

The product was diverted on the conveyor system depending on conditions.
I set it up virtually on my own - throwing objects on the conveyors and checking they went where they should.

When it went into operation it would randomly seem to do what it wanted. I was sure my logic was spot on but I saw it with my own eyes (as the operators termed it) going nuts.
There's nothing like the panic you feel when the product is backing up and not going where it should and all the operatives are cheering and laughing.

After much checking and head scratching I saw some 'end of conveyor' photo-cells actuating without anything on the conveyor.

Then it dawned on me - oh what a lovely feeling of relief.

All the operatives were wearing hi-viz jackets and the photo-cells although working correctly had been fastened to the conveyors as they came out of the box. (no sensitivity adjusting) They were seeing the operatives silver reflective strips on their jackets as they walked past.
The photo-cells had corresponding reflectors opposite but boy can they see a long way when they are not set up right.

Photosensors have gotten a lot more sophisticated in recent years in terms of rejecting extraneous reflections, and I'm glad.

I had a material storage and retrieval system that started malfunctioning after a few months in service. The main travel cart would 'take off running' even when it was already in position in front of a storage rack.

An operator gave me a clue. "It seems to be afraid of the new forklift. It wasn't afraid of the old ones."

I was using a long-distance laser sensor to determine the approximate position of the cart; it gave me an analog value proportional to the distance between the laser and a reflector bolted to the wall a hundred feet away.

There was a walkway through this storage area, and I had programmed the system to handle loss of signal when people and vehicles blocked the laser.

But I had not tested what would happen if the laser got a stronger return signal... like it did from the brand-new, shiny forklift truck.

Goody said:

All the operatives were wearing hi-viz jackets and the photo-cells although working correctly had been fastened to the conveyors as they came out of the box. (no sensitivity adjusting) They were seeing the operatives silver reflective strips on their jackets as they walked past.
The photo-cells had corresponding reflectors opposite but boy can they see a long way when they are not set up right.

Click to expand...

I has a similar thing happen last week. One of our customers just started requiring us outside contractors to wear these vests. On one of the conveyor lines, they have a diffuse sensor to detect product for a vision system, with an air jet to blow off failed product. Every time I walked past (even at quite a distance), it would trigger the camera. Since no product was present, it got a fail signal, and the air jet would fire. Took me a minute or so to realize it was seeing my vest...



-Eric

I'm tellin y'all it's sabotaaaage

In a certain aerospace factory, there are some plating and dip tanks controlled by several interconnected PLC-5 controllers, including some PLC-5 Classics.

I was brought in to diagnose why they were having numerous problems with the DH+ network, which had been installed for over 15 years.

As I was talking to the engineer who had been troubleshooting the system, he got a call on his radio, and ran out of the room without explaining what he was doing. When he returned, he had a PLC-5/25 in his hand. He opened a box next to the desk and put it in... with twenty other PLC-5/25 controllers.

"Bad, every one of them. We've cleaned out eBay", he said.

I was floored. Their troubleshooting method for this DH+ network had been reduced to basically: "if there is a comms error, replace the CPU'.

I spent the next two days carefully monitoring error counters, following DH+ cables all over the factory, and taking oscilloscope traces of the signal quality. There was definitely something wrong with the signal, but I couldn't tell if it was induced noise, damaged wiring, or a damaged device. I replaced a few corroded connections, but the errors persisted.

Until I looked at one desk, out by the operator's console, where a PC running RSView32 had a 1784-KTX card in the back. This PC was laid on its side, so the DH+ connector was 90 degrees from its usual orientation.

Very carefully balanced across the stripped conductors of the blue and clear wires on the DH+ cable was one steel staple. Like for sheets of paper. A staple.

When I removed it, the network ran perfectly. There had never been anything wrong with any of their box of 'bad' PLC-5 controllers.

New safety engineer at a plant said those cranes have to have lights on them so people know when it is moving. The stamping press 3 crane bays over would randomly stop. The strobe light off the crane was causing excess light return. When I called the manufacturer they said yea we have seen this a few times. You can use a green trash bag to filter out the excess noise. The strobe light just burned out very very quick. The safety girl never noticed it.

DH+ is a trip. I once heard from a corporate EE who was working on an intermittent DH+ problem on a PLC-5 and got shocked by 120 vac, he tracked it down to where someone had jumpered one of the wires to 120vac where it was landed on adjacent terminal blocks. And, according to this guy who seemed very believable, and a really smart and good engineer, it worked about 80 percent of the time with 120vac riding on it.

I witnessed a case of intermittent DH+ communication problems at a automotive assembly plant caused by a maintenance person hanging a bucket on the blue hose to catch water from a ceiling leak.

Was once asked to check out the wiring between two older plc's at the top of a large open pre-heat tower (270'). Me and the programmer at the time figured it out but needed to de-energize a 120v control circuit before adding a new wire. We figured this circuit wasn't doing anything so I opened the fuse holder and every large contactor around my head de-energized. The plant boss was with us with a radio. The people in the kiln control room started to go crazy over the radio, a rescue crew had to come and get us, and we made it to the bottom. Then entire top of the pre-heat tower could not be seen from black smoke, so they had to investigate what caused this before firing everything back up, and I calmly told the boss, um, I put the fuse back in, fire the damn thing up before the pre-heat tower melts. That was a Friday afternoon on a long weekend, got really drunk.

Had a checkweigher/sorter where I used to work. Being the cheap company they were at the time, they decided to buy used equipment made in Europe for the absolute bottom dollar. This particular scale was a Scanvaegt brand, made in Iceland, in 1992. They don't even sell Scanvaegt in the US. The parent company, Marel, would provide service, but the model was so old they only had one guy who knew anything about it.

Anyway, this scale would go down every few months, and it would go down BAD. Multiple-day downtime. Wouldn't hold-up production, but made it a lot more labor-intensive. So needless to say, the pressure was on. The beautiful thing about it is that it followed the classic european philosophy of being designed under the assumption that nothing will ever go wrong with it, thus any notion of making it easy to repair or using easily-obtainable parts goes right out the window. Components crammed and shoehorned in to the tightest spaces possible, almost no space between device terminals and wire duct. This was basically a box filled in circuit boards in a heavy wash-down environment. And the circuit boards were very, very custom. Even the power supply board had I/O on it. And boy did those engineers LOVE bulkhead connectors, because they put them EVERYWHERE, even inside the panel.

Anyway, I'm rambling (so much rage involved with that machine). One time the sorter went down. There's a motor aux contact which goes to an input in the power supply, and out of the power supply a wire going to a D-shell connector on the ADC board of a very custom, very proprietary PLC. The display told us the motor wasn't running when it was. After three days of downtime, my boss finally discovered the problem. There was a small inductive reactor in series with the wire from the power supply to the ADC board that was bad. We replaced it with a resistor (its purpose was the drop the voltage from 24 to 20VDC) since no electronics store nearby carried the part and it worked fine.

3 days of downtime because of a part smaller than my fingernail.

My claim to fame was when working as a service engineer for a labelling company.
I had to go to Cadburys in Bournville because there was a "Fault" with the machine we supplied.
After eventually getting on site somebody pointed the machine out to me and when I asked "what's wrong?" they said they'd go and find out.

Being a high profile customer, I thought I'd give the machine a check over while i was waiting. It wouldn't start and as soon as my hand checked to see if the emergency stop button had been pressed, the whole production hall came to a standstill.

They had interlocked every stop - even on machines that weren't running - to shut the whole lot down!

Ned_Flanders said:

They had interlocked every stop - even on machines that weren't running - to shut the whole lot down!

Click to expand...

IMHO that is quite correct - an "Emergency Stop" button should stop everything that can be seen from its location - that is why it is there.

They are intended to be used by {untrained} people who see something going pear-shaped from where they are located.

It is not a "safe" system if someone has to think about whether the EM Stop button closest to him will do the job, it should.

As for the local machine not running, that is irrelevant, an Emergency Stop system should shut down all machines in the locale...

Good thing our system isn't built like that.
From several machines I could then shut down the entire plant, cause I can see the entire plant from there.
Not to mention some of those machines require e-stop being pressed when changing setpoints on the servor drives. (needs drive inhibit to accept changes)
Change setpoint 2 mm, wait, lemme just shutdown the entire plant for that.

Had a similar system installed though. A lift at the end of each production line, which takes the products to palltizers on a different floor and also not visible because there's a wall blocking the view.
E-stop on lift/elevator would stop the entire system.
Only did that once.