Phasma In Machina

TConnolly

Lifetime Supporting Member
TConnolly
Join Date
Apr 2005
Location
Salt Lake City
Posts
6,152
I found OkiePC's story in the ONS thread humorous. I didn't want to derail that thread, but thought it would be fun to share a few ghost stories.

I used to work at a company with a very good natured Navajo PLC programmer. One day a four drop 984 just wouldn't behave. We were chasing one mysterious ghost after another through both spaghetti code and spaghetti wiring. Finally he climbed up on top of the control cabinet and did a rain dance right there. He was just kidding around and we were all laughing with him but he got down and dead panned "It'll work now."

It did. All problems disappeared.

I don't believe in anything supernatural, but I swear that is what happened that day.

🍻
 
Another tire assembly machine, another weird one. This one would have a hesitation in the drum drive (big responsive 15kW servo driven drum) and cause scrap, but only with the electrical panel doors closed, making it especially challenging to troubleshoot! This might happen once or twice in an hour and then not again all shift, but that was enough waste to raise a squeak.

Obviously getting too hot, right? No, all fans worked filters clean, this went on for days,. This machine was on the main aisle and having the doors open to an unattended electrical panel was a no-no.

This goes on for two days, before I get dragged into it. When it acted up, it jerked as if the drive run command had dropped and came right back, and now it is doing it every few seconds. They open the door and it starting working right. Well the damn thing didn't cool off that fast.

So, we put skinny little ole John inside the panel with a flashlight and a mirror so he could watch the drive run LED, then carefully closed the doors to a barely ajar postiion so we could see him and, sure enough it was dropping out.

So, I checked the wire and sure enough it was tighter than necessary at both ends, but the terminal block in the middle no one cared to look for was completely backed out.

The weight of the open door flexed the panel enough to push the wire into good contact in the terminal block until you closed the doors, and it relaxed to a loose position sometimes.
 
Last edited:
I've seen the door wiring cause problems before, too: the customer didn't take too kindly to my suggestion that the controllers was "afraid of the dark".

But the weirdest one was in 1995, in central Ontario. A PLC-5 remote rack and a PanelMate that was throwing error codes, everybody yelling at me to "fix the program" and get back into production. I was coming up on 35 hours without sleep.

I stomped off the floor and went up to the control room where the German engineering crew had been having dinner while watching me struggle and climb around the equipment, and took their chicken bones out of the trash. Found a couple paper packets of salt.

I went back down to the control console and placed a paper bag full of chicken bones on top. Hopping on my left leg, I circled counterclockwise around the console, crying out wildly while tossing pinches of salt over my left shoulder. When I ran out of salt, I pulled down the disconnect handle, gathered up into the chicken bones, ran out the back door to the edge of the woods, and hurled the bones into the night.

When I cycled power the machine came back up and ran perfectly.
 
The Plc Kid said:
Okie

You guy's put a man in the panel. That's hard core. :)
Click to expand...

Demag was trying to get their buyoff from us for their skillet line, and the transition at the ends of the skillet lines just weren't working properly. So they stuck Keith in the panel and closed the doors while the Demag engineer showed the Saturn engineer that they had everyting working right. :)

The lesson here is, always keep an eye on the man behind the curtain!
 
You have to be very careful about the types of solutions that TConnolly and Ken used.

Many years ago a former colleague of mine was in a difficult startup. At one point he slapped the side of the control panel in disgust and immediately the thing began to work properly. So, with a sharpie he marked the spot he hit with an "X" and told everybody that 1% of PLC programming was writing good code and the other 99% was knowing where to hit.

Flash forward about six months. He was called back to the same facility for a problem on a different machine. As he walked by the control panel of the first machine he couldn't help but notice a very large dimple in the control panel around that X.

So Ken, I expect that by now there is mountain of chicken bones in the woods outside that plant and the floor around the control panel is white from salt.
 
Steve Bailey said:
You have to be very careful about the types of solutions that TConnolly and Ken used.

So Ken, I expect that by now there is mountain of chicken bones in the woods outside that plant and the floor around the control panel is white from salt.
Click to expand...
I feel sorry for the German Engineers chained up in the back room.
 
OkiePC said:
Another tire assembly machine, another weird one. This one would have a hesitation in the drum drive (big responsive 15kW servo driven drum) and cause scrap, but only with the electrical panel doors closed, making it especially challenging to troubleshoot! This might happen once or twice in an hour and then not again all shift, but that was enough waste to raise a squeak.

Obviously getting too hot, right? No, all fans worked filters clean, this went on for days,. This machine was on the main aisle and having the doors open to an unattended electrical panel was a no-no.

This goes on for two days, before I get dragged into it. When it acted up, it jerked as if the drive run command had dropped and came right back, and now it is doing it every few seconds. They open the door and it starting working right. Well the damn thing didn't cool off that fast.

So, we put skinny little ole John inside the panel with a flashlight and a mirror so he could watch the drive run LED, then carefully closed the doors to a barely ajar postiion so we could see him and, sure enough it was dropping out.

So, I checked the wire and sure enough it was tighter than necessary at both ends, but the terminal block in the middle no one cared to look for was completely backed out.

The weight of the open door flexed the panel enough to push the wire into good contact in the terminal block until you closed the doors, and it relaxed to a loose position sometimes.
Click to expand...
Ah, those were the days. Or in our cases, the middle of the night. Don't miss that at ALL! :sick:
 
OkiePC said:
Another tire assembly machine, another weird one. This one would have a hesitation in the drum drive (big responsive 15kW servo driven drum) and cause scrap, but only with the electrical panel doors closed, making it especially challenging to troubleshoot! This might happen once or twice in an hour and then not again all shift, but that was enough waste to raise a squeak.

Obviously getting too hot, right? No, all fans worked filters clean, this went on for days,. This machine was on the main aisle and having the doors open to an unattended electrical panel was a no-no.

This goes on for two days, before I get dragged into it. When it acted up, it jerked as if the drive run command had dropped and came right back, and now it is doing it every few seconds. They open the door and it starting working right. Well the damn thing didn't cool off that fast.

So, we put skinny little ole John inside the panel with a flashlight and a mirror so he could watch the drive run LED, then carefully closed the doors to a barely ajar postiion so we could see him and, sure enough it was dropping out.

So, I checked the wire and sure enough it was tighter than necessary at both ends, but the terminal block in the middle no one cared to look for was completely backed out.

The weight of the open door flexed the panel enough to push the wire into good contact in the terminal block until you closed the doors, and it relaxed to a loose position sometimes.
Click to expand...
Surely a very tough and time consuming job.. Just read your entire post and I think I need to hire an expert person for this.. It might consume less time.. But thanks for sharing information
 
ArthurFord said:
Another tire assembly machine, another weird one. This one would have a hesitation in the drum drive (big responsive 15kW servo driven drum) and cause scrap, but only with the electrical panel doors closed, making it especially challenging to troubleshoot! This might happen once or twice in an hour and then not again all shift, but that was enough waste to raise a squeak.

Obviously getting too hot, right? No, all fans worked filters clean, this went on for days,. This machine was on the main aisle and having the doors open to an unattended electrical panel was a no-no.

This goes on for two days, before I get dragged into it. When it acted up, it jerked as if the drive run command had dropped and came right back, and now it is doing it every few seconds. They open the door and it starting working right. Well the damn thing didn't cool off that fast.

So, we put skinny little ole John inside the panel with flashlight and a mirror so he could watch the drive run LED, then carefully closed the doors to a barely ajar postiion so we could see him and, sure enough it was dropping out.

So, I checked the wire and sure enough it was tighter than necessary at both ends, but the terminal block in the middle no one cared to look for was completely backed out.

The weight of the open door flexed the panel enough to push the wire into good contact in the terminal block until you closed the doors, and it relaxed to a loose position sometimes.

Surely a very tough and time consuming job.. Just read your entire post and I think I need to hire an expert person for this.. It might consume less time.. But thanks for sharing information
Click to expand...
The information really helped so thanks again..
 
Same thing as Steve . An old servo drive on a feeder with a hit here X on it.

Had a stamping press locked on bottom. I had just sent the forklift driver after my. Tool boxes and gang boxes. We where looking at a couple of days teardown and rebuild because th operator missed a few numbers. The boss said he was going tk use his lucky finger and press the button one more time before we locked the machine out and started to tear it apart. His lucky finger worked. The press rolled around tk the top and stopped just like it was supposed to do.
 

Similar Topics

TConnolly
phasma intra machina redux
Yesterday Otis came up to my office asking for help with a small assembly press that he had been trying to troubleshoot for a couple of hours...
Replies
1
Views
1,254
dbh6
dbh6
Back
Top Bottom