Worst thing to ever happen!!

worked in the railways years ago new coaches with automatic doors on the trains first run with passengers a fault devoloped with a set of contacts on a rotary sw no problem I said I have a spare switch I can install before Dublin, Installed switch Train arrived in Dublin no doors worked 400 passengers with bags had to exit the train through the guards van, the spare switch was a different model I had not checked the full number last diget different
 
One of my fondest memories is of a chiller plant that I worked on back in the day. We were the SI, they were the customer and we maintained their logic... even more fun because my company was the second SI, they had canned the first.

Anyway. Plant had 8 chillers, a mix of electric and steam. They were in two rows of four down the room, very impressive. These things were the size of locomotives and supplied all HVAC cooling water for a large campus of buildings. The chillers were each controlled by... hm... SLC 5/03's and they were networked back to a PLC 5/40 (data concentrator) and a fairly impressive (for the day) Intellution SCADA. Windows NT, which gives you some idea how long ago this was.

So, the network worked for the HMI but not to remote program the SLCs, which had to be worked with one at a time by taking your laptop out to each chiller. Well, surely I could improve the situation! So I networked the SLCs back to the control room PC, and now we could remote access the chiller PLCs in air conditioned quiet comfort.

Finally, a programming change needed to be made. We wanted to work on the chillers on the left side of the room, so the load was moved to the four right side chillers, everything was rebalanced and ready, and it was my big momement. No more walking around from chiller to chiller and taking them up and down. Finally, I would show the customer how REAL control engineers operated!

So, turning to the PC I confidently configured all of the even-numbered chillers on the left side of the room for shut down. I pressed the button.

Of course, you know what happened... I shut down every chiller on the RIGHT side of the room and there was NO cooling water flowing out of the plant.

Yeah, that was quite a scramble. Fortunately the customer's maintenance techs and engineers were amused (later) and I corrected my data tables. But, that one remains memorable quite a few years later.
 
My worst moment was find that the breaker on a 100-HP ventilation fan had tripped. Rather than go get a meter and verify everything was OK. I just reset the breaker. Nope, the motor was shorted. This time the breaker didn't trip fast enough. Fortunately, the one the feeding the entire facility did. Left everyone in the dark calling on the radio trying to figure out what happened. :oops:

I was working with a coworker on a network for a 1000-bed hospital. The network was used for all patient and treatment records. We were supposed to move one of the network nodes. I get there and see all kinds of blinking lights. I go to make a phone call because it's obvious (to me anyway) that the network is still alive. At which point my coworker grabs his dikes and proceeds to cut the thick Ethernet Coax backbone for the hospital as he proclaims "It's supposed to off already." It didn't take 3 minutes for the IT guys to appear and start screaming. :whistle:

Least spectacular but most expensive was working on a cryogenic-cooled CCD. I didn't ensure the seal in the vacuum chamber was properly seated. Started pumping it down. Pfft. The seal fails. $12K CCD ruined. We couldn't even find the damage using a microscope. :cry:
 
Wrong voltage light bulb and improper (if any!)output fusing pulled down the control power supply...Is that it?
 
I was lucky enough to get our team of engineers & specialist a long waiting 3 day site stand down.

System upgrades were being done to low temperature separator gas plant. New refrigerant gas separator unit, control system upgrade to Delta-V, and new 6 stream gas chromatograph analyzer system, etc. I was responsible for the analyzer system integration. The analyzer's results were data mapped to a new Network Access Unit (NAU) which would communicate to the Delta-V via RS-485. The communication was simply not working as it should, so I made comms setting changes and saved the program to flash. I then proceed to power cycle the NAU so the new setting would take effect. That's when all Hell Broke Loose... All the sudden the very loud sound of the whole gas plant Going Down... Operator's / Technicians scramble to get the plant back up... I looked around and made the statement "Did I Do That"??? Everyone in the room looked at me "Naw it couldn't have been you".

After about 15min. operations had the plant back online and I continued my troubleshooting with the NAU. After making some additional changes in the communication setting, I proceeded again to power cycle the NAU... and you guessed it... The plant went Down, but this time operations could not get it recovered and 3 plant flares were now playing there part of the Plant Shutdown... Biggest Flaring Event I've ever seen, not just gas flaring but liquefied gas pouring out of the flare systems... talk about a heat wave and we were about a mile away.

Time forward, Plant investigators were now onsite to find the cause of the plant shutdown. Investigation led to the UPS cabinet that the new instruments, Delta-V, and my good ole NAU were powered from. Electrician found some loose neutral wires, which they tightened. Load test was performed on the UPS - 23% load, so good there... Everything seemed to check out OK. Operation was given the OK to startup the plant, which they did with no event.

So far I am still in the clear as no one has pin pointed the issue, except that the shutdown had something to do with the UPS system? This time around I had Techs helping me troubleshoot the comms problem, check wiring, loop checks, NAU power supply voltages/current draw, etc. It all checked out. This time around I ran a temporary signal cable to a newly configured Delta-V RS-485 comm. port for testing purpose and proceed to cycle the power on the NAU. This time though I let Everyone in the room know what I'm doing... and sure enough THE PLANT GOES DOWN... D@mn it not again.

Our project team (7 of us) are told that we are in Stand Down until further notice, but we had to show up onsite every day and twiddle our thumbs while the incident was being investigated. Once we have the OK to continue working, I noticed the NAU had no power and it had its power wiring completely removed from the UPS panel. I asked the Electrician about power for the NAU and he stated get an extension cord and plug is in the wall outlet over there...

To this day the NAU is still being powered by the extension cord plug into the wall outlet... and there is a LOTO on the UPS cabinet with RED WARNING tape plastered all over the Cabinet. Oh,,, and the MAIN cause of the NAU communications problem was the ribbon cable from the NAU mainboard to the 9 DSUB connector being off by one... This was supposedly checked and tested by the manufacturer..... Geesh I don't think anyone there involved was willing to tell "The Big Wigs" about the bad ribbon cable.
 
I used to work in a factory as a final test tech for Gas Chromatographs. That was where I learned that braided heater insulation was leaky. I was reinstalling a heater and was in a pinch so I had started the unit back up so it would heat while I was doing wire ties to tidy up. I accidentally touched the heater wire with one finger trying to get the wire tie around the wire bundle. European 208VAC hurts really bad running from one arm to the other.

Not the worst however.

My worst was adjusting a hydraulic bypass on a filter press. I had pulled the bypass out of the return loop and swung it off to the side so I could adjust it once the system was repressurized. I pressed the prepump extend button which started the electric pump to fast close the press. About the time it hit 200psi I heard a pop and was immediately covered head to toe with warm hydraulic fluid. Apparently I had forgotten to retighten one of the fittings and it popped the bypass loose. It didn't take me long to hit the stop, but it had already dumped about 5 gallons before I could reach it, most of it on me. Luckily I had a spare change of clothes and a nearby shower. Needless to say that pair of clothes didn't make it past the dumpster on the way out. I think it took me about a week to get all of the oil out of my hair. On the plus side my leather boots were much more waterproof after that.
 
I always try to learn from mistakes but one mistake happens always

Trouble shooting a PLC program. After 20 minutes I figure out, I am not online with the processor. How dumb am I?? lol
 
I tripped a sub-station..

bad E-Stop logic that skipped the sequencing-on routine when the button is pulled.

Sometime, I do not miss being on a SI.
 
Write the "complete" logic portion of a safety interlock for a device being added to a machine, with the exception of the contact that will disable whatever the main machine is.

this one will damage shiny new equipment real quick if your not by an e stop.
 
I had to replace a broken receptacle that was fed from a panel that also fed the scanner computers for a small log line in our sawmill. The breaker wasn't labeled and I figured I could just short the circuit with a wire to pop the breaker and narrow it down. Well, the receptacle breaker didn't trip, but the main that fed the panel tripped, shutting down the computers. The problem was worse because I had no idea where this panel was fed from. Turns out it was a sub panel from one in the basement.

Really not that serious, but downtime is in the thousands of dollars per minute.
 

Similar Topics

Ran a firmware update on a datalogger and this is the horrific splash screen it showed when the update is running. Genuinely thought I'd bricked...
Replies
8
Views
1,252
Posted just because it's has an Allen Bradley keyed selector switch https://www.youtube.com/watch?v=YeFevEGoPF0 LockPickingLawyer
Replies
6
Views
2,676
Can anyone show any worse example for this indescribably tacky marketing approach for industrial components? Control Techniques make GREAT drives...
Replies
1
Views
3,742
For years I believed that Siemens Simatic Basic panels where the worst panels ever with no reason to exist. That was until I was forced to work...
Replies
8
Views
3,128
Hey folks; I have been a long time lurker and an occassional poster and was curious if you think that PLC.net has got to be the best or worst...
Replies
17
Views
9,754
Back
Top Bottom