Operators in plants

doomsword

Member
Join Date
Aug 2016
Location
Zagreb
Posts
201
I don't know how much of appropriate topic this is, but I want to know about your experience with operators.
I'm doing automation for one site and today again I heard 'I did not click anything but machine behaved differently' phrase.

Every time something like this happens for some reasons people first come to me and ask to check sequences in PLC which have been working properly for ~100 times (we're in commissioning phase still).
Just to discover that different behavior could've only come from clicking a button, but 'operator says he didn't click anything'.


So, do you have experiences like this and how you handle it?
 
Many times, one particular case was an operator dumped one & a half tons of sauce down the drain, he claimed he did not abort the process and dump it. However we had a Scada system that logged many aspects of the process the ones that caught him out are who was logged on, an abort sequence had been initiated and the drain valve was opened. Get out of that one!, however, I have come across a couple of similar quotes from operators "I didn't touch anything or I only pressed the accept key and true enough they were correct so don't always assume the operator was the cause, in some cases a scenario can exist that causes a problem that has not been thought of in the design, usually a hardware fault although in my 30 odd years pretty rare.
 
I've definitely seen programming errors cause all kinds of mayhem, yet one's first instinct is to blame the operator. Need to do more research to be sure.
 
My opinion is that investing in operators is at least as much important as investing in equipment. Almost never had good experience in plants where operators had lower wages than normal.
Definitely we should try to install loggers and cameras as much as possible. Also important buttons should have confirmation dialog.
 
1. People generally will put blame on things that they don't understand. (The automation system)
2. People will almost never admit there own faults. (This happens with the operators and the programmers)
3. Despite a programmers best efforts to anticipate every possible scenario operators will always find the one scenario you didn't anticipate.

Just my 2 cents
 
With operators, sometimes you need to take what's said or reported with a grain of salt. The actual events may sometimes differ depending on the understanding the operator has of the machine.
I find an operator to be a very important troubleshooting tool. I rely on them to tell me what the machine is supposed to do vs. what its doing at this moment. Especially if its a machine that I haven't looked at in a while.

Then you get the operators hat have anger issues and are very hard on HMI and button equipment. The last broken HMI screen operator I dealt with said, "I just touched it and the back fell apart." When in actuality, he meant he put his fist through it and blew the entire logic module off a 10" PVP7. Only a $6000 KO punch!
 
+1 Lynx777
Make sure you ask the operators for their opinion. Try to get them to help in the process as best you can. It will save you a lot of time in the future.
 
Of course, we found programming errors as well, don't think we hadn't had those. And once we had button malfunctioning, it was like operator was giving opposite command of expected 10-15 times in 20 seconds or so. Button was closing the circuit when it wasn't supposed to do, but I have to admit automation maybe didn't address this situation properly (no time delay filter on that button, that would've filtered most of those false commands).


Beside operators operating differently then we automation guys expected, I still don't understand why people always think of PLC when they see some issues.
 
A company I worked for in the offshore world invested in the development of essentially a black box with playback capabilities. Their black box recorded every single telegram between PLC's on the network and when asked would replay the exact information on the SCADA screens so we could see what buttons were pushed and what data was available to the operator at the time.

The very first case where this investment paid itself off was when an operator did not select landing mode to land a BOP (think of 6 valves weighing about 400 tons) in the bottom of the Ocean. Wellhead and BOP damaged plus the labour lost were in the millions of dollars. He said the system didn't work without knowing the blackbox was recording and he in fact didn't activate landing or was even gentle operating the winch.

I did have one operator that knew more than the maintenance guys... it was a treat to work with him.

Edit: Landing mode was activated by pressing the function button followed by confirm button. He pressed neither.
 
when dealing with operators, you have to be kind of sneaky.

1. ask them was the machine in auto when it quit
2. if in auto, did you switch to manual.
3. what is the auto cycle supposed to do, have them explain it step by step.
4. look at the code and see where the machine stopped.
5. ask several of the same questions with a twist on it.

sooner or later, you eventually get to the truth as to what happened and will be able to find that there is a glitch in the program or what the operator did.

It took 3 hours to find the glitch in a machine while I was watching it.

james
 
My solution to that situation is to have trends of almost anything, this way you know exactly what happen at any time. I called my trends "my liar detector".
I know my software behaves properly but you never know what the operator does.
 
Audit Trail is the buzzword you want.

Most PLC companies have a function or software package that will do it, but you can also build you own.

Any time someone initiates an action on the HMI, it is recorded. Starting a pump, placing equipment in auto or manual, clearing faults, login, logout, changing a recipe, loading a recipe, etc.

You can tie some bits to certain outputs or capture events, then have the corresponding bit move a string describing the event to an array.

  • Operator puts machine in Manual Mode
  • "Manual Mode Initiated By Operator" Bit is set.
  • Use bit to move string "Operator Jim Bob initiated Manual Mode at 09:00AM 6/10/19" into another array.

The length of the audit trail is determined by the length of the array.
 
What I do is data logging. In my particular case, I use Red Lion HMIs, and you can save your tags to the SD card or Compact Flash card.
I can save any action or data value I want.
 
This happens all the time. It has nothing to do with unskilled/ignorant operators are flawless programmers but with human nature. No one wants to be at fault for any reason. The mind set I use and encourage in this type of situation is:
i) what happened?
ii) what can be done to fix it?
iii) what can be done to keep it from happening again?

Note that the word "who" was not used. Finger-pointing or a test of urine distance does not solve the problem.
This approach is critical if you are in a situation where you deal with this "customer" often or even daily. Getting the problem fixed is the goal. Treating people with respect and seeking their help in the resolution and prevention phases will make things easier the next time....and there's always a next time.
 
With operators, sometimes you need to take what's said or reported with a grain of salt. The actual events may sometimes differ depending on the understanding the operator has of the machine.
I find an operator to be a very important troubleshooting tool. I rely on them to tell me what the machine is supposed to do vs. what its doing at this moment. Especially if its a machine that I haven't looked at in a while.

Then you get the operators hat have anger issues and are very hard on HMI and button equipment. The last broken HMI screen operator I dealt with said, "I just touched it and the back fell apart." When in actuality, he meant he put his fist through it and blew the entire logic module off a 10" PVP7. Only a $6000 KO punch!

Sometimes the anger is understandable, never justified though. We had a scrubber system that the OEM put the only alarm silence button screen two or three screens deep with the 92 db horn about 6" from the screen. One operator during startup (starting scrubber was slow and nasty) said he just tapped the silence and the screen cracked. Yea, right. What would you do if the alarms came every 5 or 10 minutes until the system stabilized and the horn was at head level and you had to go through a couple of screens to get some quiet? Well $5k and 3 days off without pay was the result. He didn't have to pay for the screen. But I did change the programming to add a real pushbutton alarm silence and work on smoothing out startup. Much improved after changes. We also requested all new systems have physical alarm silence buttons if there is a loud horn.
 

Similar Topics

Good Evening , I am working on a Panelview Plus application.This is a plant that runs 24 /7. Sometimes operators don't share information ...
Replies
1
Views
1,413
Anyone have an idea of how to create a FIFO que and storage network using only the folowing instructions (and,or,not,xor). I'm working on a...
Replies
17
Views
8,115
Using CoDeSys 2.3 for the first time. I've not been able to use in line comparison operators successfully in Ladder - and haven't found any...
Replies
11
Views
9,080
Have you ever had to go in to a plant/shop with the mission of automating a manual machine and have to learn how the machine operates, from the...
Replies
8
Views
4,125
Hey everyone, Please share stories that you may have of operators, not knowing how to do their job, my most recent one was on a service call...
Replies
44
Views
10,719
Back
Top Bottom