Augmented/Mixed reality in Manufacturing

MitsM83

Member
Join Date
Jan 2017
Location
Indianapolis
Posts
90
Hello all,

I am now working in a Automotive weld shop and am interested in integrating some IAR. I would like to show sensor status and be able to pull manuals and schematics all in AR. Eventually be able to view 3d machine models when repairing or troubleshooting equipment. I have seen some stuff from Microsoft (hololens) and a few others. It is a game changer for industry for sure.

I am wondering if any of you have experience with this or know of any integrators working with this tech.
 
My first thought is...

Would you want to do anything to that machine, or control it in any way. without physically being at the machine and seeing if there are any humans in any danger zone, reaching into the machine, or otherwise could be injured, or worse, if you start, open, close or move anything
 
Hello all,

I am now working in a Automotive weld shop and am interested in integrating some IAR. I would like to show sensor status and be able to pull manuals and schematics all in AR. Eventually be able to view 3d machine models when repairing or troubleshooting equipment. I have seen some stuff from Microsoft (hololens) and a few others. It is a game changer for industry for sure.

I am wondering if any of you have experience with this or know of any integrators working with this tech.

I've seen some systems that use QR codes as links to PDF manuals or work instructions stored either on a server or within an app on a tablet.

Sensor status could potentially be pulled as well. I saw a demo of an app that read in an IP address of a device from a barcode and then did a query for diagnostic status.

I've seen DEMOS of cool VR stuff where you can see a machine, but I've never seen anything actually usable. I look forward to the day, though. I'd be cool to look around, and at a glance see the machine, and the status of any sensor/device/actuator. HEY DON'T PUT YOUR HAND THERE THAT THAT MOTOR IS HOT!
 
Not in my lifetime; probably not in yours

Consider how much effort it takes to get a machine working, with some sort of operator interface attached.

Now double that, and you'll have the effort needed to get some sort of AR functional (not necessarily good) for the same machine.

I did a project 20 years ago where there was a packaging machine with ~100 photoeyes and prox switches, any one of which could hold up the boxing sequence if it failed to read something that was ALMOST in place, or if the sensor got knocked a smidge out of alignment.

I found a "artificial intelligence" program (actually an "expert system" that could ask Yes/No questions, and connect to other systems via DDE), and came up with a methodology to reverse-code the PLC ladder logic into the AI's structure, to figure out "Why has the machine stopped?". When it came up with the answer, it would show a page with the troublesome sensor identified, a layout of the machine where the sensor where it was located, allowed the user to add notes, or say, "No, that's not it, keep trying".

I didn't see the project to completion (layoffs), but it was hitting ~80% accuracy on the first hit, and not all the PLC program had yet been entered.

It was a fun, cool project. But the only reason that it was undertaken was that it required knowledge of the machine that wasn't apparent. If you were to try to troubleshoot it, you might think that the reason that the machine had stopped was that it was waiting for this prox switch to be made. But that prox switch (DI) requires that a solenoid (DO) fire, and it hadn't because it was waiting for something else, which was in turn waiting for something else. Without knowing the relationship between the inputs and outputs, an average sparky would have no chance.

So the ROI was there. Most companies / machines don't have that ROI, and management doesn't care if it takes them a little longer to use their brains to figure out something, rather than building an AR app for a specialized machine whose life-span is only 5-7 years anyway. The maintenance guy is getting paid the same with or without the app, so how do you justify building it?

Consider how slowly "situational awareness" graphics are coming into the factory floor. And that's for normal operation to alert the operator to signs of trouble (i.e., $$$ loss), not for ABnormal operation, which is what the maintenance guy is dealing with.

The QR code thing that mk42 talked about is cheap and easy to implement: the app probably already exists in the Apple Store; the PDFs came from the vendor. AR, not so much. The way you are describing it, it needs to know where in the machine you are. Half the time, I can't even get phone service in the factory, let alone GPS. And having the app recognize which part is which in it?
https://xkcd.com/1425/
 
I agree with Aabeck.

you need an able body there to let you know what is going on!
you just can't remote into a machine from home or the other side of the plant and assume that no one is in the way.

seeing a 3d model is good, it gives you an idea of what the machine is.
but to blindly work on a machine without knowing who or what is in the way is asking for trouble.

machines do what you tell them and it doesn't matter who or what is in the way! they do what they are told, if something or someone is in the way, they move it, bend it, break it, crush it, or move until they can't force the object out of the way.

james
 
The maintenance guy is getting paid the same with or without the app, so how do you justify building it?


Without app takes 5 hours to find fault, with app takes 0.5 hours, = ROI.


However I agree, no remote control e.g. starting robots up remotely.


And keep the graphics KISS. Simple 2D. As far as PDF manuals go, yes good idea, Ignition SCADA can display these in the client.
 
There's a difference between data and information.

Without app takes 5 hours to find fault, with app takes 0.5 hours, = ROI.

Assuming that Bubba isn't sitting around waiting for the next thing to break for the next 4.5 hours.

Does it really take him 4+ hours to find a problem, and only 0.5 to fix it? Can you so the bean counters that the app will find it faster than he will with his current tools without first building the app?

As the app is described -- show the status of a sensor by pointing a camera at it -- will Bubba point to the right sensor, and does knowing what the status of the sensor tell him anything if he hasn't seen how it's used in the ladder? And if he's looking at the ladder, then he already knows the status.

Having information just by looking at something is cool -- but remember that Google Glass wasn't very successful, for much the same reason. Augmented Reality is cool, but it can be as much a distraction as an aid.
 
If the application does not identify what the camera is seeing it is not true AR and I believe that some of those applications simply show data in a viewer without direct relation to what is being viewed.

I have worked a bit with Android AR apps and I think that to identify parts of an industrial installation is really very complicated and requires purchasing library licenses of thousands of dollars a year and some months of work. This makes little sense for a single installation.

But as mk42 commented to identify QR codes instead of the part could work, it is usually considered as real AR and far cheaper to implement.
 
Last edited:
Thanks guys,

I have now opened conversations with a few companies.

Microsoft, Vital, Gravity Jack and Upskil

I will keep you all posted about what they have for me.

As far as what James and Aabeck are saying. This will only be view able at the line. Like you have to be looking at it. So no operating machines you are not at lol. Picture your looking at an existing cell and you can see all the lights on the proxes and photo eyes. Plus any manual you want or even eventually a model of the machine. That is what I am after.
 

Similar Topics

I have a system that is all currently 110 AC. It uses a 1769-L30ER with a 1769-PA2, and a 1769-OA8. I need to add 4 2-wire pressure transducers...
Replies
7
Views
1,555
Hi guys. I have a data type named P75x_IN. It is defined as User data type and it's members are (in this order): IN_BOOL: BOOL IN_DINT1: DINT...
Replies
2
Views
1,879
Hi, all. I seemed to have hit a wall on my decimal to fraction forum. Therefore, anyone know how to separate a number like 17.33333 into 17 and...
Replies
9
Views
4,019
In the process of moving into a new windows7 desktop PC, I rehosted three licenses from my old machine using the existing version of FactoryTalk...
Replies
5
Views
2,908
Hello everybody Ths is my issue : - I want to attribute for each USER a variable in my project and use it when they LOGGED So i used this...
Replies
2
Views
2,912
Back
Top Bottom