ruairicussen
Member
Hello,
I inherited a control system one of my predecessors thought it was a good idea to put logic for cant optimization and Kinetix motion control (infeed table positioning pins)in new ControlLogix PAC, and retain existing SLC 505 for the bulk of machine functions. The Status of machine input devices, and actual control of IO to pull in motor starters, fire valves, etc.. are all in the SLC. New operator console is to FlexIO ControlLogix. Not something i would have ever done, and maybe the only answer is to basically finish the project by bringing all IO into CLX and deleting SLC. I do not have a lot of experience with MSG instructions, maybe there is a way to do better with the above "System Design".
Issues are extreme delay in response to operator pressing buttons on new operator console (CLX I/O) for the machine functions (SLC 505 IO) to respond.
Physical Network Layer: a while back i replaced all of the physical plantwide control network with new Fiber/CAT6 installed by reputable contractor and test reports of all cables and fiber. I use these guys for all network projects on the OT and IT side. The switches in this star topology network are 1783-HMS16T4C AB Stratix 5400. Connections between hubs in the network topology are 10Gb fiber. No copper runs are longer than 150', the CLX and SLC involved in this system are both CAT6 runs to the same Stratix, both within 50' of the Stratix. Both cables tested 1Gb. The SLC appears to be a 10 Half Duplex device, based on that is the connection the Stratix web interface shows (i am not currently on site with these devices, but in the Logix 500 controller properties i see the ethernet port but no config other than BOOTP/Static, IP, Subnet Mask, Gateway). It also shows flashing orange "Faulty Link" intermittently in the Stratix web interface, but as far as i can tell from observing these Stratix web pages, all SLC, PLC5, and Red Lion touchscreens show the same, even though those cables test the same as all the 10/100 Full Duplex devices (primarily 1756-EN2TR and Softing 1756-EAtm in-chassis SQL appliances). I have high confidence in the physical layer.
I do not have high confidence in my own expertise with MSG instructions. I have used them a bit, but mostly was using other vendors hardware until AB got to CLX hardware, so that is mostly what i have worked with. The message trigger logic in the program seemed like it might be a little weird, so i did rewrite everything and shorten the .UnconnectedMessageTimeout for the MSG instructions. None of this made any differnece to the underlying issues, though i was able to mitigate some of the symptoms. When i started there were about 50 times per hour where MSGs would error and comms would stop for > 5 seconds. Now it is about 2 times per hour for about 2 seconds. I think better sequential calling of MSGs accounts for the less errors and the shorter timeouts help with duration.
Hoping for the usual genius suggestions here. Attachment sizes are too small to attach programs, so i will attach some screenshots.
Attachments:
Msg Control Logic 1
and
Msg Control Logic 2
These are all the rungs controlling the only 2 MSG instruction in the CLX.
Msg30SecondTrend
This is a 30 second trend, top 2 trendlines are Read MSG and Write MSG .EN bits, bottom 2 are Read MSG and Write MSG .ER bits. I had to wait a few minutes to get the error, but when it happens it is always shaped like this. Write MSG errors, then Read MSG errors, then write MSG errors again, then things go back to normal. With different overall Message control timer preset, and Read and Write LIM settings, i can make different shaped errors. but always there are MSGs erroring every few minutes.
Thank You in advance for any pointers on this.
Ruairi
I inherited a control system one of my predecessors thought it was a good idea to put logic for cant optimization and Kinetix motion control (infeed table positioning pins)in new ControlLogix PAC, and retain existing SLC 505 for the bulk of machine functions. The Status of machine input devices, and actual control of IO to pull in motor starters, fire valves, etc.. are all in the SLC. New operator console is to FlexIO ControlLogix. Not something i would have ever done, and maybe the only answer is to basically finish the project by bringing all IO into CLX and deleting SLC. I do not have a lot of experience with MSG instructions, maybe there is a way to do better with the above "System Design".
Issues are extreme delay in response to operator pressing buttons on new operator console (CLX I/O) for the machine functions (SLC 505 IO) to respond.
Physical Network Layer: a while back i replaced all of the physical plantwide control network with new Fiber/CAT6 installed by reputable contractor and test reports of all cables and fiber. I use these guys for all network projects on the OT and IT side. The switches in this star topology network are 1783-HMS16T4C AB Stratix 5400. Connections between hubs in the network topology are 10Gb fiber. No copper runs are longer than 150', the CLX and SLC involved in this system are both CAT6 runs to the same Stratix, both within 50' of the Stratix. Both cables tested 1Gb. The SLC appears to be a 10 Half Duplex device, based on that is the connection the Stratix web interface shows (i am not currently on site with these devices, but in the Logix 500 controller properties i see the ethernet port but no config other than BOOTP/Static, IP, Subnet Mask, Gateway). It also shows flashing orange "Faulty Link" intermittently in the Stratix web interface, but as far as i can tell from observing these Stratix web pages, all SLC, PLC5, and Red Lion touchscreens show the same, even though those cables test the same as all the 10/100 Full Duplex devices (primarily 1756-EN2TR and Softing 1756-EAtm in-chassis SQL appliances). I have high confidence in the physical layer.
I do not have high confidence in my own expertise with MSG instructions. I have used them a bit, but mostly was using other vendors hardware until AB got to CLX hardware, so that is mostly what i have worked with. The message trigger logic in the program seemed like it might be a little weird, so i did rewrite everything and shorten the .UnconnectedMessageTimeout for the MSG instructions. None of this made any differnece to the underlying issues, though i was able to mitigate some of the symptoms. When i started there were about 50 times per hour where MSGs would error and comms would stop for > 5 seconds. Now it is about 2 times per hour for about 2 seconds. I think better sequential calling of MSGs accounts for the less errors and the shorter timeouts help with duration.
Hoping for the usual genius suggestions here. Attachment sizes are too small to attach programs, so i will attach some screenshots.
Attachments:
Msg Control Logic 1
and
Msg Control Logic 2
These are all the rungs controlling the only 2 MSG instruction in the CLX.
Msg30SecondTrend
This is a 30 second trend, top 2 trendlines are Read MSG and Write MSG .EN bits, bottom 2 are Read MSG and Write MSG .ER bits. I had to wait a few minutes to get the error, but when it happens it is always shaped like this. Write MSG errors, then Read MSG errors, then write MSG errors again, then things go back to normal. With different overall Message control timer preset, and Read and Write LIM settings, i can make different shaped errors. but always there are MSGs erroring every few minutes.
Thank You in advance for any pointers on this.
Ruairi