A PLC network dilema

Long time viewer, first time poster here.

A little background to me. I'm an Electrical Engineering Technology student that is near my B.S. In all honesty I've only had 1 course regarding PLCs, and only covered AB.

I recently got hired on to a paperboard company as an intern, but am deemed a PLC expert here. I really have no resources to turn to when I have questions regarding this nature(minus this forum which is a HUGE help).

The question as I see it. We currently have 12 PLCs that are all AB (SLC 5/02, Mirco, Compact, and Control) in the mill. Four of the PLCs(Micro, Compact, and Control) are requested to pass data back and forth to a DCS system through Delta V. The system is currently set up for the 5000 family of processors and 1 PLC is passing data to it. The method of data gathering is VERY limited(physical connections, data tags, 500 vs 5000....) and the cost of scaling it sounds very steep.
Alongside this network, upper-management want all new PLCs the ability to provide their OEM's with remote access(currently 3).

Now I've been able to gather that I could get 1 PLC that could gather up all the information requested by the DCS system and send it, while also taking information from the DCS and writing it to the respective PLCs.

I'm wondering is there an easy way to provide what I'm asking for that I might have missed.

I'm getting direction from our distributor to use a CompactLogix as this 'server' with a 9300-ENA to translate over to the DCS domain(physically separate network). I've seen on this forum people say for this purpose, the ControlLogix is a better choice.

I've also seen material on this forum regarding I/O networks and the sheer amount of data that is passed. I've contacted the mills IT department regarding network design and so far I'm striking out. I'm not entirely sure if I need to be concerned about network volume. Do I need to put managed switches in each PLC cabinet to isolate each network? Is it better to have a centralized managed switch? Will data slow down going through multiple managed switches?

I'm looking for pros and cons of proposed systems. I'm trying to formulate costs for each respective design, and propose a solution for a rapidly growing problem.

Regards,
Jeremy

I don't fully follow your architecture or needs, but I'll tell you what I do for network architecture. As a general guideline, keep your I/O traffic on a separate network than your plantwide PLC network. I/O networks need to be troublefree because equipment and safety are at risk and you don't want plantwide networks polluting or compromising its integrity. First time someone connects in with a laptop with a competing IP address (if ethernet) to an I/O module and you suddenly get equipment/production failure.

With that said, organize your PLC's with two networks (whether RIO, DH+, Ethernet). One Network is a Machine network and each subprocess (PLC) will have one that ONLY has I/O, Drives or other intelligent devices that are pertinent to that process. Then, each PLC should be connected to a secondary Plantwide network for passing of data between each other and to supervisory processes or HMI's.

So, for example, you have a Mill ControlLogix 1756 rack that has TWO ENBT (or equivalent) comm modules. One ENBT is physically connected to a local network with isolated switches of ethernet aware I/O, Drives, etc. The second ENBT is connected to a different network along with every other PLC/DCS system in the plant.

You can have a compactlogix or controllogix rack act as just a communications gateway for massaging all the data you need from the various PLCs to the DCS. Depending on how much data you are trying to process, the compactlogix could be sufficient, but I would try to standardize on a platform within the plant and start migrating everything to compactlogix or contrologix depending on need.

I think part of my problem is that I don't fully understand my problem.

Most of the PLCs in the mill are just a PLC and HMI directly connected. I'm seeking to do plenty of standardization.

One item I'd like is that all PLC cabinets to have a flush mounted NEMA 4X Ethernet port + 120V 'receptacle' for diagnostics.

That item said, not sure if internal to all cabinets I should put a unmanaged/managed switch,
or....
if I should get a centralized managed switch to run all traffic through. My gut tells me that doing an unmanaged switch in the cases of PLC + HMI + Diagnostic Port should be fine. Then, make a connection from that unmanaged switch into a managed switch. However, I really don't have enough time/experience to make networking decisions.

In one case, I have remote I/O and drives and managed switches all integral to a ControlLogix platform. Before I was hired here, they had already paid for an equivalent of an ENBT card to get its data to the DCS system.
I start to scratch my head when I realize functionally I want a remote network for each subprocess, a way for that subprocess to be remoted into by OEM, AND a way for that subprocess to send/receive data from a DCS.

What I was really hoping for was suggestions along those lines. From the research I've done, it would seem like I want a way to isolate each subprocess from the plant network logically, while physically having all subprocess connected. On that network, have a Gateway Control/CompactLogix with a NAT or some equivalent to bridge data over to the physically separate Delta V(DCS system) network.

As of the moment, there are only 3 systems demanded into the DCS, and I believe before the year is over that will be 5. As of the moment as well, 3 systems require OEM remote access.

I just see that with the current approach this company is taking, they're going to order a NAT/ENBT for EVERY plc they want into the DCS, on top of putting PLCs directly on the plant network that require OEM access. On top of that, the DCS system is limited to 4 access points per card, and each card is limited to either Logix 500 or Logix 5000 and the amount of tags that can be passed. Maybe I just don't know enough about the costs of networking PLCs correctly, but what I see happening looks like it'll be a huge money pit to continue scaling.

On a side note, while I'm sure they could hire someone in my absence, I've been given direction to NOT design something that is very complex in terms of upkeep. While some employees have 'dabbled' in PLCs here and there, there is an overall impression that even our electricians/instrumentation guys have very limited knowledge of them(However they did manage to get RSLogix 500 V6.?? to connect and recognize a ML 1400, mystery to even me). I get the impression that in my absence, if a PLC problem occurs they'll end up consulting a 3rd party engineering firm.

It would help if you listed each PLC's part number so we know what communication it already has on board. We could make more informed suggestions if we knew that.

How much of an issue is money? I am a proponent of eliminating PLC's where I can. Your 5/02 for example could be replaced by a remote rack from aCLX PLC if that makes sense logically. I would run my whole plant on 1 PLC if I was given a choice... Others would argue against this.

I have a major production line out here that is currently controlled by 3 PLC5's and one CLX. One of my projects next year is to replace the PLC 5's. It will simply make life a whole lot easier.

allscott said:

I would run my whole plant on 1 PLC if I was given a choice... Others would argue against this.

Click to expand...

That statement is HUGELY dependent on your manufacturing process. If you are running something that is 1 piece flow, then yes have 1 PLC with lots of remote I/O can make sense since if 1 part of the plant goes down, everything up and down stream goes down anyway.

My plant is not set up like that. We have 18 seperate but very similar machines. It makes a lot more sense to have each ran independently so you can lock out one for maintenance and have the other 17 up and running. Also, if there is a processor fault, we E-STOP one line costing us $1,000 instead of 18 costing us $18,000. It all depends on the process.

Helliana said:

That statement is HUGELY dependent on your manufacturing process. If you are running something that is 1 piece flow, then yes have 1 PLC with lots of remote I/O can make sense since if 1 part of the plant goes down, everything up and down stream goes down anyway.

My plant is not set up like that. We have 18 seperate but very similar machines. It makes a lot more sense to have each ran independently so you can lock out one for maintenance and have the other 17 up and running. Also, if there is a processor fault, we E-STOP one line costing us $1,000 instead of 18 costing us $18,000. It all depends on the process.

Click to expand...

I agree with you it depends on the situation. However there is no reason you can't lock out individual machines and still keep a PLC running the rest of the plant. If your processor faults your line is probably going to stop anyway. Any yes in your situation 1 processor fault stops one machine, you also have 18 times the chance of a processor failing. With one PLC you could have a redundant processor set up along with a redundant network.

But this is another topic and I don't want to derail the thread. I only brought it up because in the Op's case it MIGHT make sense to eliminate a few processors, it cuts down on the number or processors he needs to figure out how to pull data out of.

Well, first off our DCS system > PLCs. In the direct case of the 5/02's I'm already working on a migration path plan. Never say never, but the OEM's are not around for remote, and I never see any DCS logging of data. As a matter of fact one migration path proposed is to just pull everything into DCS and eliminate the PLCs entirely. Both 5/02's are mirrors of each other running 2 exactly same machines.

That out of the way,
5/02 x2: 1747-L524
ML 1200 x3: 1762-L40AWA and BWA
ML 1100 x2: 1763-L16AWA and BWA
ML 1400 x2: 1766-L32BWA
CompL 5370: 1769-L30ER-A
CompL 5332: 1769-L32E
ContL 5573: 1756-L73

2x 2711-K3A2L1
2x 2711-M3A18L1
2711P-T6C20A and D
2711-T15C4A8
2711-PT6M2OD
2x Red Lion HMI's

The ControlLogix system is the only system with remote I/O, drives, and managed switches all set up on its own remote network. Since time was a crucial element in some decisions, all I've managed to do up to this point is throw unmanaged switches into newer cabinets doing a 'tree' topology. Major downside is any outages upstream cause everything downstream to lose connectivity.

I know for a fact the ControlLogix, CompactLogix 5370, and CompactLogix 5332 are all meant to pass data to DCS. The CompactLogix 5332 is meant to be controlled(have data passed to it) from DCS. I'm sure after certain people realize we can integrate DCS into PLCs in my plant, at least 2 other PLCs will be requested to pass data to DCS.

I know that the ML 1400, CompactLogix 5332, and the other ML 1400 are indefinitely scheduled to have remote OEM access, while the ControlLogix MIGHT be requested as well if we decided to get a service contract with AB.

My initial conversation with our IT department told me that they COULD 'virtually' isolate a physical plant network with their switches, allowing for OEM access while maintaining security by not allowing regular plant traffic on that network. However, they have not gotten back to me regarding when and if they plan to do this. If this becomes the case they'll want all links to go into a switch for easier diagnostics. This will cause on average 3 Ethernet cables per 12 cabinets to be ran to a centralized location.

I'm sure a few of you out there have had experience networking PLCs. I have never had PLCs networked, and before I started at this plant I had never touch an HMI, let alone FactoryTalk or RSLogix 5000.

I can see that if we just keep 'fixing' each immediate issue as it comes up, it'll really add up in a budget. I'm hoping that maybe a little planning and insight could drastically cut costs while increasing overall scalability.

In terms of 'money'. To my knowledge each gateway card for DCS to talk to PLCs cost $2000. The cards only have 4 ports. Once a card is configured, it can only talk to RSLogix 5000 family OR RSLogix 500 family. The first one bought is configured to RSLogix 5000 with 3 ports still available. The gateway then has you 'pack' data into a file architecture like RSLogix 500 with the file O:0, I:1, B3, so on. Each data file you bring in translates to a 'tag' consumed. We currently are limited to 100 'tags'. Expanding tags cost more money(don't know the total yet). IF this option is taken, I'll need to put some form of a NAT device into EVERY system that needs pulled into DCS. Whether its the ENBT or 9300-ENA or whatever else, those cards really start to add up($1000-$2000 each iirc). So I'm thinking as long as the budget stays in the $5000-$18,0000 range, any solution I offer contrary to this one will make my bosses happy.

IMHO it is not good enough to 'virtually' separate or tunnel machine traffic from the company network. It should be physically separated, with one or more Routers or managed switches to make a gateway between the two.

The reason is that you may want to make changes on the machine side, and you dont want to involve the IT department every time.
And on the other side, IT may think that it is wise to do network maintenance after normal office hours, not thinking about that the machines operates 24/7.

Personally I never trust a company IT department when it comes to controls they just dont understand that machinery isn't email and will come to a significant grinding halt when someone pulls out the wrong patch lead! keep plant and business seperate use a dataconcentrator plc with 2 network cards as a bridge.

Cheers

Lee

I'm very glad I came here to ask instead of trusting salesmen, almost always a bad idea.

Unfortunately, I do need a little IT involvement. Based on input from here and my research, get a managed switch to isolate each field PLC to its own network. Run all managed switches to an unmanaged multi-point switch that also has a ControlLogix 1769-L32E data-collector. Run a connection from that unmanaged switch into the plant network and ask for it to be VPN for OEM access. Get a 9300-ENA to translate the L32E to the other physical network.

Has anyone had to setup a permanent outside access plan for a PLC, while controlling the I/O network, and needing to get data from the I/O network to a physically separate network? I fear if the solution would be to get 2 NAT cards for each PLC that the project would end up costing more than the current problematic solution.

Thanks again for all of your input. It really helps to have some direction from people who have actually dealt with these systems.

Outrage said:

Personally I never trust a company IT department when it comes to controls they just dont understand ...

Cheers

Lee

Click to expand...

I always trust them to screw up the system at the worst time/ cause the most pain to me and plan accordingly. Once I get a control PC, I assign it an admin password that is unknown to IT, keep it off their domain and "island" it in it's own work group or if there is enough to justify, setting up a separate server/ domain.

As mentioned earlier, turn off Auto Updates if IT has it set up is a requirement as you don't want an email or Office patch to somehow disable your system.

I do have at least one knowledgeable IT team member that sees thing my way and runs interference for me if corporate wants to issue a "Thou Shalt Do This" edict. He buys into doing this as I take full responsibility for my PCs and he doesn't have to do anything with them except answer an occasional question on something that me and Google can't figure out.

@ PLC 'Wizard'

First lets start fresh. What is your site? Meaning is it a single process or a distributed process or hybrid?

DCS is for single process systems not for sites that have 5 different widget lines, etc.

It sounds like you have mostly rockwell PLC's? if so why keep moving forward with Delta V? Why not go with Plant PAX for your DCS? It's free and has much better integration with Rockwell products.

AS others have stated in control networks you need both logical and physical seperation from the Corporate network. Many people skip over the physical seperation and pay the price from time to time. Do it correctly and have both.

Not quite sure why you are looking to do NAT translation. It may not be needed. Using Contrologix for a data concentrator is a good solution for may cases but likely not for a DCS.

It seems to me you have been doing a lot of reading and have mixed some best practices from different scenarios together.

Answer these questions for us then we can proceed further to help you and give you the best possible answers.

One last tip. Never take advice from sales people as they are alwasy going to advise you to use their product even if it may not be the best fit for your situation.

@ The Plc Kid
I'm an industry novice so please bear with me. It might take a few trys to get the information you are actually wanting out of me.

Site: I'm at a paper-mill with 2 machines producing different types of paperboard. So I think distributed.

Unfortunately, the DCS was here before me. They've sunk a BUNCH of money into it already. From my understanding, DCS has a higher initial cost than PLCs, and that bill has already been paid.

I'm just and intern, so I doubt they'll allow me to build/maintain a network that someone internally wouldn't be able to maintain. Its actually quite the opposite, the projects I work on reduce headcount, not create them. I'm not entirely sure how much is actually in the DCS system. I know I've heard the terms Panther, ISIS, and Delta V regarding the systems. On top of all that, apparently there was only 3 PLCs even in this mill 5 years ago, and 90% of the people who matter don't care for them and wonder why we've increased PLC count 400% over the past 5 years.

Well, if I manage to logically put each PLC system into their own network(C), while physically interconnecting them all to a point(B). Off that point(B) put the controllogix as the concentrator. That's where I'd need the NAT to translate that physical network(B) over to another physical network(Delta V). Also from the point(B) where the controllogix branches from, I need internet/mill/plant networks access(A), or at least a way for IT to route remote OEM access into the secure PLC network. Hell, if the 'secure remote network' resided on the Delta V network I wouldn't care, but I need a way for OEM's to remote into their systems.

A)-----............------(DeltaV)
.........|............|
.......(B)-(CL)-NAT
......./|\
......C(C)C

A) 10.94.xxx.xxx .......................................... ---- Internet
B) to be determined........................................ ---- Centralized PLC network point
C) to be determined, mostly 192.168.xxx.xxx ---- Remote PLC networks, (PLC, HMI, Drives, Remote I/O, etc)
DeltaV) 10.20.0.xxx ...................................... ---- VIM card that acts as a gateway to our DeltaV, passing here for data logging mostly. one system will be controlled through this process)
CL) Control/CompactLogix that is datacollector
NAT) ENBT or 9300 ENA or whatever else.

Hopefully ascii art helps visualize a little of what I'm thinking.

If there are better free systems or whatnot to accomplish this, or ways to eliminate DCS, by all means let me know or point me to resources so I can educate myself to present this information to upper-level management. As it stands this is only a theoretical design standpoint. As stated before, with just the current setup and what is being asked of me, if I could keep this budget under $20,000 I could see it happening. If I can further prove that given the rate of incoming PLCs the cost will continue to scale and whatever proposed solution can scale with it with minimal cost would be an even greater plus.

Ok so you say you have 2 paper machines. I would call that a process. Since it sounds like you have more than 2 plcs. I assume there are multiple small machines that make up paper line 1 and paper line 2?

If this is the case then yes I would put each piece of individual equipment into its own VLAN.

You do need a separate physical network for your manufacturing network. For remote access the IT dept needs to give you a direct connection to there ISP and your own public IP address. They need to give this to you at the ISP DEMARC. This may be a connection to a cable modem or DSL or a bonded T1 or various other connection methods.

Do it like this it would almost be like your equipment is on a separate site.

Now there is likely data that people on the corporate network will need and the way you give that to them is via a DMZ. A DMZ lets you exchange only the specific data needed with the corporate LAN and nothing else. This keeps your production equipment safe.

AS far as the DCS it can be a good idea for a paper mill but I would be moving away from Delta V and go with Plant PAX if you have mostly Rockwell equipment. It's free so much less than the 20K you mention

Also unless you are locked out of a system and can't change IP addresses then you likely won't need to do any NAT. You will need to do routing. These are 2 very different things.