Connecting several standardized PLC controlled modules to a common Ethernet network

Imagine, if you will, a series of custom built modules (building size). Each module is setup with standardized control/automation equipment (PLC's, VFDs, HMIs etc.) which communicate to each other, within the module, via an Ethernet based industrial protocol (e.g. EthernetIP, Modbus TCP). As each module uses a standardized configuration for the control/automation equipment, PLC X in Module A will have the same IP address configuration as PLC X in Module B.

Now we install several of these modules at a site and wish to connect them to a centralized monitoring station (control room) with site-wide SCADA HMI and Historian. Since each module has the same configuration (utilizes same IP addresses) simply connecting them to a common network is problematic due to conflicting IP addresses, also configuring unique IP addresses for each Ethernet connected component across all modules is undesirable due to the extra effort, loss of configuration standardization and risk of human error (don't want the PLC in module A accidentally controlling one of the 20 or so VFDs in Module B).

This the part I want feedback on (below)…

It seems to me that a solution is to equip each module with a NAPT (Network Address Port Translation) capable device (e.g. router or NAT/PAT capable switch), we'll call it a router. The router is configured with a unique IP address on the interface attached to the common network and a standardized IP address configured on the interface attached the module network. NAT/PAT rules will be added to the router configuration to allow the common SCADA system to access module equipment (e.g. module PLC).

Can anyone suggest improvements (or point out flows) on this implementation. I'm interested to know if there is more appropriate network technology for this use case (bridges, gateways etc.) and if anyone has specific vendor/model recommendations for the networking equipment.

Also, I want to implement VLANS which extend into the module to allow separation of controls network (described above), data network (for variable computing equipment within the module), and business network.

I would get one managed switch that has NAT built into it for each port and have it do the message routing.
If it was all Allen-Bradley PLCs I would use the 1783-NATR in each module then just connect them all up to a central switch.

JaxGTO said:

I would get one managed switch that has NAT built into it for each port and have it do the message routing.
If it was all Allen-Bradley PLCs I would use the 1783-NATR in each module then just connect them all up to a central switch.

Click to expand...

Thank you, I was not aware of the 1783-NATR and it fits my use case well. I was in the head space of having to find some enterprise or "prosumer" networking hardware to fit the bill (e.g. Cisco or Ubiquiti).

Was/am curious how often the Ubiquiti EdgeMAX products (routers/switches) are used in light industrial implementations... new thread on that subject here:http://www.plctalk.net/qanda/showthread.php?t=119376

You really need to look at "Cisco Routing" They have routers that can be configured to manage multiple networks(all in a single router). Each network can have VPN tunnels that allow communication between each or any combination of networks, they can use NAT/PAT tables, you can create what is called an ACL list, which is a allow/deny service of specific IP and or Ports. They are capable of VLANs

I have similar setup with 7 machines in a
A plant using the 1783-natr in each module with all 7 natrs connected to plant network to scada so can vouch that its does what you want

I posted a similar question on spiceworks (not being sure what community would be most helpful or responsive). The very notion/validity of my use case was shot down over there. I'm guessing this crowd may have more familiarity with industrial automation and communications, while that crowd is likely more business IT centered.

Some of the comments I got:
https://community.spiceworks.com/topic/2190763-how-to-best-connect-several-standardized-modules-to-a-common-network

• "Plain and simple the strategy behind the standardization is fatally flawed... the moment a vendor pulled that **** on me, I would show them the door."
  
• "If a supplier proposed a system to me that had such a complete misunderstanding of how network addresses should work, I would be seriously concerned about what else in the design was totally fu___d up."
  
• "Yes familiar with the AB package bloat on the PLC side... It is a self serving design strategy to increase sales and support costs."

So does this crowd agree/disagree that this design is fatally flawed (or unprofessional in some way) and have some rationale for their perspective?

...Or, is use of a NAT Router like the 1783-NATR perfectly acceptable?

TBennet4u > So does this crowd agree/disagree that this design is fatally flawed

No Sir,
The design is not fatally flawed.
We have three very similar machines (just different in size) but computer-wise and IP
address wise they are identical. Which makes troubleshooting within any one machine
easy as it is the same as on the other two. And one can compare what one sees on
the working machine versus the problematic one.

To allow communications between them there is a Cisco Pix box. Within the machine
the address are 192.168.54.xxx. To communicate with the same box on a different
machine the address is 172.16.70.xxx. (Or .71 or .72 for the other two machines.)
The Pix box converts and routes the message traffic to the appropriate machine and
translates the IP address to its local box.

So one chart of IP address is all I need for the three machines - plus what the correct
172.16.7x.xxx address translation needs to be for each machine.
Poet.

I don't know that the 1783-NATR will help in setting up VLAN's as your first post mentioned you wanted also.

But as I said before you should check out implementing this project with a Cisco Router, they are more than capable doing everything you mentioned.

damica1 said:

I don't know that the 1783-NATR will help in setting up VLAN's as your first post mentioned you wanted also.

But as I said before you should check out implementing this project with a Cisco Router, they are more than capable doing everything you mentioned.

Click to expand...

My thought was to use the 1783-NATR to provide boundary and interface to the local control network (within the module), then connect the public interface of the 1783-NATR to an access port on a managed switch. The managed switch would tag packets on that port for the "controls" VLAN and provide access to the "data" VLAN and "business" VLAN for the module on alternate access ports.

I realize that a NAT capable switch (or bigger router for that matter) could likely handle the functions of both (1783-NATR and managed switch) but right now I like the idea of isolating the control network before the managed switch.

It seems that not many switches are NAT capable (could be wrong), are you suggesting that the NAT occur on a device within the Module (one per module) or on a central device common to all the modules? If you have model recommendations based on my use case I would be curious to know.

As far as Cisco goes, the client is looking for cost-effective options, while I'll likely explore and consider a Cisco solution for inter-module networking, I'll also look at less expensive options. See thread on that topic here: http://www.plctalk.net/qanda/showthread.php?p=806617

I would think this unit would be more that capable of providing you with what you need, and with only a single device.
https://www.amazon.com/Cisco-CISCO881-SEC-K9-Advanced-Services-Router-x/dp/B001N0DQJI

I know in the past I have purchased different devices and spent hours of configuration on each - to end up with a solution, when I could have purchased a single device and just an hour on configuration and be done!! So I have to ask how much does it really cost??

The NATR is really simple to use and does what it says on the box. It is limited to 32 addresses and is quite expensive for what it does. It is also din rail mounted and powered by 12-30VDC.

I imagine you may already use some sort of Ethernet switch in the build of your custom modules.

On your new builds just change this to a switch that supports NAT, so you can translate to a different public IP on the common side of the network - but your configuration of IP's can be the same on the machine or 'module' (private) side of network.

Something like an AB Stratix 5700 1783-BMS10CGN or 1783-BMS20CGN would be suitable.

On existing builds, as above just use NATR device.

Thanks for all the feedback and perspectives. For those interested in the topic I am providing a link for a white paper jointly produced by Cisco and Rockwell Automation. Examples reference the Allen-Bradley Stratix 5700 line of NAT capable switches (utilizes Cisco Catalyst switch architecture and feature set).

Deploying Network Address Translation within a Converged Plantwide Ethernet Architecture
https://literature.rockwellautomation.com/idc/groups/literature/documents/wp/enet-wp036_-en-p.pdf

@brendan.buchan thank you for the feedback and the relevant Stratix 5700 model numbers.