Ignition Cloud/VPN Network

[matt303]

I would like to install ignition in the cloud to access remote pumping stations and a few other IIOT applications. I’m having trouble figuring out a suitable VPN infrastructure that will allow access to the PLCs via a gateway. The PLCs share the same IPs so the gateway will have to have NAT or something. Then I will have to try to address the PLCs IPs in Ignition, I don’t know how to get around this since the PLCs will share the same IP. I am using s7-1200 and there is no way to change ports and can’t do port translations or anything. I need full VPN access to the Local network for maintenance and accessing PLC/VSD as the sites are all over the country. If anyone has any guidance or done something similar and would like to share their ideas as I’m stuck on how to achieve this.

 

[V0N_hydro]

I assign /24 subnets according to the year and then sequentially. The first project I did in 2018 gets 192.168.180.0/24, the next 2018 project is 192.168.181.0/24, the first project in 2019 is 192.168.190.0/24. This way if I am VPN connected to multiple projects, or a customer does what you are doing and creates a central SCADA that connects multiple sites, there are no IP address conflicts.


It is also important when I am assigning the network for the project to check with the customer to make sure they aren't using that network elsewhere.


one several hundred million dollar project I worked on just one small part of it had to re-address two whole plants because the plants used 10.0.0.0/8 and the customer was also using that on their corporate networks and wanted some connections between historians.

The simplest most straightforward thing you can do is take the time and effort to re-address each of your plants so that they can all be connected to a central location. I imagine it could take a year to get an outage at each plant.


Assuming that is unpalatable you are going to have to do NAT. If you only have a handful of devices at each plant you can just do a 1:1 IP map. If there are more than a dozen devices, or you want to be able to add devices without messing with the firewall configurations, you will have to do a NAT for the whole subnet. eg map 10.0.2.0/24 to the 192.168.5.0/24 network behind the VPN gateway 192.168.10.2.



You will have to spend a lot of time documenting what you have done so the next person has a chance of figuring it out when it stops working, which is why the simpler option of just re-addressing the plants and enforcing a rule going forwards that plants must use unique IPs would be my preferred choice.



I would say IT grade products like firewalls have a lifespan of about 10 years so if you do go for the fancy firewall NAT setup consider that you or somebody else will have to be re-building it in a decade.


I can't think of a decent industrial firewall that I would recommend. Probably a cisco industrial product like catalyst and getting an IT specialist to configure it and document the configuration will be your best bet.

 

[matt303]

That's a really good idea of naming the IP addresses.

These are just small Irrigation pump stations that already setup running using s7-1200 PLCs and Power electronic VSDs, The happen to already share the same configurations across all the sites. (same local IP address)

If I was installing the sites myself I would just have a different subnet per site, but that's not possible in this case.

All these sites are all around New Zealand and my plan was to ship pre-configured gateway to each customer to plug into the network and go.

The Gateways are Industrial raspberry pis I have had made using compute modules. So all firewall, NAT, routing will be done via IP tables.

The biggest issue is the VPN configuration since all sites will be connected 24/7 and the local subnets of all the sites share the same subnet. So the server and Ignition gateway will need some sort of NAT/Routing to use the VPN IP as the access to the local 192.168.5.0/24 network. and not pick up the wrong network.

I don't think 1:1 NAT would work since each VPN client VPN IP is on the same subnet. That would require a new VPN Tunnel/Interface on the server per site which would not scale well when you have hundreds of sites connected.

I'm looking at using wireguard VPN over openVPN, in my tests wireguard was twice the speed. The problem with wireguard on windows is you can only have one tunnel running at a time so all the clients have an IP address from that subnet.

So that possibly leaves me with OpenVPN and has a subnet per site so I can do 1:1 NAT, I was under the impression this is not a good way to use VPN as It does not scale well would need hundreds of VPN servers running at the same time.

 

[V0N_hydro]

at each pump site you'll create a new subnet, eg 10.0.X.0/24 and configure ip tables to do NAT between 10.0.X.0/24 and 192.168.5.0/24.


on your central server you'll setup a route to 10.0.X.0/24 via the VPN. Ignition will be configured to read each PLC at 10.0.X.2 which will be routed to the appropriate site and translated to 192.168.5.2.


I haven't used wireguard or open VPN so can't comment on what they can or can't do. I wouldn't have thought hundreds of VPN connections would be a problem for a modern server - how much ram and CPU can each one use if all it is doing is reading a PLC once a second - so maybe the transfer rates are 1 kbyte/s tops?



I would personally use a commercial product VPN appliance that can support 30-60 VPN connections per box and a VPN client device at each pump station that can do the subnet NAT. That way this set up won't be your baby for life, when you get bored of upgrading linux kernels, find a new job, or have another 20 projects under your belt you won't be getting calls from whoever has taken over support of a home-brew ignition VPN network cobbled together with IP tables and open vpn.



It's awesome that you can make it work with open source software and practically free hardware, but every time i've deployed a linux server in a remote plant that had nobody around who knew linux it was a major pain in the *** eventually, even moreso when I moved jobs and then the poor folks were left with all this **** they didn't understand. whereas if i'd used an off the shelf product there would have been a manual, a support line, other people familiar with the product, a newer drop-in replacement compatible version of the product when the old ones start dying (nothing lasts forever) or a new site is added that they need to integrate.



I also understand you might have budget constraints.

 

[AlThePal]

Why don't you set up NAT using port numbers? You would then connect to the first PLC using 192.168.10.2:2000, the second PLC at the same site using 192.168.10.2:2001 etc. The first PLC at the next site would be 192.168.10.3:2000 and so on. These are the addresses you would use in Ignition, which has the advantage that it's easy to see which site you're dealing with.

Your VPN will send all traffic addressed to each IP to the correct site automatically. IPtables on the Pi would then NAT the address and port combination to the correct final IP and port destination.

 

[matt303]

That's what I'm currently doing and works fine, but the issue I now have is where we have more than one plc at the site. The S7-1200 uses port 102 and you can't change the port numbers on the PLC so I need to do 1:1 NAT or something

 

[AlThePal]

You can use any destination port and then forward each one to port 102 on a different IP address e.g. forward 192.168.10.2:2000 to 192.168.5.2:102, 192.168.10.2:2001 to 192.168.5.3:102 and so on.

 

[matt303]

I didn't think about doing it that way. That will probably work just fine, I will try to do some tests today and report back. With the siemens driver in Ignition can you assign an IP address and port number? I'm new to Ignition so not quite sure. I know with other SCADA software I use I can't set ports for the siemens driver

 

[the_msp]

Yes, you can. In the device settings, click 'show advanced properties' at the bottom and the port edit field is exposed.