Industrial Ethernet issues

Fellow engineers,

I have been given an assignment to design an industrial Ethernet network and since I don't have any experience in doing so, I signed up on this forum to get your opinion and comments on my current design proposal.

Main requirement is to mitigate any single point of failure influence on overall network performance. The approved (from my supervisors) network topology is the double star topology (network A and B). Other requirements are:
- fiber optic links due to long distances and harsh environment;
- minimal latency;
- minimal or no at all convergence time;

The system consists of 15 subsystems, 6 of which are main and the focus of my concerns. Main systems are based on S7-400H Siemens
PLCs, with two CPs per each CPU (one for every network).

Heart of both networks is the core managed switch.

All of the subsystems are connected to network over dedicated unmanaged switches and media converters (copper to fiber).

Brains of the operation is PC based server (two of them for redundancy) which is managing the following actions:
- Application server (fetching the data from all subsystems, calculating the status of the system and outputting the set points, communication with other servers on the other networks);
- OPC server (Data Access, Historical Data Access, Alarms and Events, serving HMIs...);
- Network Manager Server (SNMP manager, NTP).

The two servers shall be installed in hot redundancy (active mirror) configuration providing hardware and software redundancy.
When standby server detects failure it assumes the active role automatically without any user intervention. The servers must be synchronized and they must ensure bumpless switchover by holding last data value active.

Communication with other networks is done via Demilitarized Zone with router and firewall.

There are two HMIs running on WinCC.

One simulation server that can fetch data from the OPC, other servers or manual input, run the simulations and output results.

PRP (Parallel Redundancy Protocol) is not a option due to the need for uniformity of protocols throughout the whole plant.

UTP will be used for communication between subsystems and the server.


Here are my concerns:
1. My idea is to have all of the critical data exchange to take place simultaneously on both networks. The receiving nodes, on the application layer, evaluate two received packets from two network interfaces, drop one and process the other. By implementing this design topology and data flow paths, in the case of failure of one network component, there are no convergence times associated because there is no reconfiguration of data paths (e.g. RSTP). Also, UDP packet loss is easily detected and coped with. Is this approach feasible?

2. Since core switches are not interconnected, I have basically two independent networks.
172.network.subsystem.node (subnet mask 255.240.0.0)
e.g.
- 172.20.10.101 - network A (20), Subsystem 1 (10), PLC 1 (101);
- 172.30.30.101 - network B (30), Subsystem 3 (30), PLC 1 (101);
- 172.20.100.51 - network A, high-level system (100), HMI 1 (51).

Is this good IP addressing practice? Will I have difficulties with network configuration / routing?

3. Can I connect both core switches to one router?

4. Do you recommend using managed access switches? If yes, why?

5. In the future if I want to install IP cameras (50 pcs) should I make new dedicated network for that? With media server and other components? Or I should make network A capable of handling that traffic (1000 Mbps Ethernet), put managed access switches, implement QoS and VLANs?

In the attachment you'll find network topology schematics.

Thank you in advance! Looking forward to your replies and comments!

Best regards,

mrvlica

I'm not an ethernet expert , but this book ( enet-td001) it will help you for sure:
http://search.rockwellautomation.co...&entqrm=2&ud=1&exclude_apps=1&site=literature

widelto said:

I'm not an ethernet expert , but this book ( enet-td001) it will help you for sure:
http://search.rockwellautomation.co...&entqrm=2&ud=1&exclude_apps=1&site=literature

Click to expand...

Widelto, tnx for the link. Already have the book..

With all due respect you really need some experienced help on a project like that. These items are very expensive and you don't want to get the wrong gear.

Also if not done right you just be building a very expensive playground for me and my friends.

Please enlist some experienced help as this is not an area you want to make a mistake. You could leave your whole company vulnerable.

I agree with PLC Kid.

A redundant PLC and networking system will be expensive. Wether you have to pay or not to have your system confirmed by an expert, it will be a small expense relative to the total cost and the risk.

Since it is based on Siemens S7-400H and WinCC. In the past I have gotten advice from Siemens on a redundant system. They said to my original idea that they had a better alternative, and it turned out to be even less expensive than my own idea. So they do want to just sell you the biggest and most expensive system.

And you can also ask several other experts and vendors.

Just one opinion.Why not go for a ring topology? Should cut alot of costs

chud said:

Just one opinion.Why not go for a ring topology? Should cut alot of costs

Click to expand...

Depends on the distances in the project.

The Plc Kid said:

With all due respect you really need some experienced help on a project like that. These items are very expensive and you don't want to get the wrong gear.

Also if not done right you just be building a very expensive playground for me and my friends.

Please enlist some experienced help as this is not an area you want to make a mistake. You could leave your whole company vulnerable.

Click to expand...

The Plc Kid,

thanks for the comment. Of course, the final design will be validated by experts. I just wanted to clear out my concerns in the current design phase.

JesperMP said:

I agree with PLC Kid.

A redundant PLC and networking system will be expensive. Wether you have to pay or not to have your system confirmed by an expert, it will be a small expense relative to the total cost and the risk.

Since it is based on Siemens S7-400H and WinCC. In the past I have gotten advice from Siemens on a redundant system. They said to my original idea that they had a better alternative, and it turned out to be even less expensive than my own idea. So they do want to just sell you the biggest and most expensive system.

And you can also ask several other experts and vendors.

Click to expand...

JesperMP,

I'll keep that in mind and consult myself with Siemens and other experts. Thanks for your sharing!

chud said:

Just one opinion.Why not go for a ring topology? Should cut alot of costs

Click to expand...

chud,
Subsystems are dislocated (as JesperMP noted) in a way that star topology is the best choice.

We installed a similar system about four years ago, with different brand of PLCs and HMI, but the topology is almost identical. It has been a very robust system, with no data loss issues (cabling and NIC redundancy)and system downtime of only a few minutes total over that time (almost completely due to human error). Our difference is that we have used a totally managed system, with Cisco Catalyst 3750 and 2960 hardware. The 2960s are connected to the central 3750s with fiber, as we have a fairly large total covered area spanning about 5 miles total. We have a Comm group that monitors the sytem, and has it set up to notify key personnel via email, in the event of any anomaly in operation. Having expert personnel design, install, and monitor the system has been the key to our total performance in a mission-critical application.

For your addition of IP cameras, you may want to consider a separate network; depending on the bandwidth requirements, you may see your HMI network crippled, and control is not something you want to sacrifice. Again, having the right knowledge base involved in design and implementation will help keep you from making a very expensive mistake, and potential loss of control and downtime.

mbartoli said:

We installed a similar system about four years ago, with different brand of PLCs and HMI, but the topology is almost identical. It has been a very robust system, with no data loss issues (cabling and NIC redundancy)and system downtime of only a few minutes total over that time (almost completely due to human error). Our difference is that we have used a totally managed system, with Cisco Catalyst 3750 and 2960 hardware. The 2960s are connected to the central 3750s with fiber, as we have a fairly large total covered area spanning about 5 miles total. We have a Comm group that monitors the sytem, and has it set up to notify key personnel via email, in the event of any anomaly in operation. Having expert personnel design, install, and monitor the system has been the key to our total performance in a mission-critical application.

For your addition of IP cameras, you may want to consider a separate network; depending on the bandwidth requirements, you may see your HMI network crippled, and control is not something you want to sacrifice. Again, having the right knowledge base involved in design and implementation will help keep you from making a very expensive mistake, and potential loss of control and downtime.

Click to expand...

Mbartoli, thanks for your comments! I proposed the same hardware so at least now I know I'm on the right path.
For IP cameras I'm considering separate network or separate access switches for this purpose to isolate CCTV traffic.