ETHERNET. Process network vs office network.

[idell]

Hello dear experts. Now I'm leading a contradictory discussion with main IT administrator of company that we developed an automation system, about network topology. The system has 5 Ethernet nodes - 2 PLC's and 3 Operator panels. My opinion is that the automation system should be in a different subnets than company network, but admin insists to put in the same subnet with all company's computers (including office PC's) and devices. As an automation engineer i meet difficulties to explain to IT admin my position, but my "engineer flair" tells me that I'm right. Could you share your experience about this?

 

[bce123]

try searching this: stuxnet That should be reason enough

 

[Ken Roach]

In my opinion, the automation network should be on a separate physical network, not just a separate logical network !

The logical addressing of the automation system really depends on who is responsible for maintaining and administering it. If the IT guys want to be on duty when the production foreman calls at 3AM because a PLC quit communicating, then they can administer the automation network.

There's also the issue of performance and network loading: can your automation network handle the changes in performance when somebody starts streaming video through the office network, or begins a network backup on a server, and loads down the Ethernet switch ?

The security threat is overestimated for many enterprises; an enemy hacker probably doesn't want to disrupt operations at my french fry factory. But if you're running a power plant, or treating water, or processing anything hazardous, there is probably an industry association pushing you to improve your network security, including that of the automation system.

There are many dozens of whitepapers and studies on this topic and all of them agree that you should have some degree of separation between the process network and the enterprise network.

Some examples:

http://www.automationworld.com/images/sponsored_content/wp-citect-scada.pdf

http://odva.org/Portals/0/Library/P...00269R0_ODVA_Securing_EtherNetIP_Networks.pdf

 

[Saffa]

i don't get on well at all with IT departments - and for exactly the reason you're seeing above. Just because it uses CAT5 doesn't mean that it's an Ethernet network like they're used to.

I always try and seperate control networks from business networks. Seperate routers and hardware preferably. There should only be one connection point between the business network and the control system network, which should be documented and tightly controlled. You don't want to inhibit productivity and people making use of their control system, but you certainly want to protect it.

I would have a browse through this document, section 5.2 :http://csrc.nist.gov/publications/nistpubs/800-82/SP800-82-final.pdf

 

[idell]

Thank to all for answers, as you mentioned, I firstly proposed physically separate network by router, but at start my idea was banned, and it is because main staff don't realize that this is not an usual home or office network. Unfortunately there is no exist automation department. Thank you also with links I compulsory will read all of them to get armed with arguments.

PS. Also they ask the possibility to change IP addresses of devices from the SCADA. The CPUs, we use, are from SIEMENS and they only this year have announced some new CPUs with feature to change IP from program, and OPs also are from SIEMENS and there are no chances to change from runtime. I've sent them these answers, but I think they will not believe me and think that don't do the work. Want to say that process part of system works were well til now.

 

[DamianInRochester]

I did a small system for a local company that had 4 nodes on the control network. When we went to start up the system, we were informed that no network switches were allowed in the plant other than the main network switch. Apparently a while back someone had plugged the wrong cable into the wrong port and for whatever reason caused the whole system to be down for days before they figured it out. From that point forward, it was decided that all connections go back to the main hub. So they literally took all the nodes on my panel and routed a separate cable each all the way back to the main server. I was incredulous, and nobody would even put it up for discussion. I hinted that it sounded as though the real problem was the people in charge of maintaining and diagnosing the network. That wasn't received well either.

I completely agree. At a minimum a logical separation should be there. Even better a physical separation running through a gateway. I don't understand why just because it is ethernet based they feel their IT department needs to take control of it. Almost makes me consider reverting to a non-ethernet based fieldbus. You don't see the IT dept fighting over control of your Devicenet or Canbus networks.

 

[Johnster]

+1 on the Stuxnet - is there a "Flame" thread here yet ?

Power plants have a federal group ensuring that thier corporate network is seperate than their IT network - its called NERC in the US.

I would try this - go to higher authority (the IT guy's boss, or the company VP if he is not in IT), explain the situation and offer to enter into a verbal or written agreement about support and responsibility (example - you dont get called if email doesn't work, and he doesnt call if the PLC doesnt work. You pay to repair/replace PLC's that break and he pays to repair computers that break...) - having some kind of agreement about support before problems happen will quiet a lot of concern.

The seperation you are looking for is common practice by any wise person that has a PLC.

-John

 

[The Plc Kid]

With some people you have to just give them what they ask for to teach a lesson.

I did this in my plant when I first started here. I let them connect the automation network to the corporate and any PLC,HMI,Data Collection,Drive Comms issue they were called in on at all times of the night and weekends.

Before I started using managed switch's everywhere it was a real mess that I inherited with lots of IP multicast that took down the corporate LAN and caused some serious WAN problems also.

This went on for 7 weeks and the the IT manager was almost on his knees begging me to take it all back. The plant manager,production manager,engineering, and the CEO was ripping him a new one on a daily basis.

Also as a machine builder or OEM I would have in my contract that it be on it's own network with it's own switch,etc and only have a connection at 1 point that is tightly managed in a DMZ or the machine /controls would not be under warranty period. Customers that refuse to research and understand that you are looking out for their and your best intrest is the type of customer you do not want IMHO.

Give them what they want. let them learn the hard way if they won't listen.

 

[GeoffC]

It seems it is the sight of an ethernet switch that get IT departments all hot and bothered. Interestingly the latest plcs and drives from Schneider come with built in 2 port or 4 port switches. The 4 port plc module even manages a ring topology. So you could hook up a whole network of devices without any switches. I would be surprised if anyone from IT would want to administer a plc module

 

[The Plc Kid]

It seems it is the sight of an ethernet switch that get IT departments all hot and bothered. Interestingly the latest plcs and drives from Schneider come with built in 2 port or 4 port switches. The 4 port plc module even manages a ring topology. So you could hook up a whole network of devices without any switches. I would be surprised if anyone from IT would want to administer a plc module

GeoffC

It has been my experience that all it takes is for them to see CAT 5 cable or even hear the word network.

 

[PeterMalt]

Ik work in a small factory where the "industrial" network is complete seperate form other office networks.

This network is only connected bij a cisco firwewall to communicate to MES systems, and VPN connections.
( this means also that in my Siemens case port 102 is blocked)

Industrial networks have nothing to do with office networks, the use the same cable. But a hedgetrimmer and PC have both a power cable so we should connect them togetter ?

First don't speak to IT department over a network but only use Industrial Net.


In the past a haven worked in a compagny where the IT department overnight shedule to update all computers in the network with new software. The Wincc servers and clients where just formatted and a new Xp-Sp3 where installed.
Cost where 3 day's production los, due the restore of all data!

 

[idell]

Another confusing situation, I think it was the mistake when people signed the contract, one of condition is that client have to provide computer for SCADA. this condition makes some "black" days for me, hardware of computer is quite enough for system, but they give no one administration rights in Windows, can you imagine a SCADA working on operational system with only user rights? And when I had to install the software they refused to give rights and stayed near me, and put password when each time system asked.

And one more thing that I firm stay to position - do not install any software on PC with SCADA, because they strongly want to do usual office work to with this computer also.

 

[BobB]

I have had several issues with IT departments - at Sydney Airport I made the PLC and SCADA addresses for the generator system all in the 10.10 area - IT chucked a fit. They then fiddled with the SCADA computer that was still under warranty and I found alarms from all over the airport appearing in the dedicated SCADA for the generators - immediately withdrew warranty - have not been invited back since.
A friend of mine is in charge of all programming and SCADA at a large hospital. He had to use the hospital network for all his communications with the PLCs and SCADA. One weekend IT came in and changed all the router addresses - the SCADA could no longer find the PLCs. The PLCs even monitor the temperature in the refridgerators so that if the temperature goes high (someone left a door open) they get an alarm at the nearest nurses station and go and clean the fridges out so no one gets food poisoning. Because the SCADA could not reach the PLCs no alarms were raised. Some people got food poisoning. Needless to say there was a huge investigation and the blame was placed fairly and squarely on IT. IT have now put in a separate network for the PLCs and SCADA and it is administered by engineering only.

 

[DamianInRochester]

but they give no one administration rights in Windows, can you imagine a SCADA working on operational system with only user rights? And when I had to install the software they refused to give rights and stayed near me, and put password when each time system asked.

I went through the same thing. Part of the problem was that the 2nd octet of the IP addresses they procured for me were wrong. Then out of those 4 IP addresses, one of them was already being use somewhere else. I couldn't even "ping" without them being there. Then even after that, the system would lock up every so often without warning. I finally ended up putting everything back on my switch except for their SCADA and just did the SCADA using serial modbus to remove it from their physical network. Never had a problem after that. I learned my lesson.

 

[DamianInRochester]

Also as a machine builder or OEM I would have in my contract that it be on it's own network with it's own switch,etc and only have a connection at 1 point that is tightly managed in a DMZ or the machine /controls would not be under warranty period.

PLC KID

I really this this suggestion. I am going to start using it just so I have something to back me up.

GeoffC also brings up a great point. Many products now have 2 ports or more with built in switches. Would love to see how IT wants to administer an Ethercat network.

I love using ethernet as a fieldbus but IT can really turn it into a nightmare.

Also,
Has anyone used the E.W.O.N routers? Thinking about trying that out as the single point of contact to the rest of the world as a standard.