You are not registered yet. Please click here to register!


 
 
plc storereviewsdownloads
This board is for PLC Related Q&A ONLY. Please DON'T use it for advertising, etc.
 
Try our online PLC Simulator- FREE.  Click here now to try it.

New Here? Please read this important info!!!


Go Back   PLCS.net - Interactive Q & A > PLCS.net - Interactive Q & A > LIVE PLC Questions And Answers

Reply
 
Thread Tools Display Modes
Old July 30th, 2020, 05:01 PM   #16
Ken Roach
Lifetime Supporting Member + Moderator
United States

Ken Roach is offline
 
Ken Roach's Avatar
 
Join Date: Apr 2002
Location: Seattle, WA
Posts: 15,472
In my opinion, there should be a router or firewall between the master radio and the rest of the automation or enterprise network, for exactly this kind of reason.

Once you've got an enterprise network that includes ordinary Windows workstations, or printers, or WiFi access points, or anything with Universal Plug and Play, the amount of multicast and broadcast traffic can be substantial. If you happen to have multicast I/O, it can get huge.

I once got thrown off a job shortly after showing the customer that their printer discovery traffic from Downtown was showing up over the radios at their remote pump stations.

There are certainly commercial routers and firewalls that do this sort of thing, as well as sturdy open-source stuff like PFSense.
  Reply With Quote
Old July 30th, 2020, 05:17 PM   #17
Maxkling
Member
United States

Maxkling is offline
 
Join Date: Mar 2011
Location: Atlanta
Posts: 420
You can also segregate the broadcast domain through VLANs. The problem is at some point you will have to route the traffic if devices on separate VLANs need to talk. You can also place a NAT router on the outbound side of the radios which will also stop multicast traffic and only allow addressed routable traffic.
  Reply With Quote
Old July 30th, 2020, 06:03 PM   #18
Firejo
Member
United States

Firejo is offline
 
Firejo's Avatar
 
Join Date: Jun 2008
Location: Redmond, WA
Posts: 1,428
Quote:
Originally Posted by Ken Roach View Post
In my opinion, there should be a router or firewall between the master radio and the rest of the automation or enterprise network, for exactly this kind of reason.

Once you've got an enterprise network that includes ordinary Windows workstations, or printers, or WiFi access points, or anything with Universal Plug and Play, the amount of multicast and broadcast traffic can be substantial. If you happen to have multicast I/O, it can get huge.

I once got thrown off a job shortly after showing the customer that their printer discovery traffic from Downtown was showing up over the radios at their remote pump stations.

There are certainly commercial routers and firewalls that do this sort of thing, as well as sturdy open-source stuff like PFSense.
Ken is spot on. I sold 900MHz FHSS radios for 20 years and we had an Ethernet version for about 10+ years. That technology has it's pro's and con's with the biggest pro being that it will go miles (with LOS) and the biggest con being that it is very slow as far as Ethernet is concerned. It's slow because of the time it takes for the radio to hop from one channel to another and with that you get a very small bandwidth I.E. very low throughput. That's were you are running into trouble, it might be related to some degree to the speed (or lack of) but more than likely it's the very narrow bandwidth that is causing the problem and is the main reason why you want to isolate the network traffic from the radio traffic. A router or managed switch is a good way to do this however if feasible, a completely separate network would be ideal. Long and short is the only network traffic you want to be accessing the RF network master (or the slaves if they are connected to an extended network) is network traffic with a destination that is on the other side of the RF network. Even then you need to make sure that the "expectations" are appropriate I.E. the sending/receiving devices are not looking for response times of 2ms (most Produce/Consume won't work very well if at all because of this).

Another approach depending on the throughput needs of the network are FHSS radios that aren't FHSS radios. Xetawave is an example of that. They are software defined radios and the guy who started the company was one of the founders of FreeWave and probably the smartest RF Engineer on the planet. If you're not locked into MDS or for future projects you may want to look at these guys (https://www.xetawave.com/). I've not worked with their radios before but I know Johnathan (the founder) and I'm very confident that what they list as the capabilities are real.
__________________
Go Hawks!!!
  Reply With Quote
Old July 30th, 2020, 08:28 PM   #19
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
Quote:
Originally Posted by Ken Roach View Post
Can you quantify "chatty" ?

Do you have a tap/mirror at the master side of this system that you are using the monitor the traffic ?

I just hooked up a tap and Wireshark, and monitored my 1756-ENBT using the filter:

Of course ARPs are broadcast packets and every device on the network sends them out periodically. If you've got a router on the network that's providing DHCP addresses to anything, it's probably the most common source of ARP packets.
Yes had wireshark running on the far end, would see 8-10 ARP requests between each poll containing data I wanted. and the source of the arp was the EN2TR module dedicated to this radio system.

I will have better network segmentation at some point in the not to distant future, but to do that I need to do a massive IP change at the same time (network is flat at the moment) so that will take coordination and this radio stuff I can do now. When network refurb is done this traffic gets its own vlan but if the source of the ARP is the PLC comm module, that's not gonna change much. I can add a firewall or whatever it takes and I'm aware that is a solution, but I'm just mostly curious why the EN2TR module is sourcing these at all. I wouldn't think it should.

The problem with the ARP quantity is they take over a cycle of "store and forward" required for the radio to reach the valleys from the hilltops and this duplication of data just chokes the throughput.

This are not 900mhz, these are 200mhz, licensed freq and the RF coverage is ideal for the local terrain, but of course that comes at a cost of bandwidth, which was expected and is fine.

I'm pretty stuck with the MDS radios. It's sad to say these well reviewed models have been a royal pain because the Store and Forward tech would not work and we had to prove to GE that was the case. Because of that we have been responsible for several recent firmware updates from 6 to 7x. But unless I can prove they just wont work, I can't return them. And I can't believe they wont, Me or GE just have something set wrong, ARP should not be broadcast on this type of low bandwidth communication by default.

As I said the serial comm channel works great, Quiet until called upon and responds like lightning... not so much the ethernet portion.

So the battle trudges forward.
  Reply With Quote
Old July 30th, 2020, 08:31 PM   #20
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
Quote:
Originally Posted by Maxkling View Post
Also network storms are pretty much looping and multiplying ARP requests, I know it would probably cause more havoc than you are describing, but Ive never seen how this would react with a throughput restriction of a radio in between.
I created one of those loops a while back accidentally when I was using the microwave link as a backhaul. That was memorable to put it mildly.
  Reply With Quote
Old July 30th, 2020, 08:37 PM   #21
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
Quote:
Originally Posted by Maxkling View Post
You can also segregate the broadcast domain through VLANs. The problem is at some point you will have to route the traffic if devices on separate VLANs need to talk. You can also place a NAT router on the outbound side of the radios which will also stop multicast traffic and only allow addressed routable traffic.
There is no need for this chain of equipment to talk across vlans. One module runs the radio network on it's own vlan and acts as a data collector, another EN2T module in that rack is on a different segment and plant SCADA system will post/get it's data through that interface. The PLC is common, but not the comm modules.
  Reply With Quote
Old July 30th, 2020, 11:57 PM   #22
Ken Roach
Lifetime Supporting Member + Moderator
United States

Ken Roach is offline
 
Ken Roach's Avatar
 
Join Date: Apr 2002
Location: Seattle, WA
Posts: 15,472
Wireshark will tell the tale.

There is all sorts of wild stuff you learn when you pop open the hood of your network and run Wireshark. It's easy for even experienced folks to go chasing rabbits.


> ARP requests between each poll containing data I wanted.

Great; can you associate a time in between those ? How frequent are the polls ?

If you can post that data (or share it privately) that would be interesting. In Wireshark you can just filter on (arp) and save the displayed packets.


What I've got to test is a 1756-ENBT running on a small network, testing a Baluff EtherNet/IP to IOLink block. But because I've connected everything to my local network, I see traffic from a bunch of other printers and phones and various embedded devices.

The 1756-ENBT sends out a "gratuitous ARP" about every 122.45 seconds. Gratuitous ARPs are basically network-change probes so that devices that have MAC tables can update them. The target MAC ID will be ff:ff:ff:ff:ff:ff and they should be labeled "ARP Probe" in Wireshark.

Similarly, my I/O block on the network sends out an ARP probe every 93 to 145 seconds.

I've got an RSLogix 5000 session talking to the ControlLogix too, and there are ARP packets exchanged between my computer and the ControlLogix periodically as well (anything from 5 to 300 seconds).

And other things on the network contribute ARP packets too. Phones, tablets, the printer, a Raspberry Pi, the Chromecast in the ship's main salon, the ESP32 monitoring the blackwater tank.

If all the ARP traffic you're seeing out in the field on the radio link is for devices that are on the radio network, then you likely don't have a way to reduce them unless that feature exists in the radios.

But if they're coming from devices that are not on the radio network, then you do.

There's always the possibility that something mis-configured or malfunctioning is causing more ARPs than necessary to come from the 1756-EN2T.
  Reply With Quote
Old July 31st, 2020, 10:38 AM   #23
Firejo
Member
United States

Firejo is offline
 
Firejo's Avatar
 
Join Date: Jun 2008
Location: Redmond, WA
Posts: 1,428
I forgot to mention that I have tested the 5069-SERIAL module including installing the firmware update that added DF1. It is a full suite of DF1 protocols and it works just fine.
__________________
Go Hawks!!!
  Reply With Quote
Old July 31st, 2020, 04:34 PM   #24
Saffa
Member
New Zealand

Saffa is offline
 
Join Date: Feb 2012
Location: Bay of Plenty
Posts: 1,062
I don't have any experience with the radios you're using, but we make extensive use of Schneider Trio Q series radios in the UHF band. When operating in Layer 2 mode, you have an option to filter ARP and/or broadcast traffic. If the GE radios don't have that, it might need to be a feature request?

I very rarely use Layer 2 though as the Trio radios fully support Layer 3 routing. I set up my remotes with a /27 subnet which gives me 30 addresses per radio, more than enough for a remote pump station with a couple drives and a RTU and HMI. But, you can also do larger subnets if you need.

The networking guy that did the backhaul routers and firewalls taught me a lot. We only allow DNP3, DF1 and SSH out over the radio network, the router that the master radio plugs into is a firewall as well.

These work so well that a less "radio aware" contractor went online with a remote site PLC for half a day from head office over the radio network. I only found out a few days later when he complained that it was a bit slow. I'd never tried it before because I knew how much traffic RSLinx generates when you go online with a PLC. He got a telling off, but I was kinda impressed that it worked. Its all a hell of an improvement from the 1200 baud analog radio modems we had before.
  Reply With Quote
Old August 5th, 2020, 12:35 PM   #25
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
IT dept is installing a Palo Alto firewall (for other reasons) but I am going to take advantage of that when it arrives. However...

We have also had a consistent SAF problem with this radio since purchase and we thought it was something we were doing wrong... turns we could duplicate it for GE and it isn't something we did wrong. If they can't deliver a fix in short order then I'll be changing radios and moving on to a different set of problems. This has been going on for far too long.

I'll grab a capture again when this settles out.
  Reply With Quote
Old August 5th, 2020, 12:37 PM   #26
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
Quote:
Originally Posted by Firejo View Post
I forgot to mention that I have tested the 5069-SERIAL module including installing the firmware update that added DF1. It is a full suite of DF1 protocols and it works just fine.
Well that's good news. wasn't sure they were actually going to do it.
  Reply With Quote
Old August 6th, 2020, 05:05 AM   #27
Saffa
Member
New Zealand

Saffa is offline
 
Join Date: Feb 2012
Location: Bay of Plenty
Posts: 1,062
If you do end up changing radios, Racom RIPEX radios and the Trio Q series are both products I've seen working well in the field (neighboring water authority uses Ripex). More than happy to help with configuring Trio radios if you ended up going down that path. Certainly learnt a few lessons on our roll out that resulted in a fair bit of driving around to fix config files until i figured out how to use the SSH shell!
  Reply With Quote
Old August 6th, 2020, 10:22 AM   #28
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
Thanks Saffa

Had a conference call yesterday with GE management, 3 GE engineers plus my own guy. Excuses were given and promises were made. Looks like we will have made it through 2 revisions, 6.x, 7.x and now we will be 8.x . Our problem stems only from the forwarding that has to occur to get over the terrain in this area. On the sites that are LOS we don't have this issue. And I find that not every radio has a forwarding capability.

If this meeting doesn't bear fruit I'll be looking at all options. Not familiar with RIPEX but they will be looked at.


Still can't grasp the ARP issue though. ARP is necessary for switches, and I guess the EN2T modules consider themselves switches. Seems that blocking ARP at the radio or with a firewall ahead of the radio will only "frustrate" the EN2T module as it will keep asking. Perhaps Ethernet over low bandwidth is a bad idea all together. Or maybe it will be fine once the radios work like they should.

I'm ready for a beer.
  Reply With Quote
Old August 6th, 2020, 12:59 PM   #29
V0N_hydro
Member
Canada

V0N_hydro is offline
 
Join Date: Sep 2010
Location: lower mainland
Posts: 549
Quote:
Originally Posted by TheWaterboy View Post
Still can't grasp the ARP issue though. ARP is necessary for switches, and I guess the EN2T modules consider themselves switches. Seems that blocking ARP at the radio or with a firewall ahead of the radio will only "frustrate" the EN2T module as it will keep asking. Perhaps Ethernet over low bandwidth is a bad idea all together. Or maybe it will be fine once the radios work like they should.

create a new subnet for each end of the wireless link, put a router on each end of the wireless link, put the EN2T modules in the new subnets, no ARP traffic over the wireless link.


I see EN2T modules in wireshark pumping out one ARP per second when they are trying to establish a connection.
  Reply With Quote
Old August 6th, 2020, 01:35 PM   #30
TheWaterboy
Lifetime Supporting Member + Moderator
United States

TheWaterboy is offline
 
TheWaterboy's Avatar
 
Join Date: May 2006
Location: State of Denial
Posts: 1,053
I understand that would work, but there are a lot of sites that would then need routers and they have their own housekeeping traffic so I am only wondering why the radio doesn't handle the arp cache itself so it doesn't have to broadcast it over the air.

These radios can be routers but support told me that it wasn't necessary to go that far just for this reason.
  Reply With Quote
Reply
Jump to Live PLC Question and Answer Forum

Bookmarks


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Topics
Thread Thread Starter Forum Replies Last Post
On Logix5000 system, why there are two or more Ethernet cards on the same rack? wendaiyu LIVE PLC Questions And Answers 17 April 24th, 2016 10:18 PM
UNIGATE CL Ethernet IP Connectivity Issue Hal9000 LIVE PLC Questions And Answers 6 January 11th, 2012 11:28 AM
Ethernet Modbus+ over Radio 15 Miles too much? MesquiteThorn LIVE PLC Questions And Answers 1 January 22nd, 2011 03:53 PM
rslogix 5000 read/write to ethernet radio Stache LIVE PLC Questions And Answers 11 June 21st, 2010 08:04 AM
Using a RADES9300 and serial ethernet to connect to plc...Works over eth but not dial bdoub1eu LIVE PLC Questions And Answers 5 July 28th, 2006 09:30 PM


All times are GMT -4. The time now is 07:31 PM.


.