Ethernet basics in detail

I just went into some detail on networking concepts in response to a MrPLC post about Office/Plant Ethernet options. I figure as long as I typed it, might as well share with you guys. I'm open to additions and corrections. If it's well received I can add media types, some Windows concepts, and some PLC networking concepts to make a couple page "industrial networking introduction" whitepaper.

Node here is a device on the network - PC, PLC, router, etc. Usually I'm referring to Layer 3, so it has an IP address.

Collision - Bad juju. The "red light on a hub" 2 or more nodes sending a signal on the same wire. A Collision Domain is a network segment where this can happen.

A Broadcast Domain is a network segment where nodes can communicate directly (without going through a router, or equivalent). The good is that it's easy to talk. The bad is that it's easy to talk. Each node will inspect each broadcast/multicast packet (DHCP requests, streaming video, chatty traffic, etc).

What's an Ethernet hub? It's a dumb device that gets a signal on one port and retransmits on all ports - like everyone yelling in a room. One Broadcast Domain, one Collision Domain. Great for running packet sniffers (programs to inspect all traffic). Poor for performance.

What's a bridge? A bridge effectively combines nodes, or sets of nodes in separate Collision Domains, but they maintain one Broadcast Domain. So you don't get as many wire issues (network performance problems), but everyone's still listening to everyone.

So what's a switch? A switch is what you get if you bridge each port together. One Broadcast Domain, but a Collision Domain for each port (or set of ports??). In any event, you shouldn't get collisions, but everyone is still listening to everyone else.

Everything described so far falls under Layers 1 and (mostly) 2. So assume you want 2 "networks", office and plant. We might set up 2 disconnected switches. Traffic doesn't pass between them - they're in 2 separate broadcast domains.

Suppose we put the two on separate IP subnets (a layer 3 concept). We could bridge our 2 switches with a crossover cable. It will seem like devices on one switch can't talk (directly) to one another. That's because they're on separate IP networks (layer 3). But, nodes on both switches will listen to broadcasts from each other. Remember, our bridge maintains a broadcast domain.

The fix to that is putting a router between the 2 switches. A router is a device that has 2 network cards, one on each network (logically and physically) that can pass layer 3 traffic. Now chatty traffic won't pass between switches unless it's directed to an address and goes through our router. Note that your router doesn't know about ports/traffic type, that would be layer 4+ and we'd be talking about a firewall.

This is a good place to ask yourself - does this all make sense? If so...

A managed switch allows you mess with port level settings, VLANS, and things like that.

A VLAN, "virtual local area network", is a logical layer 2 network. Think of it exactly like our "separate switches" example. If we set up the top half of our managed switch ports on VLAN1 and the bottom half on VLAN2, it would appear to users as if they were plugged into separate switches. In fact, we could plug a physical router into both VLANS and it would route traffic as we described in our example. The cool thing is that you can segment VLANs by things other than physical ports (like hardware addresses) and that you can change settings without rewiring anything. This allows you to "spread" your network as you grow.

As you get more into networking you realize that a lot of the concepts are more flexible to deal with logically, rather than physically. Things also get crazy as you introduce: Inter-VLAN routing, DHCP proxies, and policies that selectively pass higher/lower layer traffic. There are also concepts like QoS (Quality of Service) that can substantially improve network performance with the same amount of throughput. There's also a lot of things that can be tuned (packet fragmenting works with frame size, latency issues, security stuff, "packet shaping", etc) that come into play if you have bigger networks especially with encryption devices, VPN tunnels, satellite links, etc.

surferb said:

What's a bridge? A bridge effectively combines nodes, or sets of nodes in separate Collision Domains, but they maintain one Broadcast Domain. So you don't get as many wire issues (network performance problems), but everyone's still listening to everyone.

So what's a switch? A switch is what you get if you bridge each port together. One Broadcast Domain, but a Collision Domain for each port (or set of ports??). In any event, you shouldn't get collisions, but everyone is still listening to everyone else.

Click to expand...

The statement that everyone is listening to everyone else is a little misleading in these cases. I can understand (sortof) in the case of the bridge, but the switch doesn't fit as well. The main problem is that you are not making clear the distinction between broadcast traffic and point-to-point traffic. Everyone is NOT listening to everyone else on a switch, so much as everyone can speak directly to everyone else if they wish.

In a hub, all traffic is repeated to all ports. If A wants to talk to B, nodes C and D will be forced to listen to the conversation. Normally, they will not do anything with the information (unless that node is running a packet sniffer like WireShark or Ethereal to log such information.) The point is, on a hub, when node A speaks *even if the packet is only intended for node B* no other nodes can speak and all other nodes are forced to listen. If the packet was a broadcast packet, then by definition, all nodes are supposed to recieve the packet anyway.

Now, this works for smaller networks when it doesn't hurt to have C and D wait to talk until A and B are finished. It doesn't work so well when E through Z have to wait too. An early approach to solving this problem was to split the large group of nodes into two or more groups with a bridge between the two. A-M in one group and N-Z in the other in my example. If node A is speaking to node B, nodes N and O can carry on a conversation without interruption. The bridge learns what devices are in each group and inspects the packets. If one node is speaking to another node in the same group, the bridge doesn't pass the packet to the other group. If A tries to speak with N, the bridge recognizes this and forwards the packet to N's group so it can hear it. Likewise, it will forward N's reply back to A. The other notable property is that any broadcast traffic is automatically forwarded across the bridge. For those of you running packet sniffers in A's group, you will see Point to Point conversations only from A's group, but not from N's group. You will also see all broadcast traffic.

So we can see how splitting up the network into more manageable chunks using bridges is a good thing (a practice called segmenting a network.) If we take this principle to its logical end, we arrive at the switch. Every node is on its own segment. Each node is bridged to the other. What happens now is that each node only sees traffic that was intended for it. If A talks to B, C-Z don't see the traffic. In fact, A and B can carry on a full speed conversation while C and D, and E and N do the same. All nodes will still see broadcast traffic. For those of you running sniffers, you will only see traffic intended for/sourced from your node + all broadcast traffic. Please note that there are sneaky and nefarious ways around this, so this shouldn't be considered a private conversation technique as far as security is concerned. But for normal everyday networks, it's a great way to manage contention.

The only other thing that sticks out at me is that not everyone is familiar with the OSI network model, so the level 1, 2 and 3 stuff may not make sense. Also, some of the more complex routing and traffic management functions are sortof at the hand-waving level of explaination, but I'm guessing that that will be sufficient for the audience intended.

Other than that, it was a pretty good intro.

Some notes on the previous posts:

A hub retransmits a received frame on all ports *EXCEPT* the port on which the frame was received.

A very important concept about VLANs is that VLAN membership is on a frame by frame basis, NOT by port. Althoughgh it is common to assign all frames from a particular port to a VLAN, in which case every frame from that port is on the same VLAN. Without frame by frame membership, VLAN trunking would not be possible.

A router need not have 2(or more) physical ports, in fact layer 3 switches implement routers with no physical ports.

Collisions can occur even if two stations are not attached to the same physical wire. In fact with today's UTP structured cabling, it is quite rare for stations to share the same wire. Collisions occur if two stations transmit at the same time in the same collision domain.

Collisions aren't necessarily bad, they are a natural occurance of the CSMA/CD algorithm. Ethernet is designed to deal with them, although in today's microsegmented, full duplex, switched networks, the collision detect part is usually disabled as collisions are impossible

The term 'directly' used in describing a broadcast domain isn't very precise. The term 'easy' is even worse. Perhaps the phrase "stations may communicate direcly at Layer 2" is a better choice.

Packet fragmenting is NOT a part of Layer 2 and switching. Layer 2 has an MTU and that's it. Fragmentation and reassembly is handled by Layer 3(IP).

Very interesting.... thanks! to all

Now for my question, not sure on the terminology so forgive me if I am wrong.

We have a T10/100 system now, we are changing over our hubs to giga-bites, it is my understanding that from the node to the hub it will stay T10/100 and only from the hub to the server will change.

I have about 30 static nodes that belong to me (PLCs, HMIs, Drives, etc) I have been asked do I need my own network?

So what do you think? do I? or can I stay, is there an advantge of having my own, I have never had an issue in the past.

Thanks

This response was written to describe how VLANs are effictively like "separating plant/office networks", specifically, how the old school approach of demanding separate physical hardware isn't necessary. The focus on multicast/broadcast was a worst case analysis.

You're right - a switch does provide a comm path that only goes between the physical ports of nodes communicating. I alluded to this with the hub description in using a packet sniffer. And yes, switches are WAY BETTER than using hubs - they create problems! I was going off of a worst case scenario where devices on one network were interfering with nodes on another.

Good point about the OSI, TCP/IP layers and omitted info about higher layer protocols. If I write a simplified white paper it will go into more depth about that.

Brian123 said:

The statement that everyone is listening to everyone else is a little misleading in these cases. I can understand (sortof) in the case of the bridge, but the switch doesn't fit as well. The main problem is that you are not making clear the distinction between broadcast traffic and point-to-point traffic. Everyone is NOT listening to everyone else on a switch, so much as everyone can speak directly to everyone else if they wish.

The only other thing that sticks out at me is that not everyone is familiar with the OSI network model, so the level 1, 2 and 3 stuff may not make sense. Also, some of the more complex routing and traffic management functions are sortof at the hand-waving level of explaination, but I'm guessing that that will be sufficient for the audience intended.

Other than that, it was a pretty good intro.

Click to expand...

Jason - thanks, you are correct and more precise. I was trying to make it conceptually simple, but not overkill.

As long as we're getting into it, a few more points that come to mind:
-VLANs can be assigned by other critera, such as MAC address. Your frame description is much more precise.
-A router can have any number of interfaces, physical or logical. If you have a trunk link between VLANs you can actually connect a router to multiple VLANs with one link.
-Layer 3 switch and "router" are basically the same thing. Layer 3 switch uses ASICS (specialized circuits), which are much faster, router using a more general processor and memory, which is slower, but tends to be more flexible.

Good stuffs...

Jason Valenzuela said:

Some notes on the previous posts:

A hub retransmits a received frame on all ports *EXCEPT* the port on which the frame was received.

A very important concept about VLANs is that VLAN membership is on a frame by frame basis, NOT by port. Althoughgh it is common to assign all frames from a particular port to a VLAN, in which case every frame from that port is on the same VLAN. Without frame by frame membership, VLAN trunking would not be possible.

A router need not have 2(or more) physical ports, in fact layer 3 switches implement routers with no physical ports.

Collisions can occur even if two stations are not attached to the same physical wire. In fact with today's UTP structured cabling, it is quite rare for stations to share the same wire. Collisions occur if two stations transmit at the same time in the same collision domain.

Collisions aren't necessarily bad, they are a natural occurance of the CSMA/CD algorithm. Ethernet is designed to deal with them, although in today's microsegmented, full duplex, switched networks, the collision detect part is usually disabled as collisions are impossible

The term 'directly' used in describing a broadcast domain isn't very precise. The term 'easy' is even worse. Perhaps the phrase "stations may communicate direcly at Layer 2" is a better choice.

Packet fragmenting is NOT a part of Layer 2 and switching. Layer 2 has an MTU and that's it. Fragmentation and reassembly is handled by Layer 3(IP).

Click to expand...

Thanks to all. I know nothing of networking and I am enjoying the opportunity of learning more.

geniusintraining said:

Very interesting.... thanks! to all

Now for my question, not sure on the terminology so forgive me if I am wrong.

We have a T10/100 system now, we are changing over our hubs to giga-bites, it is my understanding that from the node to the hub it will stay T10/100 and only from the hub to the server will change.

I have about 30 static nodes that belong to me (PLCs, HMIs, Drives, etc) I have been asked do I need my own network?

So what do you think? do I? or can I stay, is there an advantge of having my own, I have never had an issue in the past.

Thanks

Click to expand...

Having your own network (whether it's separate physical hardware or your own virtual network (VLAN)) comes with some perks. Not having to worry about the guy in the cube down the hall accidentally installing the latest malware that causes broadcast storms or other such disruptive network traffic. On the flip side, if you misconfigure something, the rest of the company is less likely to be affected. Also, if the IT department firewalls your segment properly, your list of security concerns goes down. You don't have to worry about someone trying to access your section of the network without specific permission. This may, however, cause YOU to jump through another hoop or two to access your own segment from outside (not necessarily a bad thing.)

In general, it is a Good Thing to have industrial controls separated from general office type networks.

Agreed - often that is still better managed by IT. Especially if you need high assurance, security, remote access, etc. Physically separate equipment makes sense in some cases - you'll often be able to get better results when dealing logically - especially if the administrator is skilled/experienced (which might also be you locking up a separate switch in a cabinet).

Brian123 said:

In general, it is a Good Thing to have industrial controls separated from general office type networks.

Click to expand...