One minute MVS: Networking subnets.

I’ve understood and used subnets, (in a hand waving way), but found it hard to write down what they are good for, and why we have them. There are many explanations on the web but they all seemed to describe how to use subnets, and not why we need them. Some of what I say below, may be strictly not true, but I hope it gives the right concepts.

You can use an Ethernet cable to join two machines. This is not very interesting.
You can have an Ethernet router. You plug the Ethernet cable from your machine to one of the ports on the Ethernet router.
The Ethernet router has devices attached to it with addresses such as 192.168.1.160, 192.168.1.24, 192.168.1.74. The router handles traffic for 192.168.1.* The IP address is 32 bits long, and the router is configured so that if the top 24 bits of an address are 192.168.1, then pass the traffic to the router. This is written as 192.168.1.0/24. The remaining 8 bits can be used for devices attached to the router, so almost 256 devices. (192.168.1.0 and 192.168.1.255 are reserved).
If you had a large building you could configure a router with address 192.169/16 and have 65,000+ devices attached to it. This may not be a good idea.
- The router sends out management packets to all devices in the subnet saying, for example “does anyone have this IP address”. With many devices the router could spend all its time processing these management packets, and not handling user data
- You may want to segregate different areas, so addresses 192.168.1.* is for the first floor, and 192.168.2.* is for the second floor. If you want to have a firewall for the first floor it is much easier configuring all traffic going to 192.168.1.* rather than for some machines within 192.168.* and so all users are using the firewall – which may not be what you want.
- Each floor has a confidential printer. It is easier to configure the printer so that only machines with the same subnet address, IP address 192.168.1.* can send print files to the printer on 192.168.1.22, rather than filter out users on the second floor.
With IP V6 there are 128 bits available for subnetting. Mostly a subnet of /64 is used. I have an address 2a00:9999:8888:7777:a0cd:ec92:bceb:91ab/64 so 2a00:9999:8888:7777 is the address of my router (64 bits), and the device on the router is currently a0cd:ec92:bceb:91ab (64 bits).

Basic connectivity

Single point to point cable

My laptop is connected to my server by an Ethernet cable.

I’ve defined the address at each end 10.1.0.2/24 at the laptop and 10.1.0.3/24 at the server. I can ping between the two machines. When I changed the server to have 12.1.0.3/24 there was no connectivity – because they were in different subnets.

Wireless connection – IPV4

My system was configured automatically to have the laptop 192.168.1.222/24 and the server 192.168.1.222/24. These are the same subnet, so traffic goes from my laptop up the wireless connection to the wireless router, and to the server over the wireless connection.

Wireless connection – IPV6

My system was configured automatically to have the laptop a prefix (subnet) of fe80 and specific address c82d:b94c:21fa:3d1c with this this subnet. The server had prefix (subnet) fe80 and specific address c82d:b94c:21fa:3d1c.

The default routing is via device (the wireless router) with prefix (subnet) fe80 and address c82d:b94c:21fa:3d1. These are both “internal to the router” addresses.

Today my laptop also has IP address 2a00:9999:8888:7777:a0cd:ec92:bceb:91ab/64 and my server has address2a00:9999:8888:7777:605a:2d22:5daf:53d7/64. These can be used to contact sites on the internet, because they are external addresses.

Getting out of the subnet.

My server has a connection over virtual Ethernet to z/OS. The server end of this link has address 10.1.3.1/24. If I use wireless connection from my laptop to the server, I cannot easily access this link, because the wireless router does not know about 10.1.3.1 – and I have no way of configuring it.

On Linux I can configure the server to be a software router (radvd), and have a physical Ethernet cable to the it from my laptop. This way I can control the IP routing to and from the server.

You can also use a bridge … but that is an advanced topic.

How does my network interface get an IP address, and is a generated address ok to use?

This article has some good concepts about IP V6 addresses.

What addresses does an interface have?

An interface (think end of an Ethernet cable) typically has one IP V4 address which is usually manually assigned to it, and/or multiple IPV6 addresses.

An interface can have multiple IPV6 addresses – why?

You can explicitly define it

You can assign your own IP address(es) to the interface. You can do this so it has a global address, reachable from other networks.

Dynamic Host Configuration Protocol (DHCP)

If you are using a DHCP client, it communicates with a DHCP server which gives it configuration information, such as an IP address, and subnet, the client then configures the interface.

There has been a shortage of IP V4 addresses for many years. Consider an office which has a “drop-in area”. Rather than give each of the potential uses an IP address, the office is given as many IP addresses as there are desks in the drop-in area. This means perhaps 10 ip addresses are needed instead of 100. This is the advantage of DHCP.

For client devices this is fine. Your machine connects to the server and passes its IP address. The server does some work and sends the response back to the requester.

Tomorrow you may get a different IP address. It works. This means no IP address information is saved across different days. It is stateless.

A server needs either

a fixed IP address so clients can contact it,
a dynamic address, and an update to the DNS router to say “today megabank.com is on IP address 9.99.99.1”. It can take time to propagate an IP address across the worldwide DNS network.

IPv6 Stateless Address Auto-configuration (SLAAC)

The ideas behind DHCP have been extended in IPV6, the concepts are similar. Stateless Address Auto-configuration is well documented here

Within a network or router domain the system can generate an address, and it is only used within this domain, it could have a different address every time the interface is started. This is known as Stateless Address Autoconfiguration.

When an interface is started it generates an “internal use” address composed of FE00 + a mangled MAC address.

The interface then asks all devices on the local network, does anyone have this address FE00… This is for Duplicate Address Detection (DAD). There should be no reply. If there is a reply, then there is a duplicate address on the network (and the interface generates another address and repeats the process).

The interface then sends out a Router Solicitation request asking “Are there any routers out there?”. A router will then reply giving information. It will contain the “global IP prefix” such as 2001:db8::. which means to the outside world, my the address of the router is 2001:db8::/64. From this information plus the MAC address the interface can generate its IP address. The router also contains the address of the gateway (the address of the router with connections to the outside world) so traffic can now be routed externally.

This means you configure the router, and not the individual devices. If you have many devices this can save you a lot of work. If you change the router configuration, it is propagated to all the devices attached to it.

IPV6 privacy extensions will generate a “random address” (within the subnet). This is to prevent bad actors from monitoring traffic and using your IP address to monitor what sites you visit. By having a different IP address every day means they cannot correlate the traffic for a user.

Does it matter if my address is auto generated?

This is another of the “it depends” answers.

For machines that initiate work, it may not matter that the allocated IP address is different everyday or the same every day. Your IP address is passed with the request to the server. The server does the work, and sends the response back through the network. Assuming the network is configured properly the response will get back to you.

If your machine is a server machine, clients need to know the server’s address. If this changes your clients may need to use DNS to find the new address, and not use the dotted IP address. You may want to allocate a permanent IP address (within the router’s subnet).

Routers, firewalls and filters.

If your machines address is within the router’s subnet, traffic should be able to get to your router and so to your machine. If you change the subnet, traffic may not get to your router.

A firewall can be configured to allow traffic to and from specified addresses (and ports). If you use a different address, either at client or the server, the firewall may not let your packets through. Similarly with a network filter.

I was playing around with configuring IP V6 on my laptop, and the connection to z/OS failed. This was because I had been using one IP address, which I could see flowing to the back-end. When I tried some other configuration, there were more IP addresses allocated to the client, and a different IP address was used as the originator’s IP address in the ping request. The back-end server did not know how to route traffic back to this address, and so the return packets were thrown away and the ping timed out.

You need to be aware which addresses are used for your work. With some IP programs you can bind to a specific local IP address, and force the application to use a particular IP address. For example the Linux Ping command -I interface option.

Why Linux is not responding – it’s the flaming file-wall!

I could not ping my Linux Server, and could not issue a traceroute command. It turns out the firewall was blocking the traceroute flow.

This blog posts describes how I checked this, and fixed the firewall problem.

Traceroute sends (by default) a UDP packet to a port address in the range 33434-33523. It usually responds with a “timed out” type response. If there is no response then there is a good chance that the packet is being dropped by a firewall.

See Understanding traceroute (or tracerte).

Using wireshark I could see UDP packets going in to my Linux, but there was no corresponding reply being returned.

When traceroute worked I got the out inbound UDP packet, and the outbound response with “destination unreachable” (which looks like a problem but actually shows normal behaviour) as shown in the data below. Wireshark highlights it with a black background, because it thinks it is a problem.

Source	Destination	Dst Port	port	Protocol	Info
2001:db8::2	2001:db8::7	33434	52119	UDP	52119 → 33434 Len=24
2001:db8::7	2001:db8::7	33434	52119	ICMPV6	Destination Unreachable (Port unreachable)

When traceroute failed I only got the inbound UDP packet

Source	Destination	Dst Port	port	Protocol	Info
2001:db8::2	2001:db8::7	33434	52119	UDP	52119 → 33434 Len=24

If the packets is blocked by a firewall, then the traceroute output will have “*” as the node name.

Useful Fire Wall (ufw) is documented here.

I was on Ubuntu Linux 20.04.

Display the status of the firewall

sudo ufw status verbose

This gave me

Status: active
Logging: off (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW IN    10.1.1.2                  
22/tcp                     ALLOW IN    Anywhere                  
20,21,10000:10100/tcp      ALLOW IN    Anywhere                  
21/tcp                     ALLOW IN    Anywhere                  
20/tcp                     ALLOW IN    Anywhere                  
22/tcp (v6)                ALLOW IN    Anywhere (v6)             
20,21,10000:10100/tcp (v6) ALLOW IN    Anywhere (v6)             
21/tcp (v6)                ALLOW IN    Anywhere (v6)             
20/tcp (v6)                ALLOW IN    Anywhere (v6)

By default,

incoming data is blocked
outbound data is allowed
routed data is blocked.

Logging is off, and problems are not reported.

The displays shows there are no rules for UDP – so any incoming UDP request is blocked (quietly dropped = dropped without telling anyone).

You may want to issue the command and pipe the output to a file, ufw.txt, to keep a record of the status before you make any changes. If you make any changes, they persist – even across reboot.

Enable logging to see what is being blocked

sudo ufw logging on

Rerun your traceroute or command.

At the bottom of /var/log/ufw I had (this has been reformatted to make it display better)

Nov 28 12:27:43 colinpaice kernel: [ 3317.641508] [UFW BLOCK] IN=enp0s31f6 OUT= MAC=8c:16:45:36:f4:8a:00:d8:61:e9:31:2a:86:dd SRC=2001:0db8:0000:0000:0000:0000:0000:0002 DST=2001:0db8:0000:0000:0000:0000:0000:0007
LEN=80
TC=0
HOPLIMIT=1
FLOWLBL=924186
PROTO=UDP
SPT=48582
DPT=33434
LEN=40

Wireshark gave me

Frame 4: 94 bytes on wire (752 bits), 94 bytes captured (752 bits) on interface enp0s31f6, id 0   
Ethernet II, Src: Micro-St_e9:31:2a (00:d8:61:e9:31:2a), Dst: LCFCHeFe_36:f4:8a (8c:16:45:36:f4:8a)
Internet Protocol Version 6, Src: 2001:db8::2, Dst: 2001:db8::7
    0110 .... = Version: 6
    .... 0000 0000 .... .... .... .... .... = Traffic Class: 0x00 
    .... .... .... 1110 0001 1010 0001 1010 = Flow Label: 0xe1a1a
    Payload Length: 40
    Next Header: UDP (17)
    Hop Limit: 1
    Source: 2001:db8::2
    Destination: 2001:db8::7
User Datagram Protocol, Src Port: 48582, Dst Port: 33434
    Source Port: 48582
    Destination Port: 33434
    Length: 40
    Checksum: 0x6ebd [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
    [Timestamps]
Data (32 bytes)

From this, we can see the fields match up

flow label (0xe1a1a = 924186)
source 2001:db8::2 = 2001:0db8:0000:0000:0000:0000:0000:0002
destination 2001:db8::7 = 2001:0db8:0000:0000:0000:0000:0000:0007
source port 48582
destination port 33434.

Port 33434 is used by traceroute, so this is a good clue this is a traceroute packet.

The reason the record was written to the log is [UFW BLOCK]. The firewall blocked it.

The request came in over interface enp0s31f6.

How to enable it.

You can specify different filters, and granularity of parameters.

For example

sudo ufw rule allow log proto udp from 2001:db8::2
sudo ufw rule allow in on enp0s31f6 log comment ‘colin-ethernet’
sudo ufw rule allow proto udp to 2001:db8::7 port 33434:33523 from 2001:db8::2

Where enp0s31f6 is the name of the ethernet link where the traffic comes from.

When running with either of these, I had in the log file

Nov 28 17:03:12 colinpaice kernel: [19847.112045] 
[UFW ALLOW] 
IN=enp0s31f6 OUT= MAC=8c:16:45:36:f4:8a:00:d8:61:e9:31:2a:86:dd SRC=2001:0db8:0001:0000:0000:0000:0000:0009 
DST=2001:0db8:0000:0000:0000:0000:0000:0007 
LEN=60 TC=0 HOPLIMIT=1 FLOWLBL=0 PROTO=UDP SPT=33434 DPT=33440 LEN=20

and the traceroute worked.

Note: The comment ‘…’ is an administration aid to give a description. It does not come out in the logs.

Display the rules

sudo ufw status numbered

gave

Status: active

     To                         Action      From
     --                         ------      ----
[ 1] Anywhere                   ALLOW IN    10.1.1.2                  
[ 2] 22/tcp                     ALLOW IN    Anywhere                  
[ 3] 20,21,10000:10100/tcp      ALLOW IN    Anywhere                  
[ 4] 21/tcp                     ALLOW IN    Anywhere                  
[ 5] 20/tcp                     ALLOW IN    Anywhere                  
[ 6] Anywhere on enp0s31f6      ALLOW IN    Anywhere                   (log)
[ 7] 22/tcp (v6)                ALLOW IN    Anywhere (v6)             
[ 8] 20,21,10000:10100/tcp (v6) ALLOW IN    Anywhere (v6)             
[ 9] 21/tcp (v6)                ALLOW IN    Anywhere (v6)             
[10] 20/tcp (v6)                ALLOW IN    Anywhere (v6)             
[11] Anywhere (v6) on enp0s31f6 ALLOW IN    Anywhere (v6)              (log)
[12] 2001:db8::7 33434/udp      ALLOW IN    2001:db8::2                (log)

There is now a rule [12] for udp to 2001:db8::7 port 33434

You can use commands like

sudo ufw delete 6

to delete a row.

Note: Always display before delete. Having deleted the rule 6 – rule 7 now becomes rule 6, etc.

Now that it works…

Any changes to ufw are remembered across reboots.

You may want to turn off the logging, until the next problem

sudo ufw logging off

and remove the log from the fire wall rules, by deleting and re-adding the rule.

sudo ufw rule delete allow log proto udp from 2001:db8::2

sudo ufw rule allow proto udp from 2001:db8::2