As part of understanding how IP V6 dynamic routing works, I managed to get my little home network to talk using IP V6.

Privacy

There are sites which can give you your geographic location from your IP V6 address. One site gave me the top part of my post code, the latitude and longitude of my garage, and my ISP provider. So instead of giving real IP addresses, Ive used xxxx:xxxx:xxxx:xxxx for the IP V6 address provided by my Internet Service Provider, and using the 2001:db8::/64 address which is assigned for documentation use.

My network for the easy bit

I have 2 Laptops and a Server all running Linux.

From my internet router I could see the information for LT2

• GUA (Temporary): xxxx:xxxx:xxxx:xxxx:d539:f842:755d:8927
• GUA (Permanent): xxxx:xxxx:xxxx:xxxx:4593:2842:db0b:4630

Where GUA is the Global address (known outside of my network).

On Linux I can use the ip -6 addr command to display the address of the connection. A connector can have more than one address.

On the server machine, the addresses were

wlxd037450ab7ac: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
  inet6 xxxx:xxxx:xxxx:xxxx:7694:3711:8f98:7271/64 scope global temporary dynamic... 
  inet6 xxxx:xxxx:xxxx:xxxx:a5e4:61a:b3b2:9d8b/64 scope global dynamic mngtmpaddr noprefixroute ...
  inet6 fe80::216a:2b1d:c908:eb39/64 scope link noprefixroute      

Every time I rebooted, the xxxx… was the same, but the rest of the address was different.

The global address share the same prefix xxxx:xxxx:xxxx:xxxx/64, which is the address my Internet Service Provider allocated to my home.

The Link-Local fe80:: is only used within the use of its interface. The address should be stable across reboots, but may change if you change the configuration.

Talking between the Linux systems.

On LT2 I could issue ping xxxx:xxxx:xxxx:xxxx:7694:3711:8f98:7271 to the Server, and this worked.

Create your own addresses

The string xxxx:xxxx:xxxx:xxxx:7694:3711:8f98:7271 is very long, and changes every time I restart Linux. This means I had to manually type the long address in when trying to use it This was tedious for ping, and if I used ssh to logon to the box, every time it asked me about the authentication of the host.

You can create your own address for an interface.

sudo ip -6 addr add xxxx:xxxx:xxxx:xxxx::cccc dev wlxd037450ab7ac

It has to use the same 64 left bits xxxx:xxxx:xxxx:xxxx,;the remaining bits can be almost anything. I like nice short ::4 or ::cccc type values

I issued that command on the server, and I was able to successfully ping that address from my laptop. How does that work?

• On my laptop I issued the ping command. Linux did not have any routing information for it, so it was sent using the default route to the wireless router.
• The wireless router did not know about the address, so issued a multicast address to all(3) systems connected to it. “Does any one have xxxx:xxxx:xxxx:xxxx::cccc”.
• Laptop1 got the request – and as it did not have the address – it ignored the request.
• The Server got the request – and as it did have the address – it replied “I have xxxx:xxxx:xxxx:xxxx::cccc”.
• The wireless router then forwards the ping request to the server, and cached the routing information.

If you look at the response times of the ping you can see the first request takes a long time

colinpaice@colinpaice:~$ ping  xxxx:xxxx:xxxx:xxxx::cccc
PING xxxx:xxxx:xxxx:xxxx::cccc(xxxx:xxxx:xxxx:xxxx::cccc) 56 data bytes
64 bytes from xxxx:xxxx:xxxx:xxxx::cccc: icmp_seq=1 ttl=64 time=383 ms
64 bytes from xxxx:xxxx:xxxx:xxxx::cccc: icmp_seq=2 ttl=64 time=74.5 ms
64 bytes from xxxx:xxxx:xxxx:xxxx::cccc: icmp_seq=3 ttl=64 time=6.95 ms

This information is cached on the Linux, and in the router.

For example ip -6 neigh gives

xxxx:xxxx:xxxx:xxxx::88 dev wlp4s0 lladdr 00:24:d6:5e:2e:d2 REACHABLE

After a few minutes the output is

xxxx:xxxx:xxxx:xxxx::88 dev wlp4s0 lladdr 00:24:d6:5e:2e:d2 STALE

The time for an entry to become stale is based on /proc/sys/net/ipv6/neigh/…/base_reachable_time. On my Linux system this is 30 seconds.

The above was the easy bit….

A more complex example – adding in a host without wireless access.

Laptop to z/OS

This configuration is the same as the first example with the addition of an Ethernet connection going to z/OS.

For z/OS to work with the Laptops, the wireless router needs to be told about the IP addresses on z/OS

One of the addresses for z/OS on the Ethernet-like connection from the server is 2001:db8:e::9. There are no 2a00… (xxxx…) addresses on z/OS because z/OS is not attached to the wireless network.

If I ping the 2001:db8:e::9 address from a laptop, it does not complete successfully. Looking at the traffic from the wireless router to the server, there is no traffic for 2001:db8:…

I had to use radvd on the Server to act as a router. The radvd configuration for the wireless router was

interface wlxd037450ab7ac
{
   AdvSendAdvert on;
   MaxRtrAdvInterval 60;
   MinDelayBetweenRAs  60; 
   AdvDefaultLifetime 3000;

   route 2001:db8:1::/64
   {
     AdvRoutePreference medium;
     AdvRouteLifetime 3100;
   
   };
};

The route…{} says I (the Linux system) know about this IP address range.

When this was activated, I could see a router advertisement of 2001:db8:e::/64 being sent to the wireless router. After this, there was traffic from the wireless router down to the server.

Ping to 2001:db8:1::9 on z/OS was successful. I stopped the radvd process on the server, and after about 3 minutes ping stopped working. This is because the information sent from radvd into wireless router gets stale and eventually deleted. Typically the radvd tasks sends the information regularly, so the wireless router has up to date routing information.

The source of the ping was xxxx:xxxx:xxxx:xxxx:84ce:f350:1dce:b4bf, the Laptop end of the wireless connection.

On z/OS this was routed through via the default route, xxxx:xxxx:xxxx:xxxx:84ce:f350:1dce:b4bf which was the Server end of the connection from z/OS.

The server either knew the route to my laptop, or it used the default to send it view the wireless router, which knew to send it to the laptop.

Pinging from z/OS

From z/OS I could successfully ping xxxx:xxxx:xxxx:xxxx:84ce:f350:1dce:b4bf. The request went via the z/OS default route to the server, and then via the wireless router to the laptop.

The reply (destination 2001:db8:1::9) went via the route described in laptop to z/OS above.

An even more complex example – doing it without using defaults.

I had this situation because the wireless dongle on the server was not very reliable and kept dropping the connection. This made it very hard to diagnose problem, as sometimes a ping would work – and a short while later, it would not work – and I assumed it was the configuration changes I was making.

Now that I understand more about Dynamic Routing and Neighbor Discovery, setting this up was remarkably easy. I’m sure there must be something I have missed.

z/OS setup

Define the interface

INTERFACE IFPORTCP6 
    DEFINE IPAQENET6 
    CHPIDTYPE OSD 
    PORTNAME PORTCP 
    INTFID 7:7:7:7 
                                                
INTERFACE IFPORTCP6 
    ADDADDR 2001:DB8:1::9 

Define an empty route table

You cannot define an empty routing table, so I defined a route with stupid addresses.

This means all routing addresses will be dynanic

BEGINRoutes 
;     Destination    FirstHop       LinkName  Size 
ROUTE 2999:db8::/80  2999:db8:3:0:3f57:d0f0:7e58:a7ed IFPORTCP6   MTU 5000 
ENDRoutes 

Linux Server setup

I had one radvd.conf file with a section for each interface

Interface to z/OS

interface  tap1
{
   AdvSendAdvert on; 
   MaxRtrAdvInterval 600;
   MinDelayBetweenRAs  600; 
   # make this a non default router
   AdvDefaultLifetime 0;
  # AdvDefaultLifetime 60;
  
   AdvManagedFlag  on;
   AdvOtherConfigFlag on;   
   
   prefix 2001:db8:1::/64
   {
     AdvRouterAddr on;
   };
  route 2001:db8::/64 {};
  route 2008::/64{};
};

• AdvDefaultLifetime 0; Do not use this as a default route.
• prefix 2001:db8:1::/64{…} use this route to get to 2001:db8:1:: range.
• route 2001:db8::/64 {}; z/OS uses this information to build its dynamic routing.
• route 2008::/64 {}; z/OS uses this information to build its dynamic routing.

Ethernet interface from server to the laptop

interface  eno1
{
   AdvSendAdvert on; 
   MaxRtrAdvInterval 600;
   MinDelayBetweenRAs  600;
   
   prefix 2008::/64
   {
     AdvRouterAddr on;
   };  
 
   prefix xxxx:xxxx:xxxx:xxxx::/64
   {
     AdvRouterAddr on; 
   };
   
    # I support this address range
   route 2001:db8:1::/64{};    
 
};

• prefix 2008::/64{…} The server can use this interface to get to 2008::/64 on the laptop.
• prefix xxxx:xxxx:xxxx:xxxx::/64 The server can use this interface to get to the wireless interface on the laptop.
• route 2001:db8:1::/64{}; The laptop can use this information to define its dynamic routes to get to 2001:db8:1::/64 range on the server.

Wireless interface – when it worked

interface wlxd037450ab7ac
{
   AdvSendAdvert on;
   MaxRtrAdvInterval 60;
   MinDelayBetweenRAs  60; 
   AdvDefaultLifetime 3000;

   route 2001:db8:1::/64
   {
     AdvRoutePreference medium;
     AdvRouteLifetime 3100;
   
   };
};

I used this so that other devices on my home network, could get to the z/OS using the Wireless interface.

Start up script

I used

#!/bin/bash -x


sudo sysctl -w net.ipv6.conf.all.forwarding=1
sudo sysctl -w net.ipv6.conf.tap1.forwarding=1
sudo sysctl -w net.ipv6.conf.eno1.forwarding=1
sudo sysctl -w net.ipv6.conf.all.accept_ra=2
sudo sysctl -w net.ipv6.conf.tap1.accept_ra=2
sudo sysctl -w net.ipv6.conf.eno1.accept_ra=2

#kill off the running radvd agent
ps ax |grep radvd |grep -v grep |awk '{print $1 }'|sudo xargs kill  -9
ps -A |grep radvd

sudo ip -6 neigh flush all 
sudo ip -6 route flush root 2001:db8:1::/64
sudo ip -6 -statistics route flush proto ra 

# restart the radvd agent 
sudo rm /var/log/radvd.log
sudo radvd -d 5 -l /var/log/radvd.log -m logfile  -C $p
sleep 1
less /var/log/radvd.log

Although I had specified

sudo sysctl -w net.ipv6.conf.all.accept_ra=2

This did not seem to work (bug?), and I had to explicitly set

sudo sysctl -w net.ipv6.conf.tap1.accept_ra=2
sudo sysctl -w net.ipv6.conf.eno1.accept_ra=2

Laptop setup

Define a short address on the laptop.

sudo ip -6 addr add 2008::5 dev enp0s31f6

And that’s almost it.

There was a tiny little complication.

Usually when I used ping on my laptop, it used the IP address I had created, 2008::5. I could see this in the Wireshark trace going to z/OS. When playing around with the wireless adapter, sometimes ping used the wireless address, and the ping failed, because z/OS did not know how to route back to the wireless address. Ping gave me a warning Warning: source address might be selected on device other than: enp0s31f6.

I fixed this by creating a

route xxxx:xxxx:xxxx:xxxx::/64{}

in the tap1 configuration for z/OS

and a

prefix xxxx:xxxx:xxxx:xxxx::/64{…}

statement for the eno1 interface to the laptop.

This blog post arose when trying to document IP V6 routing. The behaviour did not work as expected.

• Displaying a dynamic route
• How to change a dynamic route
• How do I delete a dynamically created route?
• Creating a default router

Static routes

Linux

On Linux, you can use command like

sudo ip -6 route add 2001:0db8:1::9/64 dev tap1
sudo ip -6 route del 2001:0db8:1::9/64 dev tap1
sudo ip -6 route flush 2001:0db8:1::9/64 dev tap1
sudo ip -6 -statistics route flush 2001:0db8:1::9/64 dev tap1

Some information may be stored in the neighbour cache so you may need to use commands like:

sudo ip -6 neigh flush to 2001:db8:1::9
sudo ip -6 neigh flush dev tap1

z/OS

You define static routes between BEGINRoutes and ENDRoutes. If you want to change one entry, you have to replace all entries. You cannot add or remove individual entries.

You cannot have an empty BEGINRoute… ENDRoute. If used, it has to have at least one entry. You can create a dummy entry that will never be used.

You can change this file, and use the OBEY command to activate it

v tcpip,tcpip,obeyfile,USER.Z24C.TCPPARMS(routefc)

To delete an entry, remove it from the file, and activate the file.

Dynamic routes

Dynamic routes are created from facilities like radvd on Linux. This defines capability available on an interface.

For example

interface  tap1
{
   AdvSendAdvert on; 
   AdvDefaultLifetime 60;

   prefix 2001:db8:1::/64
   {
   };
   route 2001:db8::/64 {AdvRouteLifetime 1800};
   route 2008::/64{ AdvRouteLifetime 600};
};

At the Linx(sender)

• it creates an internal address(fe80::…) for the tap1 interface
• it create a route saying to get to 2001:db8::1/64 route it over the tap1 interface.

At the remote end, it receives a Router Advertisement packet. This includes

• The IP address of tap1 at the Linux end (created above)
• Route statements which have
  • The address range
  • The route lifetime.

From this, the remote end creates a dynamic route for every route in the packet.

How to change a dynamic route

Change the radvd configuration file, then either

• cancel and restart ravd
  • ps ax |grep radvd |grep -v grep |awk ‘{print $1 }’|sudo xargs kill -9
  • sudo radvd -d 5 -l /var/log/radvd.log -m logfile -C radvd.conf
• send a SIGHUP (sudo kill -s HUP <pid> )

The route expires after AdvRouteLifetime seconds. For the route to remain current, there needs to be a regular Router Advertisement message.

Displaying a dynamic route

On Linux ip -6 route gave me (for one of the dynamic routes) for one entry

2008::/64 dev eno1 proto ra metric 100 pref medium
2008::/64 dev eno1 proto kernel metric 256 expires 86293sec pref medium

Where 86293sec is just under 1 day.

On z/OS you can display this dynamic routing information using the TSO NETSTAT ROUTE RADV DETAIL command.

Example output

DestIP:   2008::/64 
  Gw:     fe80::8024:bff:fe45:840c 
  Intf:   IFPORTCP6  MTU:  0 
  Metric: 00000002   LifetimeExp: 12/20/2022 12:32 
  GwReachable:  Yes  IntfActive:  Yes 

This shows there is an entry for 2008::/64, and it is due to expire at 20th December 2022 at 12:32.

How do I delete a dynamically created route?

You have several ways

• Remove it from the radvd configuraton file. Restart radvd, and let the definition expire – possibly hours later.
• On Linux, remove the entry from the config file, restart radvd, use route delete….
• Cause it to expire earlier
  • Edit the configuration file
  • Set AdvRouteLifetime to a low value like 10 seconds,
  • Restart the radvd agent. This sends the RA message to the remote system, and sets the expiry time of the one of interest,
  • Delete the route from the config file,
  • Restart the radvd agent again. This sends the RA which will not have the route.

The information may still be in the neighbourhood cache, and this may need to be flushed.

Creating a default router

For the interface statement, set a default life time > 0. A value of 0 says this is not a default router.

interface  tap1
{  
   AdvSendAdvert on;
   AdvDefaultLifetime 0;
...

To remove the default router, set AdvDefaultLifetime to 0; and redeploy.

If there is a static definition for the default route, this will be used in preference to the dynamically defined router.