Category: IP V6

One minute: Understanding TCPIP routing: Static, RIP, OSPF

This is another blog post in the series “One minute…” which gives the basic concepts of a topic, with enough information so that you can read other documentation, but without going too deeply.

IP networks can range in size from 2 nodes(machines), to millions of nodes(machines), and a packet can go from my machine to any available machines – and it arrives! How does this miracle work?

I’ll work with IP V6 to make it more interesting (and there is already a lot of documentation for IP V4)

I have and old laptop, connected by Ethernet to my new laptop. My new laptop is connected by wireless to my server which is connected to z/OS. I can ping from the old laptop to z/OS.

  • Each machine needs connectivity for example wireless, Ethernet, or both.
  • Each machine has one or more interfaces where the connectivity comes in (think Ethernet port, and Wireless connection). This is sometimes known as a device.
  • Each interface has one or more IP addresses.
  • You can have hardware routers, or can route through software, without a hardware router. A hardware router can do more than route.
  • Each machine can route traffic over an interface (or throw away the packet).
    • If there is only one interface this is easy – all traffic goes down it.
    • If there is more than one interface you can specify which address ranges go to which interface.
    • You can have a default catch-all if none of the definitions match
    • You can have the same address using different interfaces, and the system can exploit metrics to decide which will be used.
    • You can have policy based routing. For example
      • packets from this premier user, going to a specific IP address should use the high performance (and more expensive) interface,
      • people using the free service, use the slower(and cheaper) interface.

Modern routing uses the network topology to manage the routing tables and metrics in each machine.

Static

The administrator defines a table of “if you want get to… then use this interface, the default is to send the packet using this … interface”. For example with z/OS

BEGINRoutes 
;     Destination   SubnetMask    FirstHop    LinkName  Size 
; ROUTE 192.168.0.0 255.255.255.0       =     ETH2 MTU 1492 
ROUTE 10.0.0.0      255.0.0.0           =     ETH1 MTU 1492 
ROUTE DEFAULT                     10.1.1.1    ETH1 MTU 1492 
ROUTE 10.1.0.0      255.255.255.0   10.1.1.1  ETH1 MTU 1492 

ROUTE 2001:db8::/64 fe80::f8b5:3466:aa53:2f56 JFPORTCP2 MTU 5000 
ROUTE fe80::17      HOST =                    IFPORTCP6 MTU 5000 
ROUTE default6      fe80::f8b5:e4ff:fe59:2e51 IFPORTCP6 MTU 5000
                                                                      
ENDRoutes

Says

  • All traffic for 10.*.*.* goes via interface ETH1.
  • If no rule matches (for IP V4) use the DEFAULT route via ETH1. The remote end of the connection has IP address 10.1.1.1
  • All traffic for IPV6 address 2001:db8:0:* goes via interface JFPORTCP2
  • If no rule matches (for IP V6) use the DEFAULT6 route via IFPORTCP6. The remote end of the connection has IP address fe80::f8b5:e4ff:fe59:2e51.

On Linux the ip route command gave

default via 192.168.1.254 dev wlxd037450ab7ac proto dhcp metric 600 
10.1.0.0/24 dev eno1 proto kernel scope link src 10.1.0.3 metric 100 
10.1.1.0/24 dev tap0 proto kernel scope link src 10.1.1.1

This says

  • The default is to send any traffic via device wlxd037450ab7ac.
  • Any traffic for 10.1.0.* goes via device eno1
  • Any traffic for 10.1.1.* goes via device tap0.

Routing Information Protocol(RIP)

Manually assigning metrics (priorities) to hint which routes are best, quickly becomes unmanageable when the number of nodes(hosts) increases.

If the 1980’s the first attempt to solve this was using RIP. It uses “hop count” of the destination from the machine as a metric. A route with a small hop count will get selected over a route with a large hop count. Of course this means that each machine needs to know the topology. RIP can support at most 15 hops.

Each node participating in RIP learns about all other nodes participating in RIP.

Every 30 seconds each node sends to adjacent nodes “I know about the following nodes and their route statements”. Given this, eventually all nodes connected to the network will know the complete topology.
For example, from the frr(Free Range Routing) trace on Linux

RIPng update timer expired!
RIPng update routes on interface tap1
  send interface tap1
  SEND response version 1 packet size 144
   2001:db8::/64 metric 1 tag 0
    2001:db8:1::/64 metric 1 tag 0
   2002::/64 metric 2 tag 0
    2002:2::/64 metric 2 tag 0
   2008::/64 metric 3 tag 0
    2009::/64 metric 1 tag 0
    2a00:23c5:978f:6e01::/64 metric 1 tag 0

This says

  • The 30 second timer woke up
  • It sent information to interface tap1
  • 2001:db8::/64 metric 1 this is on my host(1 hop)
  • 2002::/64 metric 2 this is from a router directly connected to me (2 hops).
  • 2008::/64 metric 3 is connected to a router connected to a router directly connected to me (3 hops.)

On z/OS the command F OMP1,RT6TABLE gave me message EZZ7979I . See OMPROUTE IPv6 main routing table for more information

DESTINATION: 2002::/64 
  NEXT HOP: FE80::E42D:73FF:FEB1:1AB8 
  TYPE:  RIP           COST:  3         AGE: 10 
DESTINATION: 2001:DB8::/64 
  NEXT HOP: FE80::E42D:73FF:FEB1:1AB8 
  TYPE:  RIP*          COST:  2         AGE: 0

This says

  • To get to 2002::/64 go down interface with the IP address FE80::E42D:73FF:FEB1:1AB8.
  • This route has been provided by the RIP code.
  • The destination is 3 hops away (in the information sent from the server it was 2 hops away)

The fields are

  • RIP – Indicates a route that was learned through the IPv6 RIP protocol.
  • * An asterisk (*) after the route type indicates that the route has a directly connected backup.
  • Cost 3 – this route is 3 hops away.
  • Age 10 -Indicates the time that has elapsed since the routing table entry was last refreshed

OSPF (Open Shortest Path First)

OSPF was developed after RIP, as RIP had limitations – the maximum number of hops was 15, and every 30 seconds there was a deluge of information being sent around. The OSPF standard came out in 1998 10 years after RIP.

Using OSPF, when a system starts up it sends to the neighbouring systems “Hello,_ my router id is 9.3.4.66, and I have the following IP addresses and routes.” This information is propagated to all nodes in the OSPF area. When a node receives this information it updates its internal map (database) with this information. Every 10 seconds or so each node sends a “Hello Packet” to the adjacent node to say “I’m still here”. If this packet is not received, then it can broadcast “This node …. is not_responsive/dead”, and all other nodes can then update their maps.

If the configuration changes, for example an IP address is added to an interface, the node’s information is propagated throughout the network. In a stable network, the network traffic is just the “Hello packet” sent to the next node, and any configuration changes propagated.

One of the pieces of information sent out about node’s route is the metric or “cost”. When a node is deciding which interface to route a packet to, OSPF can calculate the overall “cost” and if there are a choice of routes to the destination it can decide which interface gives the best cost.

To make it easier to administer, you can have areas, so you might have an area being the UK, another area being Denmark, and another area being the USA.

What does tso netstat neighbour give you?

The command TSO NETSTAT ND gave me 

Query Neighbor cache for 2001:db8:1:0:8024:bff:fe45:840c 
  IntfName: IFPORTCP6          IntfType: IPAQENET6 
  LinkLayerAddr: 82240B45840C  State: Reachable 
  Type: Router                 AdvDfltRtr: No 

Query Neighbor cache for fe80::8024:bff:fe45:840c 
  IntfName: IFPORTCP6          IntfType: IPAQENET6 
  LinkLayerAddr: 82240B45840C  State: Reachable 
  Type: Router                 AdvDfltRtr: No 

Query Neighbor cache for fe80::9863:1eff:fe13:1408 
  IntfName: JFPORTCP6          IntfType: IPAQENET6 
  LinkLayerAddr: 9A631E131408  State: Reachable 
  Type: Router                 AdvDfltRtr: No

On Linux the

ip -6 addr

command gave me

tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
    inet6 2001:db8:1:0:b0fd:f92b:8362:577b/64 ...
    inet6 2001:db8:1:0:8024:bff:fe45:840c/64 ...
    inet6 fe80::8024:bff:fe45:840c/64 ...

tap2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
    inet6 fe80::9863:1eff:fe13:1408/64 ...

The TSO output means

  • Query Neighbor cache for 2001:db8:1:0:8024:bff:fe45:840c. The address is one of the addresses on the remote end of the connection. There is an entry because some traffic came via the address.
  • IntfName: IFPORTCP6 The z/OS Interface name used to create the defintion
  • IntfType: IPAQENET6 the OSA-Express QDIO interfaces statement
  • LinkLayerAddr: 82240B45840C
  • State: Reachable Other options can include stale, which means z/OS has not heard anything from this address for a while
  • Type: Router
  • AdvDfltRtr: No. The information passed in the Router Advertisement, said this was connection does not Advertise a Default Router(AdvDfltRtr).

From the NETSTAT ND output we can see data has been received from

  • IFPORTCP6:2001:db8:1:0:8024:bff:fe45:840c
  • IFPORTCP6:fe80::8024:bff:fe45:840c
  • JFPORTCP6:fe80::9863:1eff:fe13:1408

To get data to flow down the 2001…. address I had to use

ping -I 2001:db8:1:0:8024:bff:fe45:840c 2001:db8:1::9

Where the -I says use the interface address.

You can get information about bytes processed by interface (not by address) using the TSO NETSTAT DEVLINKS command.

Why has my packet suddenly decided to go over there? Grrr

As one of the many problems I had trying to get IPV6 routing to work, I found that I could run a configuration script, and it would all work successfully (including a ping) – then a few seconds later, a manual ping would not work.

I had a shell script to display all my IP configuration information, to display the route information all in one line… including

option="-6 -o"
echo "==ROUTE"
ip $option route  |awk '{ print "ROUTE", $0 } '

When it worked, my route was 

ROUTE ::1 dev lo proto kernel metric 256 pref medium
ROUTE 2001:db8::/64 dev enp0s31f6 proto ra metric 100 pref medium
ROUTE 2001:db8::/64 dev enp0s31f6 proto kernel metric 256 expires 86395sec pref medium
ROUTE 2001:db99::/64 dev enp0s31f6 proto ra metric 100 pref medium
ROUTE 2a00:23c5:978f:6e01::/64 dev wlp4s0 proto ra metric 600 pref medium
ROUTE fe80::/64 dev enp0s31f6 proto kernel metric 100 pref medium
ROUTE fe80::/64 dev wlp4s0 proto kernel metric 600 pref medium
ROUTE default via fe80::966a:b0ff:fe85:54a7 dev wlp4s0 proto ra metric 600 pref medium
ROUTE default via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra metric 1024 expires 132sec hoplimit 64 pref medium
ROUTE default via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra metric 20100 pref medium

Some interesting information in this display (see the man page here)

ROUTE 2001:db8::/64 dev enp0s31f6 proto ra metric 100 pref medium
  • 2001:db8::/64, this is the prefix of length 64 bits so 2001:db8:0:0. It is the address range 2001:0db8:0000:0000…. where …. is 0000:0000:0000:0000 to ffff:ffff:ffff:ffff
  • dev enp0s31f6 is the device (also known as the interface)
  • proto ra. The protocol was installed by Router Discovery protocol
  • metric 100. When there is a choice of valid routes, the lower the metric, the more it is favoured.
  • pref medium. Preference medium (out of high, medium, low).

Another interesting one is

ROUTE default via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra metric 20100 pref medium
  • If no other routes are found use the default, route, via enp0s31f6, installed by router discovery protocol(ra).
  • The metric is 20100 – so a low priority value.

A short while later, when ping failed, there was an additional route

ROUTE default via fe80::966a:b0ff:fe85:54a7 dev wlp4s0 proto ra metric 600 pref medium

With this the metric is 600 – which is lower than 20100 from before, so packets were sent to the wireless interface – which did not know what to do with them, and dropped them!

Solution

I used 

sudo ip -6 route replace default via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra metric 200 pref medium

where the metric value was lower than the metric value for the wireless connection, and ping worked.

The above solution worked, but the IP v6 address changed from day to day. The following worked better as it has a permanent global address.

sudo ip -6 route replace default via 2001:db8::2 dev enp0s31f6 proto ra metric 200  pref medium

where 2001:db8::2 is the IP address of the connection on the remote, server, machine. This was done using 

sudo ip addr add 2001:db8::2/64 dev eno1

Getting IP v6 static routing from Linux to/from z/OS

For me this was an epic journey, taking weeks to get working. It was like a magical combination lock, which will not open unless all of the parameters are correct, today has an ‘r’ in the month, and you are standing on one leg. Once you know the secrets, it is easy.

With IP V6 there is a technology called dynamic discovery which is meant to make configuring your IP network much easier. Each node asks the adjacent nodes what IP addresses they have, and so your connection to the next box magically works. I could not get this to work, and thought I would do the simpler task of static configuration – this had similar problems – but they were smaller problems.

There were two three four five six seven key things that were needed to get ping to work in my setup:

The key things

Allow forwarding between interfaces

On Linux

sudo sysctl -w net.ipv6.conf.all.forwarding=1

The documentation says “… conf/all/forwarding – Enable global IPv6 forwarding between all interfaces”.

Clearing the cache

Routing and neighbourhood definitions are cached for a period. If you change a definition, and activate it, an old definition may still be used. I found I got different results if I rebooted, re-ipled, or went for a cup of tea; it worked – then next time I tried it with the same definitions, it did not work. Clearing the routing and neighbourhood cache made it more consistent.

On z/OS use V TCPIP,,PURGECACHE,IFPORTCP6

On Linux use sudo ip -6 neigh flush all

Put a delay between creating definitions and using them.

I had a 2 second delay between creating a definition, and using it, which helped getting it to work. I think data is propagated between the system, and issuing a ping or other command immediately after a definition, was too fast for it,

A timing window

I had scripts to clear and redefine the definitions. Some times if I ran the laptop script then the server script, then ping would not work. If I reran the laptop script, then usually ping worked. Sometimes I had to rerun the server script.

The default route would often change.

The wireless connection to the server was unreliable. There would be a route from my laptop to the server via the wireless. Then a few minutes later the connection to the server would stop, and so alternate routes had to be used, because traffic via the wireless would be dropped.

I got around this problem, by explicit coding of the routes and not needing to use the default definitions. (Also disabled the wireless connection while debugging)

The correct route syntax

I found I was getting “Neigbor Solicitation” instead of the static routing. To prevent this the route on the laptop needed the via… 

sudo ip -6 route add 2001:db8:1::9/128 via 2001:db8::2 dev enp0s31f6

and not

sudo ip -6 route add 2001:db8:1::9/128                dev enp0s31f6

See Is “via” needed when creating a Linux IP route?

The z/OS IP address kept changing across IPLs

Why is my z/OS IP address changing when using zPDT, and routing does not work?

Configuration

  • The laptop had an Ethernet connection to the server.
  • The server had an Ethernet like connection to z/OS. This was a tunnel(tap1), looking like an OSA to z/OS

The addresses:

Laptop Ethernet (enp0s31f6)2001:db8:::7
Server Ethernet (eno1)2001:db8:::2
Server Tunnel (tap1)2001:db8:1::3
Z/OS interface (ifacecp6)2001:db1::9

The Laptop side had prefix 2001:db8:0::/64, the z/OS side had prefix 2001:db8:1::/64 . See One minute topic: Understanding IP V6 addressing and routing if these numbers look strange.

Definitions

z/OS routing definitions

BEGINRoutes 
;     Destination      FirstHop          LinkName   Size 
ROUTE default6         2001:db8:99::3    IF2        MTU  1492
ROUTE 2001:db8:99::/64 2001:db8:99::3    IF2        MTU 5000 

ROUTE 2001:db8::/64    2001:db8:1::3     IFPORTCP6  MTU 5000 
ROUTE 2001:db8:1::/64  2001:db8:1::3     IFPORTCP6  MTU 5000 
                                                                              
ENDRoutes

Where

  • default6 says if no other routes match, then send the traffic down IF2 connection. At the remote end of the IF2 connection, it has IP address 2001:db8:99::3.
  • Traffic for 2001:db8:99::/64 should be sent down interface IF2 – which has an address 2001:db8:99::3 at the remote end
  • Traffic for 2001:db8::/64 (2001:db8:0::/64) should be sent down interface IFPORTCP6 which has address 2001:db8:1::3 at the remote end.
  • Traffic for 2001:db8:1::/64 should be sent down interface IFPORTCP6 which has address 2001:db8:1::3 at the remote end.

I needed a route for both 2001:db8::/64 and 2001:db8:1::/64 as one was the route to the laptop, the other was the route to the Linux server.

Linux Server machine

On my Linux machine I had

from ip -6 addr

tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
 inet6 2001:db8:1::3/64 scope global 
    valid_lft forever preferred_lft forever
 inet6 2001:db8::3/64 scope global 
    valid_lft forever preferred_lft forever
 inet6 fe80::e852:31ff:fe0f:81da/64 scope link 
    valid_lft forever preferred_lft forever

I used the global address 2001:db8:1::3 in my z/OS routing statement.

The documentation implies I should use the link-local address fe80::e852:31ff:fe0f:81da in my static z/OS definitions, but I could not see how to use this, as it changed every time I ipled my z/OS. This means I need to explicitly define an address on Linux for this connection ( 2001:db8:1::3).

Linux Server definitions

On my Linux server I defined static definitions. 

sudo sysctl -w net.ipv6.conf.all.forwarding=1

# clear the state every time
sudo ip -6 route flush root 2001:db8:1::/64
sudo ip -6 route flush root 2001:db8::/64
sudo ip -6 neigh flush all 

# define the interface to z/OS
sudo ip -6 addr del 2001:db8:1::3/64 dev tap1
sudo ip -6 addr add 2001:db8:1::3/64 dev tap1

sudo ip -6 addr del 2001:db8::2/64 dev eno1
sudo ip -6 addr add 2001:db8::2/64 dev eno1


sudo ip -6 route del 2001:db8::/64 dev eno1
sudo ip -6 route add 2001:db8::/64 dev eno1

sudo ip -6 route del 2001:db8:1::9 dev tap1
sudo ip -6 route add  2001:db8:1::/64   dev tap1

# sudo traceroute -d -m 2 -n -q 1 -I    2001:db8::7 
# ping 2001:db8::7 -c 1 -r
# ping 2001:db8:1::9 -c 1 -r

This script grew as I added all of the options to get it to work.

The statements are

sudo sysctl -w net.ipv6.conf.all.forwarding=1

This enables the cross interface traffic.

sudo ip -6 route flush root 2001:db8:1::/64
sudo ip -6 route flush root 2001:db8::/64
sudo ip -6 neigh flush all

These clear the routing for the two addresses, and for the neighbourhood cache. I do not know if these are required, without them the results were not consistent.

#give the interface to z/OS an explicit address
sudo ip -6 addr del 2001:db8:1::3/64 dev tap1
sudo ip -6 addr add 2001:db8:1::3/64 dev tap1


#give the connection to the Laptop an explicit address
sudo ip -6 addr del 2001:db8::2/64 dev eno1
sudo ip -6 addr add 2001:db8::2/64 dev eno1

These deleted then created global addresses for the server end of the interfaces.

sudo ip -6 route del 2001:db8::/64 dev eno1
sudo ip -6 route add 2001:db8::/64 dev eno1


sudo ip -6 route del 2001:db8:1:: dev tap1
sudo ip -6 route add 2001:db8:1:: dev tap1

These deleted and created routes the traffic to the interfaces. I could have used route rep…

Linux Laptop definitions

#Give the ethernet connection to the server an explicit address
sudo ip -6 addr add 2001:db8::19 dev enp0s31f6

#create the route to the server using the via
sudo ip -6 route del 2001:db8:1::/64 dev enp0s31f6
sudo ip -6 route add 2001:db8:1::/64 via 2001:db8::2 dev enp0s31f6

I needed to specify

  • an explicit to the address of the interface to the server, so it could be used as a destination from z/OS.
  • the route to get to the server. I needed to specify the via, so the static route was used directly. Without the via, it tried to use Neighbourhood discovery.

Pinging

For “ping” to work, the packet has to reach the destination and the reply get back to the originator. See Understanding ping and why it does not answer.

If I pinged 2001:db8:1::9 (z/OS) from the Linux server (the end of the IFPORTCP6 connection) the traffic came from address 2001:db8:1::3, The reply was sent back using the matching 2001:db8:1::/64 definitions.

If I pinged 2001:db8:1::9 (z/OS) from my laptop, through the Linux server to z/OS, the traffic came from address 2001:db8::7. The reply was sent back using the matching 2001:db8::/64 definitions.

If I pinged 2001:db8::7 (laptop) from z/OS it was sent back using the matching 2001:db8::/64 definitions.

One minute topic: Understanding IP V6 addressing and routing

Understanding IP addressing and routing is not difficult, but there are some subtleties you need to be aware of.

This is a good place to start.

IP V4 addressing

An IP V4 address is like 192.6.24.56, where each number is between 0 and 255 inclusive (8 bits). You see routing statements like 192.6.24.9/24 which means the left 24 bits are significant for routing. 192.6.24.99/24 is routed the same as 192.6.24.22/24 because 192.6.24.n/24 refers to the range 192.6.24.0 to 192.6.24.255.

IP V6 addressing

IP V6 addresses are like abcd:efgh:ijkl:mnop:qrst:uvwx:yzab:cdef – or 8 groups of 4 hex digits.

Within each group leading zeros can be dropped.

The longest sequence of consecutive all-zero fields is replaced with two colons (::).

fe80:0000:0000:0000:11ad:b884:0000:0084 can be written fe80:0:0:0:11ad:b884:0:84 which can be written fe80::11ad:b884:0:84, which is a more manageable number to use.

I tend to use addresses like fe00::4 because they are short!

IP V6 prefixes

An Internet Service Provider (ISP) provides connectivity to its users. Each enterprise customer, or end user, is allocated a prefix, usually 48 digits long, and you have 16 digits for routers (the subnet) within your organisation. Normally the total prefix length is 64.

At home with a wireless router, my laptops address is 2a00:dddd:ffff:1111:65fa:229:f923:84b8. 2a00:dddd:ffff from my ISP and my subnet is 1111 within my organisation.

An address like 2001:db8::/64 is the range 2001:db8:0:0:0:0:0:0 to 2001:db8:0:0:ffff:ffff:ffff:ffff.

An address like 2001:db8::9/128 is the single address 2001:db8:0:0:0:0:0:9, because all digits are significant.

There are different levels of IP V6 addresses

  • Addresses starting with fe80::, called link-local addresses, are assigned to interfaces for communication on the attached link. If you think of lots of machines on an Ethernet connection, they have a fe80… address. They tend to be used internally by Dynamic Routing. I haven’t explicitly used one.
  • “global” addresses – or not on an Ethernet cable.
    • fc00::/7 Unique Local Addresses (ULA) – also known as “Private” IPv6 addresses. They are only valid within an enterprise.
    • 2…::/16 Global Unique Addresses (GUA) – Routable IPv6 addresses. These addresses allow you to access resources, such as web sites, outside of your domain. My ISP provides me with an address 2a00:abcd:….

Reserved addresses

Some addresses are reserved, for example

  • 2001:db8::/32 is reserved for documentation, these addresses do not leave your enterprise.
  • fe80::/10 Addresses in the link-local prefix. These are allocated to the “cable” or connection between two nodes. Two different “network cables” can have the same fe80… address because they are on different cables.
  • fc00/12 are addresses which are within your enterprise. Routers will not send these addresses out of its domain.
  • ff02::1 Multicast, all nodes in the link-local
  • ff02::2 Muticast, all routers in the link-local
  • ff05::2 All routers in the site-local (in your machine)

See here for a more complete list.

Defining an address

If I define an address for connection (on Linux) I can use

  • sudo ip -6 addr add 2001::99 dev tap1, this is one address. When displayed this gives 2001::99/128
  • sudo ip -6 addr add 2001::999/64 dev tap1, this is an address, and when used in routing, use the left 64 bits. When displayed this gives 2001::999/64

Routing

It is important to understand how the prefix affects the routing behaviour.

If I have two Ethernet connections(interfaces) into my laptop. I want traffic for 2001::a:0:0:0 to go via interface A, and traffic for 2001::b:0:0:0 to go via interface B.

If I use

sudo ip -6 route add 2001::a:0:0:0/64 dev A
sudo ip -6 route add 2001::b:0:0:0/64 dev B

then this will not always work. With 2001::a:0:0:0/64 the prefix is 2001:0:0:0:a:0:0:0/64. When comparing a packet with address 2001::b:0:0:0 with each route; both routes are available, because 2001:0:0:0 matches both, and if the packet gets sent to 2001::b:0:0:0 it will be lost.

You either need to move the a/b to make them significant, 2001:0:0:a::/64 and 2001:0:0:b::/64 or use 2001:0:0:0:a::/80 and 2001:0:0:0:b::/80 in the routing statements.

The system needs to be able to route traffic to the correct interface, so you need to be careful how you set up the routing.

Does this make sense?

Specifying

sudo ip -6 route add 2001:db8::99/64 dev eno1 metric 1024 pref medium

is the same as

sudo ip -6 route add 2001:db8::/64 dev eno1 metric 1024 pref medium

because the routing only looks at the left 64 bits. The ::99 is ignored.

Having the :99 makes it a bit more confusing for those stumbling about trying to understand this topic. I’ve had to rewrite some of my blog posts where I use 2001:db8::99/64 in the routing.

In the case of

sudo ip -6 route add 2001:db8::99/128 dev eno1 metric 1024 pref medium

the ::99 is relevant. A packet for 2001:db8::98 would not be routed down this definition because all 128 bits of the route definition are relevant.

Is “via” needed when creating a Linux IP route?

To get static routing working I needed a route like one of

# specific destination
sudo ip -6 route add fc:1::9/128 via fc::2 dev enp0s31f6r
sudo ip -6 route add fc:1::9/128  via fc::2 
#range of addresses
sudo ip -6 route add fc:1::/64 via fc::2  dev enp0s31f6
sudo ip -6 route add fc:1::/64 via fc::2

If I a route without the via 

sudo ip -6 route add fc:1::9/128 dev enp0s31f6

then it ignored my static routing and did Neighbor Solicitation; it asked adjacent systems if they had knew about the IP address fc:1::9. This is an IP V6 Neighbour Discovery facility.

There were hints around the internet that if the next hop address is not specified, then the “next hop router” will try to locate the passed address.

So the short answer to the question is: “yes. You should specify it when using static routing”.

Understanding IP V6 NETSTAT ROUTE on z/OS information

I struggled with the output of the TSO NETSTAT ROUTE command.

Below is an example from my system. The IBM documentation is here

IPv6 Destinations 
DestIP:   Default 
  Gw:     2001:db8:1::3 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UGS               MTU:     1492 
DestIP:   ::1/128 
  Gw:     :: 
  Intf:   LOOPBACK6         Refcnt:  0000000000 
  Flgs:   UH                MTU:     65535 
DestIP:   2001:db8::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   US                MTU:     5000 
DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UD                MTU:     9000 
DestIP:   2001:db8:1::3/128 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UHS               MTU:     5000

DestIP: Default this is statically set up with default6

Gw: 2001:db8:1::3 This is (one of) the IP address at the remote end of the connection.

Intf: IFPORTCP6 The z/OS interface name is IFPORTCP6

The flags are

U – The route is up.

G – The route uses a gateway.

H – The route is to a host rather than to a network.

S – The route is a static route not replaceable by a routing daemon or router advertisements (IPv6).

D – The route was created dynamically by ICMP processing or router advertisements (IPv6) (possibly OMPROUTE).

DestIP: 2001:db8::/64 This is for IP addresses 2001:0DB8:0000:0000:something there are 64 bits in the significant part of the address. This applies to 2001:0DB8:0:0:0:0:0:99 and 2001:0DB8:0:0:FFFF:0:0:99 for example

DestIP: 2001:db8:1::3/128 This says all 128 bits are significant in the address. This is the 2001:db8:0:0:0:0:0:3 and no other address.

RefCnt Reference count – the current number of active users for the route. See below.

Where does an entry come from?

  • An entry can be statically configured between BEGINRoutes… ENDoutes.
  • An entry can be dynamically configured from an adjacent system. For example
    • a prefix entry when using radvd – this defines IP address ranges into or through the z/OS host
    • a route entry when using radvd, this defines IP address ranges going off the host, to the other end of the connection.
  • An entry be generated dynamically from OSPF and RIP. On z/OS these are usually configured with the OMPROUTE address space. See below.

A statically defined entry has an S in the Flgs

A dynamic entry has a D in the Flgs -sometimes – see below.

Why does Gw: sometimes have a value?

Gw: has a value when

  • it was specified in the static definitions
  • the DestIP entry was created dynamically, for example as a route …{} statement in radvd. This is an output entry, so the Gw: is part of the definition.

Note: a radvd prefix… {} entry is inbound, so the gateway is irrelevant.

I see this as it is only relevant for connections out of z/OS. When traffic comes into the host, you do not care which gateway it came from.

What does refcnt mean for a DestIP?

The documentation it says “Reference count (RefCnt): The current number of active users for the route.”

When I pinged z/OS ten times from 2001:db8::7, the RefCnt for DestIP: 2001:db8::/80 increased by 10.

When I pinged z/OS ten times from another address, the RefCnt values were unchanged.

Issuing a traceroute to the system did not increment any values.

I could find no active connections to this interface, so all in all this field is bit of a mystery.

The Linux documentation says The reference count (i.e. attached processes via this socket), so the z/OS meaning may be a partial historical count of usage rather than the number of active users.

What is the default value?

This was a surprise. I had defined a static route using default6, and this was in the netstat route display output.

When I used 

tso netstat route radv

to display the routes added via Router Advertisement it gave me a list including a Default.

IPv6 Destinations 
DestIP:   Default 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db8::/80 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db8:0:0:1::/80 
  Gw:     :: 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db9::/32 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2002:db8::/64 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0

If the Router Advertisment data has AdvDefaultLifetime > 0 for the interface then a “Default” is generated, else no default is generated.

The wireshark trace has 

Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    ...
    Cur hop limit: 64
    Flags: 0xc0, Managed address configuration, ...
    Router lifetime (s): 0

The MTU value is what was passed in via the RA data. Change this value in the radvd configuration, and the z/OS value changes.

When I removed my statically defined default6, this default became active with

DestIP:   Default 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UGD               MTU:     9000

Note: It seems you can have only one active Default, even with IPCONIG6 MULIPATH option. I do not know which default becomes active if you have more than one dynamically defined

The detail option

If you use TSO NETSTAT ROUTE DETAIL you get additional information.

Metric: 00000001 
MVS Specific Configured Parameters: 
  MaxReTransmitTime:  120.000   MinReTransmitTime: 0.500 
  RoundTripGain:      0.125     VarianceGain:      0.250 
  VarianceMultiplier: 2.000     DelayAcks:         Yes d

These numbers look like defaults, and I got them even when not traffic had flowed over the connection.

OMPROUTE

OMPROUTE can

  • Provides some “dynamic” information about default IP6 routes
  • It listens to messages from other routers, and can update the routing tables

Sometimes

Without OMPROUTE, routes that were dynamically created, for example using radvd on Linx, which broadcast z/OS address ranges to z/OS, and advertised “come to me for these address ranges”.

These could be seen as Dynamic, for example the D in UD below.

DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UD                MTU:     9000

If you start OMPROUTE, the “Dynamic address” now come out as “C”

DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UC                MTU:     9000

What is a link-local address and a global address in TCPIP V6?

When reading about routing in IP V6 there is frequent use of the terms link-local and global address. There are several definitions of what these mean – but I think you need to be an expert to understand them. Below are my definitions – they may be right – they may be wrong – but I hope they provide some useful information.

I found this site was a good basic guide to IP addresses.

Some basic information

  • A connection can either support IP V4 or IP V6 configuration
  • Each end of a connection has at least one IP address
  • IP addresses are hierarchical
    • The top 48 bits are global routing
      • If the top value is FE80 then the address is local. A router will not pass this outside.
      • If the top value is 2 then this is a global and the “ISP provider” is within the 48 bits. My Laptop has an IP V6 address like 2a00:1234:5678:9ABC. I used this site to display who my service provider was.
    • The next 16 bits are the subnet
    • The last 64 are the interface id
  • With IP V4, addresses starting 192.168 are local addresses and do not pass through a router – so FE80::… is not that new a concept.
  • A server needs fixed IP address, so clients can access it.
  • A client does not need a fixed IP address, as when it requests a service from the server, it sends its address as part of TCPIP. The server sends the reply back to this address.

The link local address

The link-local address at the end of a connection can have its address allocated:

  • Manually. This means you can give the IP address to your end users (or a DNS)
  • Allocated automatically. This can be based on the MAC address. IP V6 has network and neighbourhood discovery. As a connection becomes active it knows the IP address of the remote end, and can ask the remote end for the configuration it knows about. When the connection is closed, the network forgets the information about the connection, and its neighbourhood. The next time the connection starts, if it has a different IP address makes no difference – the information about the previous IP address has been forgotten
  • The advantage of having a different IP address is it obscures the client’s origin. If you have the same address each time, it is possible to build up a profile of your usage. If you use a different IP address – it makes it harder to do this.

If you have multiple network connections, you may need to help TCPIP send the traffic to the correct destination. For example

ping6 -I tap2 ….

which says sent it down the tap2 interface.

If you use ADCD the find_IO command gives

 FIND_IO for "colin@colin-ThinkCentre-M920s" 
                                                                                                
         Interface IPv4    IPv6           
 Path    Name      Address Address        
------   --------- ---------------------- 
 . 
  A0     tap0     10.1.1.1 fe80::d442:b0ff:fe0c:96ab%tap0  
  A1     tap1     10.1.2.1 fe80::a015:53ff:fe0b:8685%tap1  
d

With FTP

ftp fe80::7:7:7:7%tap1

Setting up IP V6 Linux to z/OS with ADCD

This post follows on from getting TCPIP to work with ADCD (zPDT and ZD&T) and allows me to FTP to z/OS from my Linux machine. There is a Q&A Has anyone configured z/OS running on ZD&T to support IPV6 protocol? but this was not enough information for me.

I’ve written about the IP V6 concepts and how they fit with z/OS.

You need to:

  1. Configure the Linux device map to add an OSA entry in the device map. You specify the path, and z/OS device addresses.
  2. Configure z/OS to support AF_NET6. You can make this change dynamically – or just re-ipl.
  3. Configure TCPIP to add an IFCONFIG6 entry. You need the z/OS AF_NET6 change before the IFCONFIG6 change is accepted. You need to restart TCPIP (or re-ipl).
  4. Configure a VTAM TRLE pointing to the devices defined in the devmap entry. This can be configured dynamically.
  5. Define a TCPIP interface, pointing to the PORTNAME of the TRLE definition. This defines a IP V6 address. The change can be configured dynamically.
  6. Test it!

Configuring the Linux Devmap

Create the Linux devmap entry

[manager]  # tap2 define network adapter (OSA) for communication with Linux
name awsosa 0019 --path=A2 --pathtype=OSD --tunnel_intf=y --tunnel_ip=172.25.1.6 
  # QDIO mode
device 408 osa osa 
device 409 osa osa 
device 40a osa osa

This session has IP V4 address 172.25.1.6, and uses device addresses 408,409 and 40a. It uses path A2.

Restart zD&T to pick up the changes, and re-ipl z/OS.

Configuring z/OS

You have to configure both z/OS and TCPIP to enable TCPIP V6 support.

Update BPXPRM

Use D OMVS,S to show the BPXPRMxx members being used.

Update bpxprmxx with AF_INIT6, by adding the following into a BPXPRMxx member.

NETWORK DOMAINNAME(AF_INET6) 
        DOMAINNUMBER(19) 
        MAXSOCKETS(50000) 
        TYPE(INET)

Check if your AF_INET is INET or CINET (Common INET is used when you have multiple TCPIP stacks), and specify the same value.

Re IPL.

If you mis configure it

DOMAINNUMBER value 19 is required ( see DOMAINNUMBER ) When I used a different value I got

BPXF202I DOMAIN AF_INET6 WAS NOT ACTIVATED FOR FILE SYSTEM
TYPE INET. RETURN CODE = 0000045A, REASON CODE = 743A0000

and, when TCPIP was started

EZZ0695I IPCONFIG6 STATEMENT ON LINE 1 NOT VALID – IPV6 SUPPORT IS NOT
ENABLED

Where 045A is EAFNOSUPPORT The address family is not supported.

Check AF_INET6 is configured

The command D OMVS,PFS gave me

PFS TYPE   ENTRY      ASNAME    DESC      ST    START/EXIT TIME         
 INET      EZBPFINI   N/A       SOCKETS   A     2022/09/20 04.08.00     
 NFS       GFSCINIT   NFSC      REMOTE    A     2022/09/20 04.07.23     
 ZFS       IOEFSCM    N/A       LOCAL     A     2022/09/20 04.07.19     
 AUTOMNT   BPXTAMD    N/A       LOCAL     A     2022/09/20 04.07.19     
 UDS       BPXTUINT   N/A       SOCKETS   A     2022/09/20 04.07.19     
                                                                        
PFS TYPE  DOMAIN        MAXSOCK  OPNSOCK  HIGHUSED                      
 INET     AF_INET6       50000        5         5                      
          AF_INET         64000        8         8                      
 UDS      AF_UNIX         10000        2         2

Check AF_INET6 is in the list.

Configure TCPIP

I added “include user.Z25A.tcpparms(iconfig6)” into the TCPIP PROF.

This member had just

IPCONFIG6

Restart TCPIP.

The only change when TCPIP was restarted was the additional message

EZZ0300I OPENED INCLUDE FILE ‘USER.Z25A.TCPPARMS(ICONFIG6)’

Check the configuration

On Linux the find_io command gave

FIND_IO for "colinpaice@colinpaice" 
                                                                                                
         Interface Current     MAC     IPv4       IPv6           
 Path    Name      State       Address Address    Address        
------   --------- ----------- ------- ---------  ----------------  -------------- 
  A0     tap0      UP, RUNNING fa:...  10.1.1.1   fe80::f85c:c2ff:fe0a:1415%tap0  
  A1     tap1      UP, RUNNING 5e:...  172.26.1.6 fe80::5cda:64ff:feee:eeaa%tap1  
  A2     tap2      UP, RUNNING 4a:...  172.25.1.6 fe80::4850:5fff:fe5e:87c5%tap2

Check the interface is UP, RUNNING

Define a VTAM TRLE

You need to create a VTAM TRLE resource. I invented PORTCP, and created member user.z25a.vtamlst(TRLE).

OSATRL3 VBUILD TYPE=TRL                                                 
OSATRL3E TRLE LNCTL=MPC,READ=(0408),WRITE=(0409),DATAPATH=(040A),      X
               PORTNAME=PORTCP,                                        X
               MPCLEVEL=QDIO

This uses address 0408,0409, and 040a (matching the devmap entry above)

Use V net,act,id=trle to activate it.

Note: USER.Z25A.VTAMLST is in the DD concatenation for //VTAMLST.

Use D NET,TRL to display the defined TRLs. This showed

TRLE = OSATRL3E STATUS = NEVAC

Showing the TRLE above, and the status. It becomes ACTIVE when the TCPIP interface is activated.

Create the TCPIP interface definition

IP V6 uses an interface definition instead of a link and device.

  INTERFACE IFPORTCP6  DELETE 
  INTERFACE IFPORTCP6 
    DEFINE IPAQENET6 
    CHPIDTYPE OSD 
    PORTNAME PORTCP 
    INTFID 7:7:7:7  
    IPADDR FD00::67:1:1

I activated these using

  • v tcpip,,stop,ifportcp6
  • v tcpip,,obey,USER.Z25A.TCPPARMS(IFACE6)
  • You might need v tcpip,,stop,ifportcp6

I found it better to stop the interface before updating it, as sometimes the updates were not all made.

Once these definitions were activated, TSO NETSTAT HOME gave

IntfName:   IFPORTCP6
  Address:  fd00::67:1:1
    Type:   Global
    Flags:
  Address:  fe80::7:7:7:7
    Type:   Link_Local
    Flags:  Autoconfigured

This shows an address fd00::67:1:1 and address fe80::7:7:7:7 based on the INTFID. If you do not specify an INTFID you get a name like fe80::a2:a201:a2:a2a2, based on the chpid (value a2). If the chpid was changed, you would get a different IP address. You can see the chpid from the Linux from_io command, or the z/OS d NET,ID=OSATRL3E,E command.

I could not get any IP address specified in the IPADDR parameter, to work. I could ping to it, but there were no responses.

The interface gets a MAC address based on the CHPID value – for example MACADDRESS: 02A2A2A2A2A2.

NETSTAT ROUTE gave

IPV4 DESTINATIONS                                                     
DESTINATION        GATEWAY         FLAGS    REFCNT     INTERFACE      
127.0.0.1/32       0.0.0.0         UH       0000000000 LOOPBACK       
172.26.1.2/32      0.0.0.0         UH       0000000000 ETH1           
172.26.1.20/32     0.0.0.0         H        0000000000 EZAZCX         
172.26.1.20/32     0.0.0.0         H        0000000000 EZASAMEMVS     
IPV6 DESTINATIONS                                                     
DESTIP:   ::1/128                                                     
  GW:     ::                                                          
  INTF:   LOOPBACK6         REFCNT:  0000000000                       
  FLGS:   UH                MTU:     65535                            
DESTIP:   FD00::67:1/128                                              
  GW:     ::                                                          
  INTF:   IFPORTCP6         REFCNT:  0000000000                       
  FLGS:   UHS               MTU:     1492                             
DESTIP:   FD00::67:1:1/128                                            
  GW:     ::                                                          
  INTF:   IFPORTCP6         REFCNT:  0000000000    
  FLGS:   UH                MTU:     9000             
DESTIP:   FE80::7:7:7:7/128                   
  GW:     ::                                          
  INTF:   IFPORTCP6         REFCNT:  0000000000       
  FLGS:   UH                MTU:     9000

The Linux find_io command gave

                                                                                              
      Interface  Current    MAC       IPv4        IPv6           
 Path Name       State      Address   Address     Address        
----- --------- ----------- --------  ----------  -------------- 
...      
. 
  A0 tap0       UP, RUNNING da:...    10.1.1.1    fe80::...tap0  
  A1 tap1       UP, RUNNING 92:...    172.26.1.6  fe80::...%tap1  
  A2 tap2       UP, RUNNING 42:...    172.25.1.6  fe80::...%tap2

Update the Linux route information

I did this to try to get the IPADDR to work. It did not work, and so is this is not needed.

sudo ip -6 route add fd00::6:1:1/128 dev tap2

Test it!

Use TSO NETSTAT HOME to find the IP V6 address. For example

Address: fe80::7:7:7:7, Type: Link_Local

On Linux use the find_io command to display information about the tunnels to z/OS. Find the tapn matching the chpid being used on z/OS.
Use the

ping fe80::7:7:7:7%tap2

command to send data to z/OS.

The response to the ping will be sent back down the connection the request arrived on.

You can use the tso netstat devlinks(intfname IFPORTCP6 command (where IFPORTCP6 is my interface) to display information about just the specified interface; for example Inbound packets, BytesIn, Outbound packets, BytesOut.

You can use

FTP fe80::7:7:7:7%tap2

then use

tso NETSTAT CONN
or
tso netstat conn (port 21

to see the connections.

You can use NETSTAT ND to display the neighbours. This gave me

Query Neighbor cache for fe80::6a:ffff:feaf:c0e4
IntfName: IFPORTCP6 IntfType: IPAQENET6
LinkLayerAddr: 026AFFAFC0E4 State: Reachable
Type: Host AdvDfltRtr: No

The value fe80::6a:ffff:feaf:c0e4 matches up with the value from find_io on Linux, and

02:6a:ff:af:c0:e4 matches up with the MAC address.

IP V6 concepts and using IP V6 with ADCD

This post follows on from getting TCPIP to work with ADCD (zPDT and ZD&T) and allow me to FTP to z/OS from my Linux machine. There is a Q&A Has anyone configured z/OS running on ZD&T to support IPV6 protocol? but this was not enough information for me.

Background

With IP V4 there is a limit of the number of IP addresses available. IP V6 has many addresses, and so this should not be a problem. There is no smooth migration from IP V4 to IPV6, it is more start with IP V4, run IP V4 and V6 at the same time, move stuff from IP V4 to IP V6, – and possibly (unlikely) run with just IP V6.

Wikipedia has many good articles

  • IP V6 in general
  • IP V4 uses addresses like 192.6.7.1. IP V6 uses addresses like 2001:0db8:0000:0000:0000:ff00:0042:8329 (=2001:db8::ff00:42:8329). See here.
  • Each IP V6 has a local address (link-local) fe80::….
  • An IP V6 address can have :: to mean replace with as many zeros as needed to make this a valid IP V6 address. So 2001:0db8:0000:0000:0000:ff00:0042:8329 can be written 2001:db8::ff00:42:8329. You can only have one :: in a value.
  • TCPIP can support IPV4 and IP V6 at the same time (dual stack)
  • You can wrap an IP V4 address into an IP V6. For example ::ffff:192.0.2.128
  • For security clients often get a “temporary” (or randomised) IP address instead of a hard coded address. This uses a randomiser function with a secret key. This IP address can expire, and a new(different) IP address obtained. This can make it hard(impossible) for a server to do a reverse DNS lookup. This temporary IP address is useful, as it means you cannot be tracked by your IP address.

Other information

  • Each IP node has a IP V4 address and an IP V6 address.
  • When defining connections between systems, it looks like you need at least one IP V4 route, and at least one IP V6 route – I could be wrong.
  • An IPV6 host usually has more than one IP address.

IPV6 has reserved IP ranges

  • 2001:db8::/32 Addresses used in documentation and example source code.
  • fe80::/10 are the link-local unicast [RFC4291] addresses. Addresses within this block should not appear on the public Internet. Your router should not externalise this.
  • fd00::/7 for private internets. These are the unique-local addresses [RFC4193]. Addresses within this block should not appear by default on the public Internet. This means you can use them within your organisation.
  • ffxx is for multicast to all links matching the address. For example ff02::5 is used by the dynamic routing protocol OSPF to say to all routers in the (local) network “hello – anyone there”.

Getting started

The end of each connection needs at least one IP address. If you have 5 connections, you will have at least 5 IP addresses

What is my IP address on Linux ?

You can use hostname -I

which gave me

192.168.1.223
10.1.1.1
172.24.1.6
172.22.1.6
2a00:23c5:8888:9999:0000:1111:2222:3333
2a00:23c5:9999:0000:1111:2222:3333:4444

You can also use ifconfig or ip addr show.

You can also use the z109x find_io command.

What is my IP address on z/OS?

You can use TSO NETSTAT HOME, or the operator command V TCPIP,,NETSTAT,HOME .

This gives information like

LinkName:   ETH1
  Address:  172.26.1.2
    Flags:  Primary
IntfName:   IFPORTCP6
  Address:  fc00::67:1:1
    Type:   Global
    Flags:
  Address:  fe80::a1:a101:a1:a1a1
    Type:   Link_Local
    Flags:  Autoconfigured

This shows

  • 172.26.1.2 an IP V4 address for use within a private network, for connection ETH1.
  • fc00::67:1:1 an IP V6, unique local address, for interface IFPORTCP6.
  • fe80::a1:a101:a1:a1a1a link-local address, for interface IFPORTCP6. The a1:a1… is based on the MAC address of the device. You can override this on z/OS by specifying the INTFID.
  • Note: If you do not specify the INTFID, it will default to the MAC address. If you reconfigure the system, you may get different MAC address, and so the IP address via this interface will change. By specifying the INTFID you can specify what the IP address for this interface, which will not change if the system is reconfigured.

What are the routes on my machine?

IP V4 On Linux ip route or ip -4 route

gives

default via 192.168.1.254 dev wlp4s0 proto dhcp metric 600 
10.1.1.0/24 dev tap0 proto kernel scope link src 10.1.1.1 
169.254.0.0/16 dev wlp4s0 scope link metric 1000 
172.25.1.0/24 dev tap2 proto kernel scope link src 172.25.1.6 
172.26.1.0/24 dev tap1 proto kernel scope link src 172.26.1.6 
172.27.1.0/24 via 172.25.1.6 dev tap2 
192.168.1.0/24 dev wlp4s0 proto kernel scope link src 192.168.1.223 metric 600

IP V6 on Linux ip -6 route gives

:1 dev lo proto kernel metric 256 pref medium
2a00:xxx:xxxx:xxxx::/64 dev wlp4s0 proto ra metric 600 pref medium
fe80::/64 dev tap0 proto kernel metric 256 pref medium
fe80::/64 dev tap1 proto kernel metric 256 pref medium
fe80::/64 dev tap2 proto kernel metric 256 pref medium
fe80::/64 dev wlp4s0 proto kernel metric 600 pref medium
default via fe80::966a:b0ff:fe85:54a7 dev wlp4s0 proto ra metric 20600 pref medium

On z/OS, TSO NETSTAT route or V tcpip,,netstat,route gives

IPv4 Destinations
Destination        Gateway         Flags    Refcnt     Interface
-----------        -------         -----    ------     ---------
Default            172.25.1.6      GS       0000000000 IFPORTCP
127.0.0.1/32       0.0.0.0         UH       0000000000 LOOPBACK
172.26.1.2/32      0.0.0.0         UH       0000000000 ETH1
IPv6 Destinations
DestIP:   ::1/128
  Gw:     ::
  Intf:   LOOPBACK6         Refcnt:  0000000000
  Flgs:   UH                MTU:     65535
DestIP:   fc00::67:1:1/128
  Gw:     ::
  Intf:   IFPORTCP6         Refcnt:  0000000000
  Flgs:   UH                MTU:     9000
DestIP:   fe80::a1:a101:a1:a1a1/128
  Gw:     ::
  Intf:   IFPORTCP6         Refcnt:  0000000000
  Flgs:   UH                MTU:     9000

find_io

FIND_IO for "colinpaice@colinpaice" 
                                                                                                
      Interface  Current MAC      IPv4      IPv6           
 Path Name       State   Address  Address   Address        
----- ---------- ------- ------- ---------- -------------- 
  A0 tap0 UP, RUNNING    ea:...  10.1.1.1   fe80::e8e8:69ff:fe20:435b%tap0  
  A1 tap1 UP, RUNNING    22:...  172.26.1.6 fe80::2090:14ff:fee0:5f20%tap1  
  A2 tap2 UP, RUNNING    22:...  172.25.1.6 fe80::2047:afff:fef7:1caf%tap2

Joining it all up

To FTP from Linux to z/OS, I use

ftp fe80::a1:a101:a1:a1a1%tap1

The fe80::a1:a101:a1:a1a1 came from the z/OS NETSTAT HOME, and is the z/OS end of the connection.

Using another interface (defined with the INTFID 7:7:7:7) I could use

ftp fe80::7:7:7:7%tap2

The ip -6 route command gave me

fe80::/64 dev tap0 proto kernel metric 256 pref medium
fe80::/64 dev tap1 proto kernel metric 256 pref medium
fe80::/64 dev tap2 proto kernel metric 256 pref medium
fe80::/64 dev wlp4s0 proto kernel metric 600 pref medium

So the request for FE80…. can be routed to any of these. I know that it was configured using tunnel interface tap1, so the address to use is fe80::a1:a101:a1:a1a1%tap1.

Once the connection to FTP was established, z/OS TSO NETSTAT CONN gave me 

FTPD1    0000003D ESTBLSH                             
  LOCAL SOCKET:   FE80::A1:A101:A1:A1A1..21           
  FOREIGN SOCKET: FE80::2090:14FF:FEE0:5F20..42572

The request is processed by z/OS address FE80::A1:A101:A1:A1A1 (port 21)

This ties up with DestIP: fe80::a1:a101:a1:a1a1/128 … Intf: IFPORTCP6 which shows the request came in on interface IFPORTCP6

The request came from FE80::2090:14FF:FEE0:5F20, which ties up with fe80::2090:14ff:fee0:5f20%tap1. The request came in over the tap1 interface.