Understanding radvd with IPV6 on Linux.

My two day project to deploy IP V6 dynamic routing, turned into an eight week project before I got it to work.

I am documenting a lot of what I learned – today’s experience is understanding what radvd is and what the configuration options mean. I found a lot of documentation – most of which either assumed you know a lot, or only provided an incomplete description.

High level view of what radvd provides

There are several ways of providing configuration to TCPIP. One way is using radvd (on Linux).

In a configuration file you give

  • An interface name
    • Which IP addresses (and address ranges) are available at the remote end of the connection.
    • Which IP addresses (and address ranges) are available at the local end of the connection

From the information about the local end, the remote end can build its routing tables.

Background IP routing

With IP you can have

  • static routing, where you explicitly give the routes to a destination – such as to get to a.b.c.d – go via xyz interface. You have to do a lot of administration defining the addresses of each end of an interface (think Ethernet cable)
  • dynamic routing, and neighbourhood discovery, where the system automatically finds the neighbours and there is less work for an administrator to do.

With IPv6 you have

  • global addresses with IP addresses like 2001:db8:1:16…..
  • link-local addresses. These are specific to an interface. An Ethernet connection can have many terminals hanging off ‘the bus’. The link-local address is only used within the connection. A different Ethernet cable can use the same address. There is no problem as the addresses are only used with the cable.
  • Neighbour Discovery. Rather than specify every thing as you do with static routing, IP V6 supports Neighbour Discovery, where each node can tell connected nodes, what routes and IP addresses it knows about. This is documented in the specification. This supports
    • Router Advertisement (RA), (“Hello, I’m a router, I know about the following addresses and routes”),
    • Router Solicitation (RS), (“Hello, I’ve just started up, are there any routers out there?”),
    • Neighbour Solicitation(NS) (“Does anyone have this IP address?”), and
    • Neighbour Advertisement (Usually in response to a Neighbour Solicitation, “I have this address”).

What is radvd?

The radvd program is a Router Advertisment Daemon (RADvd) which provides fakes router information – but is not a router.

You specify a configuration file. The syntax is defined here.

You can specify that this interface provides a “default” route. See here.

Example definition

I have a Linux Server, and a laptop running Linux connected by an Ethernet cable.

For the server, the radvd.conf file has

# define the ethernet connection
interface  eno1
{
   AdvSendAdvert on; 
   MaxRtrAdvInterval 60;
   MinDelayBetweenRAs  60; 
   
   prefix 2001:ccc::/64 
   {
   #     AdvOnLink on;
   #     AdvAutonomous on;
     AdvRouterAddr on; 
   };
  
   route 2001:ff99::/64
   {
   #   AdvRoutePreference medium;
   # 3600 = 1 hour
      AdvRouteLifetime 3600;
   
   };
};

The key information is

Comments

Data following #

The name of the interface

interface eno1{….};

prefix statement

prefix 2001:ccc::/64{…}

2001:ccc::/64 is the ipv6 address range or, to say it another way, ipv6 addresses with the left 64 bits beginning with 2001:0ccc:0000:0000. In IP V6 this is known as the prefix.

Basically this prefix statement means “this interface is a route to the specified prefix”.

This creates some addresses on the server for the connection.

eno1    inet6 2001:ccc::e02a:943b:3642:1d73/64 scope global temporary dynamic...       
eno1    inet6 2001:ccc::dbf:5c90:61a6:20ae/64 scope global dynamic mngtmpaddr...
eno1    inet6 2001:ccc::c48b:e8f1:495c:5b52/64 scope global temporary dynamic ...       
eno1    inet6 2001:ccc::2d8:61ff:fee9:312a/64 scope global dynamic mngtmpaddr ...

create routes

The prefix statement creates a route on the server to get to the laptop

2001:ccc::/64 dev eno1 proto ra metric 100 pref medium
2001:ccc::/64 dev eno1 proto kernel metric 256 ...

This says that on the server, if there is a request for an address in the range 2001:ccc::/64 send it via device eno1.

route 2001:ff99::/64 statement.

This passes information to the remote end of the connection, see below. It says “I (the server) know how to route to 2001:ff99::/64”.

It the routing address does not show up in any ip -6 commands on the server.

Polling

The radvd code periodically sends information along the connection to the other end, at an interval you specify. See radvdump below on how to display it.

At the other end of the connection…

At the remote (laptop) end of the connection, using WiresShark to display the data received, shows a Router Advertisement with

Internet Protocol Version 6, Src: fe80::a2f0:9936:ddfd:95fa, Dst: ff02::1
Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    ...
    ICMPv6 Option (Prefix information : 2001:ccc::/64)
    ICMPv6 Option (Route Information : Medium 2001:ff99::/64)
    ICMPv6 Option (Source link-layer address : 00:d8:61:e9:31:2a)

Where

  • fe80::a2f0:9936:ddfd:95fa is the link-local address on the server machine
  • ff02::1 the multicast address “All nodes” on a link (link-local scope)”
  • Prefix information : 2001:ccc::7/64 the prefix of the IP address 2001:ccc:0:0…
  • Route Information : Medium 2001:ff99::/64 This is from the “route” in the radvd configuration file on the Linux Server. It tells the laptop that this connection knows about routing to 2001:ccc::/64 .

From the route information it dynamically creates a route on the laptop to the server.

2001:ff99::/64 via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra metric 100 ...

This creates a route to 2001:ff99::/64 via the IP address fe80::….95fa, from the laptop end of the Ethernet connection with name enp0s31f6 to where-ever the connection goes to (in this example it goes to my server).

If I ping 2001:ffcc::9 from the laptop, it will use this route on its way to the z/OS server.

Connection to z/OS

Within the radvd.conf file is the definition to get to z/OS. This interface looks like an Ethernet (the ip -6 link command gives link/ether). This is for a different radvd configuration file to the earlier example.

interface  tap1
{
   AdvSendAdvert on; 
   MaxRtrAdvInterval 60;
   MinDelayBetweenRAs  60; 
   AdvManagedFlag  on;
   AdvOtherConfigFlag on;
   
   prefix 2001:db8:1::/64
   {
     AdvOnLink on;
     AdvAutonomous on;
     AdvRouterAddr on;
    
   };
   prefix 2001:db8:1::99/128
   {
     AdvOnLink on;
     AdvAutonomous on;
     AdvRouterAddr on;
    
   };

   route 2001:db8::/64
   {
     AdvRoutePreference medium;
     AdvRouteLifetime 3100;
   
   };
};

This says there are IP addresses in the range 2001:db8:1::/64 via this connection. z/OS reads the Router Advertisement and creates a route for them. TSO Netstat route gives

DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UD                MTU:     9000 

The explicit IP address, with the 128 to specify use the whole value, rather than just the prefix,

 prefix 2001:db8:1::99/128
   {
     AdvOnLink on;
     AdvAutonomous on;
     AdvRouterAddr on;    
   };

creates a route in z/OS; from TSO NETSTAT ROUTE

DestIP:   2001:db8:1::99/128 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UHD               MTU:     9000 

In UHD, the U says the interface is Up, the H says this is for a host (a specific end point), and the D says this is dynamically created.

If you try to ping 2001:db8:1::99 from the server, a Neighbour Solicitation request is sent from the server to z/OS, asking “do you have 2001:db8:1:99?”. My z/OS did not have that defined and so did not respond.

When I defined this address on z/OS TCPIP using the obeyfile

INTERFACE IFPORTCP6  DELETE 
INTERFACE IFPORTCP6 
    DEFINE IPAQENET6 
    CHPIDTYPE OSD 
    PORTNAME PORTCP 
    INTFID 7:7:7:7 
                                             
INTERFACE IFPORTCP6 
    ADDADDR 2001:DB8:1::9 
INTERFACE IFPORTCP6 
    ADDADDR 2001:DB8:9::9 
INTERFACE IFPORTCP6 
    ADDADDR 2001:DB8:1::99 

(And used

  • v tcpip,,sto,ifportcp6
  • v tcpip,,obeyfile,USER.Z24C.TCPPARMS(IFACE61)
  • v tcpip,,sta,ifportcp6

to activate it)

After this, the ping was successful because there was a neighbour solicitation for 2001:db8:1::99, and z/OS replied with Neighbour Advertisement of 2001:db8:1::9 saying I have it.

Create a default route

If you specify AdvDefaultLifetime 0 on the interface, this indicates that the router is not a default router and should not appear on the default router list in the Router Advertiser broadcasts. If the value is non zero, the recipient, can use this connection as a default route, for example there is no existing default route. A statically defined default will be used in preference to a dynamically defined one.

Use radvdump to display what it sent in the RA

You can use sudo radvdump to display what is being sent in the Router Advertiser message broadcast over multicast. It looks just like the radvd.conf file.

New (well, old) job name symbol simplifies JCL!

This function has been around since z/OS 2.3, so is not that new!

There are JCL symbols

&SYSJOBNM

You can use it in started tasks and Jobs

S OMPROUTE.OMP4 JCL symbol &SYSJOBNM gives OMROUTE

S OMPROUTE,JOBNAME=OMP4  JCL symbol &SYSJOBNM gives OMP4

You can use this like

//PARM1 DD DISP=SHR,DSN=USER.PARMLIB(X&SYSJOBNM)
//PARM2 DD DISP=SHR,DSN=USER.PARMLIB(Y&SYSJOBNM)  

and so have data sets based on the invoked job name.  This means you can simplify your started task procedures, and may not need the user to specify overrides on the start command.  

Note: If you move to using JOBNAME= instead of specifying a .identifier, you may need to change other definitions. For example TCPIP defines port 520 with jobname OMPROUTE. If you switch to using OMPROUTE,JOBNAME=OMP1 this jobname will need to change to OMP1.

&SYSJOBID

gives a value like STC08460 for a started task.

&SYSUID

gives the userid like IBMUSER

&JOBNAME

gives the job assigned to the address space in which the JCL is converted, so JES2 for me.

How do I see what’s changing in my Linux network configuration?

I was trying to find out what changes were being made to my IP V6 network.

The short answer is the command

sudo nmcli general logging level DEBUG domains ALL

and reset it to the default with

sudo nmcli general logging level INFO domains ALL

You see the last 5 minutes worth of trace using

journalctl -u NetworkManager -S -5m >nw.txt

See man NetworkManager.conf.

Debian Man gives slightly different information including

ALL : all log domains
DEFAULT : default log domains
DHCP : shortcut for “DHCP4,DHCP6”
IP : shortcut for “IP4,IP6”

ADSL : ADSL device operations
AGENTS : Secret agents operations and communication
AUDIT : Audit records
AUTOIP4 : AutoIP operations
BOND : Bonding operations
BRIDGE : Bridging operations
BT : Bluetooth operations
CONCHECK : Connectivity check
CORE : Core daemon and policy operations
DBUS_PROPS : D-Bus property changes
DCB : Data Center Bridging (DCB) operations
DEVICE : Activation and general interface operations
DHCP4 : DHCP for IPv4
DHCP6 : DHCP for IPv6
DISPATCH : Dispatcher scripts
DNS : Domain Name System related operations
ETHER : Ethernet device operations
FIREWALL : FirewallD related operations
INFINIBAND : InfiniBand device operations
IP4 : IPv4-related operations
IP6 : IPv6-related operations
MB : Mobile broadband operations
NONE : when given by itself logging is disabled
OLPC : OLPC Mesh device operations
PLATFORM : OS (platform) operations
PPP : Point-to-point protocol operations
PROXY : logging messages for proxy handling
RFKILL : RFKill subsystem operations
SETTINGS : Settings/config service operations
SHARING : Connection sharing. With TRACE level log queries for dnsmasq instance
SUPPLICANT : WPA supplicant related operations
SUSPEND : Suspend/resume
SYSTEMD : Messages from internal libsystemd
TEAM : Teaming operations
VLAN : VLAN operations
VPN : Virtual Private Network connections and operations
VPN_PLUGIN : logging messages from VPN plugins
WIFI : Wi-Fi device operations
WIFI_SCAN : Wi-Fi scanning operations
WIMAX : WiMAX device operations

it produces data like

<debug> [747.1872] ndisc-lndp[...,"eno1"]: received router advertisement at 17121
<debug> ndisc[...,"eno1"]: scheduling next now/lifetime check: 163 seconds
<debug> ndisc[...,"eno1"]: neighbor discovery configuration changed [GAR]:
<debug> ndisc[...,"eno1"]:   dhcp-level none
<debug> ndisc[...,"eno1"]:   hop limit      : 64
<debug> ndisc[...,"eno1"]:   gateway fe80::a2f0:9936:ddfd:95fa pref medium exp 179.8153
<debug> ndisc[...,"eno1"]:   gateway fe80::9b07:33a1:aa30:e272 pref medium exp 162.8153
<debug> ndisc[...,"eno1"]:   address 2001:bbb::573e:5c69:2ab3:4ae6 exp 81751.8153
<debug> ndisc[...,"eno1"]:   address 2001:ccc::dbf:5c90:61a6:20ae exp 86399.8153
<debug> ndisc[...,"eno1"]:   route 2001:db8::99/128 via :: pref medium exp 86399.8153
<debug> ndisc[...,"eno1"]:   route 2001:ff99::/64 via fe80::a2f0:9936:ddfd:95fa pref medium exp 3099.8153
<debug> ndisc[...,"eno1"]:   route 2001:ccc::/64 via :: pref medium exp 86399.8153
<debug> ndisc[...,"eno1"]:   route 2001:bbb::/64 via :: pref medium exp 81751.8153
<debug>  platform: (eno1) route: append     IPv6 route: 2001:db8::/80 via :: dev 2 metric 100 mss 0 rt-src ndisc
<debug>  platform: (eno1) signal: route   6   added: 2001:db8::/80 via :: dev 2 metric 100 mss 0 rt-src rt-ra

Is “via” needed when creating a Linux IP route?

To get static routing working I needed a route like one of

# specific destination
sudo ip -6 route add fc:1::9/128 via fc::2 dev enp0s31f6r
sudo ip -6 route add fc:1::9/128  via fc::2 
#range of addresses
sudo ip -6 route add fc:1::/64 via fc::2  dev enp0s31f6
sudo ip -6 route add fc:1::/64 via fc::2 

If I a route without the via

sudo ip -6 route add fc:1::9/128 dev enp0s31f6

then it ignored my static routing and did Neighbor Solicitation; it asked adjacent systems if they had knew about the IP address fc:1::9. This is an IP V6 Neighbour Discovery facility.

There were hints around the internet that if the next hop address is not specified, then the “next hop router” will try to locate the passed address.

So the short answer to the question is: “yes. You should specify it when using static routing”.

Understanding ping and why it does not answer.

I’m sure every one reading this post has the kindergarden level of knowledge of ping (when it works), the hard part is when ping does not work. Ping can do so much more.

Pinging 101

If you successfully ping an IP address you get a response like

PING 2001:db8::7(2001:db8::7) 56 data bytes
64 bytes from 2001:db8::7: icmp_seq=1 ttl=64 time=0.705 ms
64 bytes from 2001:db8::7: icmp_seq=2 ttl=64 time=0.409 ms

Forwarding

If the route is through servers, then the servers need to be enabled for forwarding. For example

  • Linux: sudo sysctl -w net.ipv6.conf.all.forwarding=1
  • z/OS: IPCONFIG DATAGRAMFWD

If forwarding is not specified, the ping request will come in on one interface and be thrown away.

Pinging to a multicast address

With multicast you can send the same data to multiple destinations on a connection(interface), or on a host.

You can issue

ping ff02::1%tap1

where

  • ff02::1 is a multi cast address – ff02 is for everything on this link
  • %tap1 says use the interface tap1. Without it, ping does not know which link to send it to.

Wireshark shows the source was fe80::5460:31ff:fed4:4587 which is the address of the interface used to send out the request.

The output was

PING ff02::1%tap1(ff02::1%tap1) 56 data bytes
64 bytes from fe80::5460:31ff:fed4:4587%tap1: icmp_seq=1 ttl=64 time=0.082 ms
64 bytes from fe80::7:7:7:7%tap1: icmp_seq=1 ttl=255 time=3.36 ms (DUP!)
64 bytes from fe80::5460:31ff:fed4:4587%tap1: icmp_seq=2 ttl=64 time=0.082 ms
64 bytes from fe80::7:7:7:7%tap1: icmp_seq=2 ttl=255 time=3.01 ms (DUP!)
64 bytes from fe80::5460:31ff:fed4:4587%tap1: icmp_seq=3 ttl=64 time=0.083 ms
64 bytes from fe80::7:7:7:7%tap1: icmp_seq=3 ttl=255 time=3.22 ms (DUP!)

The z/OS host, has two IP addresses for the interface – and both of them replied.

Pinging from a different address on the machine

I had a server where there as

  • an Ethernet connection to my laptop. The server end of the connection had address 2001:db8::2
  • an Ethernet like connection to z/OS running through a tunnel. The device (interface) was called tap1.

To ping to the multicast address, as if it came from 2001:db8::2, the address of an Ethernet connection on the same machine, you can use

ping -I 2001:db8::2 ff02::1%tap1

Wireshark shows the source was 2001:db8::2.

The output was

PING ff02::1%tap1(ff02::1%tap1) from 2001:db8::2 : 56 data bytes
64 bytes from 2001:db8:1::9: icmp_seq=1 ttl=255 time=3.15 ms
64 bytes from 2001:db8:1::9: icmp_seq=2 ttl=255 time=1.22 ms
64 bytes from 2001:db8:1::9: icmp_seq=3 ttl=255 time=3.21 ms

without the duplicate responses (I do not know why). (It may be due to the global address 2001… compare with the link-local address 9e80…)

You might use this ping from a different address when checking a firewall. The firewall may be restricting the source of a packet.

The problems of ping using a different address on the machine

I had a wireless connection, and an Ethernet connection to my laptop. If I pinged through my server to z/OS, the “return address” was from the wireless connection. z/OS was not configured for this, so the reply to the ping was lost.

Even trying to force the interface id to use with

ping -I enp0s31f6 2001:db8:1::9

The wireless connection was chosen, and ping gave a message

ping: Warning: source address might be selected on device other than: enp0s31f6

I had to give my Ethenet connection an address, and change the route to add the src

sudo ip -6 addr add 2001:db8::7 dev enp0s31f6

sudo ip route replace 2001:db8:1::/64 via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 … src 2001:db8::7

Only then did the ping request get to z/OS – but z/OS did not know how to get back to my laptop!

A normal Wireshark trace

This shows the request and the reply.

Why can ping fail?

If you only get the request data in the Wireshark trace, this means no reply was sent back.

This could be for many reasons including

  • The IP address (2001:db8::1:0:0:9 in the wireshark output above) could not be reached. This could be due to
    • A fire wall dropped it
    • It could not be routed on
    • The address did not exist
  • The response could not be sent back
    • A firewall blocked it
    • There is no routing from the destination back to the originator
    • There is no routing on an intermediate hop

Example of failure

I had a radvd configuration which included

prefix 2001:db8:0:0:1::/64

   {
     AdvOnLink on;
     AdvAutonomous on;
     AdvRouterAddr on;

   };  

   route 2001:db8::/64
   {
     AdvRoutePreference medium;
     AdvRouteLifetime 3100;
   
   };

The 2001:db8:0:0:1::/64 says traffic for 2001:db8:0:0 is on this system, and traffic for 2001:db8::/64 is off this host.

When ping tried to reply – it tried to send the packet to 2001:db8::/64 – which was routed to the same host and so IP just dropped the packet.

I needed 2001:db8:0:0:1::/80. This says traffic for 2001:db8:0:0:1 is on this system. I also used 2001:db8::/80 which is 2001:db8:0:0:0/80 is off this host. The /80 gave the finer granularity.

Once you know these things, it is obvious. This is called experience.

Another example of a failure

As part of writing up another blog post, I created my network to use only address fc00:…

With this, ping failed to work.

The reason for this was that at the back-end, I could see the source was an 2001:db8:… address, which was not configured in my back-end.

On my front end system my Ethernet device had

2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fc::7/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 2001:db8::4cca:6215:5c30:4f5e/64 scope global temporary dynamic 
       valid_lft 84274sec preferred_lft 12274sec
    inet6 2001:db8::51d8:9a9f:784:3684/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 84274sec preferred_lft 12274sec
    inet6 fe80::9b07:33a1:aa30:e272/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

I deleted this using

sudo ip -6 addr del 2001:db8::4cca:6215:5c30:4f5e/64 dev enp0s31f6

and ping worked!

When I added it back in, ping continued to work. I cannot find which interface address ping uses.

Of course I could have used

ping -I fc::7  fc:1::9

to which interface address to use!

A failure with a hint

I had a WiresShark output

The destination Unreachable had

Internet Control Message Protocol v6
    Type: Destination Unreachable (1)
    Code: 3 (Address unreachable)
    ...
    Internet Protocol Version 6, Src: fc:1::9, Dst: fc::a
    Internet Control Message Protocol v6

This is saying that at the server end of the link to z/OS, where the server end had address fc:1::3 ( see the data at the start of the black line) was unable to deliver the packet to dst: fc::a. This shows the problem is with the server in the middle rather than z/OS.

The solution turned out to be more complex than I first though.

I tried

sudo ip -6 route add fc::/64 dev eno1 via fc::7

but this gave

RTNETLINK answers: No route to host

On the laptop I did

ip -6 addr

which gave me

enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fc::7/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::9b07:33a1:aa30:e272/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

back on the server I replaced fc::7 with fe80::9b07:33a1:aa30:e272

sudo ip -6 route add fc::/64 dev eno1 via fe80::9b07:33a1:aa30:e272

and then ping worked!

Digging into this I found the documentation on Neighbourhood discovery section 8 says

For static routing, this requirement implies that the next- hop router’s address should be specified using the link-local address of the router.

Sometimes

sudo ip -6 route add fc::/64 dev eno1 via fc::7

worked fine. ip -6 route gave

fc::7 dev eno1 metric 1024 pref medium
fc::/64 via fc::7 dev eno1 metric 1024 pref medium
fc:1::/64 dev tap1 metric 1024 pref medium

I think this just goes to show that this is a complex area, and there are things happening which I do not understand.

Understanding IP V6 NETSTAT ROUTE on z/OS information

I struggled with the output of the TSO NETSTAT ROUTE command.

Below is an example from my system. The IBM documentation is here

IPv6 Destinations 
DestIP:   Default 
  Gw:     2001:db8:1::3 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UGS               MTU:     1492 
DestIP:   ::1/128 
  Gw:     :: 
  Intf:   LOOPBACK6         Refcnt:  0000000000 
  Flgs:   UH                MTU:     65535 
DestIP:   2001:db8::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   US                MTU:     5000 
DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UD                MTU:     9000 
DestIP:   2001:db8:1::3/128 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UHS               MTU:     5000 

DestIP: Default this is statically set up with default6

Gw: 2001:db8:1::3 This is (one of) the IP address at the remote end of the connection.

Intf: IFPORTCP6 The z/OS interface name is IFPORTCP6

The flags are

U – The route is up.

G – The route uses a gateway.

H – The route is to a host rather than to a network.

S – The route is a static route not replaceable by a routing daemon or router advertisements (IPv6).

D – The route was created dynamically by ICMP processing or router advertisements (IPv6) (possibly OMPROUTE).

DestIP: 2001:db8::/64 This is for IP addresses 2001:0DB8:0000:0000:something there are 64 bits in the significant part of the address. This applies to 2001:0DB8:0:0:0:0:0:99 and 2001:0DB8:0:0:FFFF:0:0:99 for example

DestIP: 2001:db8:1::3/128 This says all 128 bits are significant in the address. This is the 2001:db8:0:0:0:0:0:3 and no other address.

RefCnt Reference count – the current number of active users for the route. See below.

Where does an entry come from?

  • An entry can be statically configured between BEGINRoutes… ENDoutes.
  • An entry can be dynamically configured from an adjacent system. For example
    • a prefix entry when using radvd – this defines IP address ranges into or through the z/OS host
    • a route entry when using radvd, this defines IP address ranges going off the host, to the other end of the connection.
  • An entry be generated dynamically from OSPF and RIP. On z/OS these are usually configured with the OMPROUTE address space. See below.

A statically defined entry has an S in the Flgs

A dynamic entry has a D in the Flgs -sometimes – see below.

Why does Gw: sometimes have a value?

Gw: has a value when

  • it was specified in the static definitions
  • the DestIP entry was created dynamically, for example as a route …{} statement in radvd. This is an output entry, so the Gw: is part of the definition.

Note: a radvd prefix… {} entry is inbound, so the gateway is irrelevant.

I see this as it is only relevant for connections out of z/OS. When traffic comes into the host, you do not care which gateway it came from.

What does refcnt mean for a DestIP?

The documentation it says “Reference count (RefCnt): The current number of active users for the route.”

When I pinged z/OS ten times from 2001:db8::7, the RefCnt for DestIP: 2001:db8::/80 increased by 10.

When I pinged z/OS ten times from another address, the RefCnt values were unchanged.

Issuing a traceroute to the system did not increment any values.

I could find no active connections to this interface, so all in all this field is bit of a mystery.

The Linux documentation says The reference count (i.e. attached processes via this socket), so the z/OS meaning may be a partial historical count of usage rather than the number of active users.

What is the default value?

This was a surprise. I had defined a static route using default6, and this was in the netstat route display output.

When I used

tso netstat route radv

to display the routes added via Router Advertisement it gave me a list including a Default.

IPv6 Destinations 
DestIP:   Default 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db8::/80 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db8:0:0:1::/80 
  Gw:     :: 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2001:db9::/32 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 
DestIP:   2002:db8::/64 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         MTU:  0 

If the Router Advertisment data has AdvDefaultLifetime > 0 for the interface then a “Default” is generated, else no default is generated.

The wireshark trace has

Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    ...
    Cur hop limit: 64
    Flags: 0xc0, Managed address configuration, ...
    Router lifetime (s): 0 

The MTU value is what was passed in via the RA data. Change this value in the radvd configuration, and the z/OS value changes.

When I removed my statically defined default6, this default became active with

DestIP:   Default 
  Gw:     fe80::dce0:8fff:fe42:127f 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UGD               MTU:     9000 

Note: It seems you can have only one active Default, even with IPCONIG6 MULIPATH option. I do not know which default becomes active if you have more than one dynamically defined

The detail option

If you use TSO NETSTAT ROUTE DETAIL you get additional information.

Metric: 00000001 
MVS Specific Configured Parameters: 
  MaxReTransmitTime:  120.000   MinReTransmitTime: 0.500 
  RoundTripGain:      0.125     VarianceGain:      0.250 
  VarianceMultiplier: 2.000     DelayAcks:         Yes d

These numbers look like defaults, and I got them even when not traffic had flowed over the connection.

OMPROUTE

OMPROUTE can

  • Provides some “dynamic” information about default IP6 routes
  • It listens to messages from other routers, and can update the routing tables

Sometimes

Without OMPROUTE, routes that were dynamically created, for example using radvd on Linx, which broadcast z/OS address ranges to z/OS, and advertised “come to me for these address ranges”.

These could be seen as Dynamic, for example the D in UD below.

DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UD                MTU:     9000 

If you start OMPROUTE, the “Dynamic address” now come out as “C”

DestIP:   2001:db8:1::/64 
  Gw:     :: 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UC                MTU:     9000 

Why Linux is not responding – it’s the flaming file-wall!

I could not ping my Linux Server, and could not issue a traceroute command. It turns out the firewall was blocking the traceroute flow.

This blog posts describes how I checked this, and fixed the firewall problem.

Traceroute sends (by default) a UDP packet to a port address in the range 33434-33523. It usually responds with a “timed out” type response. If there is no response then there is a good chance that the packet is being dropped by a firewall.

See Understanding traceroute (or tracerte).

Using wireshark I could see UDP packets going in to my Linux, but there was no corresponding reply being returned.

When traceroute worked I got the out inbound UDP packet, and the outbound response with “destination unreachable” (which looks like a problem but actually shows normal behaviour) as shown in the data below. Wireshark highlights it with a black background, because it thinks it is a problem.

SourceDestinationDst PortportProtocolInfo
2001:db8::22001:db8::73343452119UDP52119 → 33434 Len=24
2001:db8::7 2001:db8::7 33434 52119 ICMPV6 Destination Unreachable (Port unreachable)

When traceroute failed I only got the inbound UDP packet

SourceDestinationDst PortportProtocolInfo
2001:db8::22001:db8::73343452119UDP52119 → 33434 Len=24

If the packets is blocked by a firewall, then the traceroute output will have “*” as the node name.

Useful Fire Wall (ufw) is documented here.

I was on Ubuntu Linux 20.04.

Display the status of the firewall

sudo ufw status verbose

This gave me

Status: active
Logging: off (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW IN    10.1.1.2                  
22/tcp                     ALLOW IN    Anywhere                  
20,21,10000:10100/tcp      ALLOW IN    Anywhere                  
21/tcp                     ALLOW IN    Anywhere                  
20/tcp                     ALLOW IN    Anywhere                  
22/tcp (v6)                ALLOW IN    Anywhere (v6)             
20,21,10000:10100/tcp (v6) ALLOW IN    Anywhere (v6)             
21/tcp (v6)                ALLOW IN    Anywhere (v6)             
20/tcp (v6)                ALLOW IN    Anywhere (v6)         

By default,

  • incoming data is blocked
  • outbound data is allowed
  • routed data is blocked.

Logging is off, and problems are not reported.

The displays shows there are no rules for UDP – so any incoming UDP request is blocked (quietly dropped = dropped without telling anyone).

You may want to issue the command and pipe the output to a file, ufw.txt, to keep a record of the status before you make any changes. If you make any changes, they persist – even across reboot.

Enable logging to see what is being blocked

sudo ufw logging on

Rerun your traceroute or command.

At the bottom of /var/log/ufw I had (this has been reformatted to make it display better)

Nov 28 12:27:43 colinpaice kernel: [ 3317.641508] [UFW BLOCK] IN=enp0s31f6 OUT= MAC=8c:16:45:36:f4:8a:00:d8:61:e9:31:2a:86:dd SRC=2001:0db8:0000:0000:0000:0000:0000:0002 DST=2001:0db8:0000:0000:0000:0000:0000:0007
LEN=80
TC=0
HOPLIMIT=1
FLOWLBL=924186
PROTO=UDP
SPT=48582
DPT=33434
LEN=40

Wireshark gave me

Frame 4: 94 bytes on wire (752 bits), 94 bytes captured (752 bits) on interface enp0s31f6, id 0   
Ethernet II, Src: Micro-St_e9:31:2a (00:d8:61:e9:31:2a), Dst: LCFCHeFe_36:f4:8a (8c:16:45:36:f4:8a)
Internet Protocol Version 6, Src: 2001:db8::2, Dst: 2001:db8::7
    0110 .... = Version: 6
    .... 0000 0000 .... .... .... .... .... = Traffic Class: 0x00 
    .... .... .... 1110 0001 1010 0001 1010 = Flow Label: 0xe1a1a
    Payload Length: 40
    Next Header: UDP (17)
    Hop Limit: 1
    Source: 2001:db8::2
    Destination: 2001:db8::7
User Datagram Protocol, Src Port: 48582, Dst Port: 33434
    Source Port: 48582
    Destination Port: 33434
    Length: 40
    Checksum: 0x6ebd [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
    [Timestamps]
Data (32 bytes)

From this, we can see the fields match up

  • flow label (0xe1a1a = 924186)
  • source 2001:db8::2 = 2001:0db8:0000:0000:0000:0000:0000:0002
  • destination 2001:db8::7 = 2001:0db8:0000:0000:0000:0000:0000:0007
  • source port 48582
  • destination port 33434.

Port 33434 is used by traceroute, so this is a good clue this is a traceroute packet.

The reason the record was written to the log is [UFW BLOCK]. The firewall blocked it.

The request came in over interface enp0s31f6.

How to enable it.

You can specify different filters, and granularity of parameters.

For example

  • sudo ufw rule allow log proto udp from 2001:db8::2
  • sudo ufw rule allow in on enp0s31f6 log comment ‘colin-ethernet’
  • sudo ufw rule allow proto udp to 2001:db8::7 port 33434:33523 from 2001:db8::2

Where enp0s31f6 is the name of the ethernet link where the traffic comes from.

When running with either of these, I had in the log file

Nov 28 17:03:12 colinpaice kernel: [19847.112045] 
[UFW ALLOW] 
IN=enp0s31f6 OUT= MAC=8c:16:45:36:f4:8a:00:d8:61:e9:31:2a:86:dd SRC=2001:0db8:0001:0000:0000:0000:0000:0009 
DST=2001:0db8:0000:0000:0000:0000:0000:0007 
LEN=60 TC=0 HOPLIMIT=1 FLOWLBL=0 PROTO=UDP SPT=33434 DPT=33440 LEN=20

and the traceroute worked.

Note: The comment ‘…’ is an administration aid to give a description. It does not come out in the logs.

Display the rules

sudo ufw status numbered

gave

Status: active

     To                         Action      From
     --                         ------      ----
[ 1] Anywhere                   ALLOW IN    10.1.1.2                  
[ 2] 22/tcp                     ALLOW IN    Anywhere                  
[ 3] 20,21,10000:10100/tcp      ALLOW IN    Anywhere                  
[ 4] 21/tcp                     ALLOW IN    Anywhere                  
[ 5] 20/tcp                     ALLOW IN    Anywhere                  
[ 6] Anywhere on enp0s31f6      ALLOW IN    Anywhere                   (log)
[ 7] 22/tcp (v6)                ALLOW IN    Anywhere (v6)             
[ 8] 20,21,10000:10100/tcp (v6) ALLOW IN    Anywhere (v6)             
[ 9] 21/tcp (v6)                ALLOW IN    Anywhere (v6)             
[10] 20/tcp (v6)                ALLOW IN    Anywhere (v6)             
[11] Anywhere (v6) on enp0s31f6 ALLOW IN    Anywhere (v6)              (log)
[12] 2001:db8::7 33434/udp      ALLOW IN    2001:db8::2                (log)

There is now a rule [12] for udp to 2001:db8::7 port 33434

You can use commands like

sudo ufw delete 6

to delete a row.

Note: Always display before delete. Having deleted the rule 6 – rule 7 now becomes rule 6, etc.

Now that it works…

Any changes to ufw are remembered across reboots.

You may want to turn off the logging, until the next problem

sudo ufw logging off

and remove the log from the fire wall rules, by deleting and re-adding the rule.

sudo ufw rule delete allow log proto udp from 2001:db8::2

sudo ufw rule allow proto udp from 2001:db8::2

Understanding traceroute (or tracerte)

I was trying to use traceroute to find the route between two nodes and I did not understand the output. Like many things, once you understand it, is obvious.

This is another of the little topics which I thought I understood, and found I did not.

For example

traceroute ibm.com

produces

traceroute to ibm.com (23.39.199.16), 30 hops max, 60 byte packets
 1  bthub.home (192.168.1.254)  4.139 ms  7.528 ms  10.629 ms
 2  * * *
 3  * * *

What does the “*” mean – and why are there 3 “*”?

On Linux there is the command

ip -6 route get to 2001:db8:1::9

which display information like

2001:db8:1::9 from :: via fe80::a2f0:9936:ddfd:95fa dev enp0s31f6 proto ra src 2001:db8::e02a:bfec:a02e:f17 metric 100 pref medium

Traceroute logic

At a conceptual level, the logic of traceroute is:

At the origin

  • Send a UDP packet over a link towards the remote node, with hop limit = 1.
  • Set a timer
  • Wait for the reply (with time out)

At an intermediate node

  • Set hop limit = hop limit -1
  • If hop limit = 0
    • then send a UDP packet back to the originator, giving the IP address of the intermediate node, and information like “Destination unreachable (Port unreachable)”, or “request timed out”.
    • else send the packet over a link towards the remote destination.

Back at the origin

  • Wait for the response. When the response arrives, stop the timer and calculate the duration.
  • Lookup the IP address of the intermediate node, to find the node name.
  • Display original hop count, the name of the intermediate node, IP address of the intermediate node, and duration of the request.

For example

1 colin.Linux.Server (2001:db8::2) 0.988 ms

This gives you information on the first hop in the chain.

Go to the next level.

You can take this further.

  • Repeat the operation multiple times. This allows you to get multiple response times, so you can see the range of responses times, and get an idea of the variation (or consistency) of the response time.
  • Repeat it with hop count = 2,3,4,5… . When the hop count is 1, you get information about the first hop, when the hop count is 2, you get information about the next hop etc.

For example for Linux

traceroute to 2001:db8:1::9 (2001:db8:1::9), 30 hops max, 80 byte packets
1 colin.Linux.Server (2001:db8::2)0.267 ms 0.207 ms 0.140 ms
2 Colin.zOS (2001:db8:1::9) 3.794 ms 6.215 ms 6.920 ms

It gets more interesting.

If you send multiple request, a node may decide to route the request down a different link, so you may get multiple IP addresses for each hop.

What if there is a problem?

Unknown address

Traceroute will report as much as it can. For example 2001:db8:1::10 does not exist.

traceroute to 2001:db8:1::10 (2001:db8:1::10), 30 hops max, 80 byte packets
1 colinpaice (2001:db8::7) 3053.091 ms !H 3052.807 ms !H

This reports as far as it got (colinpaice 2001:db8::7); and !H. On Linux you can have additional information (!H)

  • !H host unreachable
  • !N network unreachable
  • !P protocol unreachable
  • !S source route failed
  • !F fragmentation needed
  • !X communication administratively prohibited
  • !V host precedence violation
  • !C precedence cutoff in effect
  • !<num> ICMP unreachable code <num>

Lost or dropped packets

A intermediate node may not be able to send the response back, for example, a firewall may block (and drop) any UPD packets. The originator times-out waiting for the reply. In this case it reports “*” as the IP address, and cannot provide the duration of the requests. This can occur if the router does not support traceroute, there is no link back to the originator, or there is a firewall which drops packets (going out, or coming back).

More advanced requests

Specify a different home

By default traceroute uses the IP address of the connection it will use to send the packet.

For example I have a system with two interfaces

  • tap1,my end of the connection is 2001:db8:1::3 with the remote end having 2001:db8:1::9 (my z/OS)
  • eno1, my end of the connection is 2001:db8::2 with the remote end having 2001:db8::7 (my laptop)

If I use traceroute to my laptop, I can issue

traceroute 2001:db8::7

by default traceroute uses 2001:db8::2 as its starting point (the IP adddess of the direct connection). I can see this in the wireshark trace.

I can use traceroute to my laptop , and say start from the IP address of the connection to z/OS

traceroute6 -s 2001:db8:1::3 2001:db8::7

-s says use a different starting address 2001:db8:1::3 – corresponding to the link to z/OS as its starting point.

When I used this command, my request was blocked, as my firewall was configured to accept traffic from 2001:db8:2, and not from 2001:db8:1::3.

Make traceroute fail quicker

The traceroute defaults are to try a maximum of 30 hops and wait 5 seconds so you could wait for over 2 minutes if there was a problem). If you know your network is small (at most 3 hops) and responds in under a second, you can use

traceroute -6 -w 2 -q 5 -m 3 2001:db8::7

  • -6 for IP V6 (or just use traceroute6)
  • -w 2 wait for up to 2 seconds
  • -q 1 send out 1 UPD request on each hop
  • -m 3 a maximum of 3 hops.

On z/OS use the tracerte command

tso tracerte 2001:db8::7 (try 1 wait 1 max 2

I’m bored with giving the same reply to messages on the console.

When using z/OS I have to reply to messages, for example at startup and shutdown. After several months of this I was getting bored, and found z/OS has an auto reply capability.

In the SYS1.PARMLIB concatenation you can have AUTORxx members.

For example in SYS1.PARMLIB(AUTOR00) is

/* ARC0380A RECALL WAITING FOR VOLUME volser IN USE BY HOST procid, */ 
/*          FUNCTION function. REPLY WAIT, CANCEL, OR MOUNT         */ 
/*                                                                  */ 
/* Rule: 1                                                          */ 
/*                                                                  */ 
   Msgid(ARC0380A)   Delay(60S) Reply(CANCEL)                          

So if you get message ARC0380A, after 60 seconds it will reply CANCEL. If you are quick you could reply with something else.

If you always reply with the same value you could specify DELAY(0S)… but this means you cannot reply to the message with a different value… DELAY(5S) may be better.

You can specify multiple values in the REPLY(a,b,c), and can use system symbolics such as &SYSNAME.

REPLY(‘system=&SYSNAME.’,‘,option1,option2’)

My parmlib member IEASYS00 includes AUTOR=(00,DT), so members AUTOR00 and AUTORDT are used.

Getting OMPROUTE to work on ADCD

While working on getting IP V6 to work, I had problems getting my routing on z/OS working properly. I spent a few days working with TCPIP dynamic routing. What I have written below is what I have learned, it may well be wrong!

The problem…

With TCPIP you can define static routes. For example

  BEGINRoutes 
  ; Destination        SubnetMask    FirstHop       LinkName    Size 
  ROUTE 9.114.209.0    255.255.255.0    =            ETH2   MTU 1492 
  ROUTE DEFAULT             10.1.1.1                 ETH1   MTU 1492 
  ROUTE DEFAULT6  FE80::8C46:FAFF:FE86:1721    IFPORTCP6  MTU 1492 
  ENDRoutes 

This says

  • To send data IP V4 address9.114.209.* go via the link ETH2.
  • Otherwise send all IPV4 traffic to link ETH1 which has address 10.1.1.
  • Send all IP6 traffic to link IFPORTCP6 – which has an IP address of FE80::8C46:FAFF:FE86:1721.

Maintaining lists like these quickly become unmanageable as the size of the network increases.

Dynamically finding the configuration

To solve this problem TCPIP was enhanced to allow the system to discover information. The OMPROUTE daemon manages this.

When OMPROUTE is started, it queries the TCPIP configuration. It merges this information with its configuration data, and sends a request out to participating interfaces “Hello – who are you connected to”. From this information it can build a pictures of the network.

The “Hello – who are you connected to” could produce a lot of output; you can get information about all the links and routers in your environment.

You can divide your network ( from an OMPROUTE perspective ) into areas, and so only routers within an area exchange information. The areas have a number like 1.1.1.1.

The main backbone has number 0.0.0.0 You can send requests between areas. Routers which connect an area to the backbone are called Area Border Routers.

An Area Border Router can be configured as a default route – if a router does not know where to send the packet, send it to the Area Border Router.

Where is this information stored?

It looks like this configuration information is stored in the TCPIP address space, because if you shut down OMPROUTE, the configuration persists. I do not think the information will be updated, until OMPROUTE is restarted.

Routing packets

When a packet needs to be sent to a destination it uses the information provided by OMPROUTE. It uses two algorithms,

  • OSPF -Open shortest path first. As the state of an element in the network changes – the change is broadcast to all the routers. This means a router can quickly respond to a configuration change.
  • RIP – Routing interface protocol. This uses the number of hops. (I would have thought this was the same as the shortest path.) A RIP router has a 30 second heartbeat with its connected networks. If it has not received a response after 180 seconds, it assumes the network link is down.

It looks like OSPF is the better protocol, it is faster to respond to changes, and listens to changes, rather than using a heartbeat, and looks like it scales better.

Creating OMPROUTE

The documentation is not the clearest. I think it assumes you are an expert before you start.

The IBM documentation is here.

My additional steps included

  • The …TCPIP(PROF2) had “520 UDP OROUTED ; RouteD Server”. This name needs to match your started task proc. Either change the TCPIP profile, or use OROUTED as your proc name.
  • OMPROUTE procedure needs a userid with an OMVS segment.
  • Create RACF started task profile. Profile OMPROUTE.* with CLASS(STARTED). STDATA INFORMATION user=…
  • I created USER.Z25A.tcpparms(omp) and copied TCPIP.SEZAINST(EZAORCFG) into it.
  • Initially I forgot to use define the RACF profile CLASS(OPERCMDS) MVS.ROUTEMGR.OMPROUTE

My OMPROUTE procedure was

//OMPROUTE PROC 
//OMPROUTE EXEC PGM=OMPROUTE,REGION=0M,TIME=NOLIMIT, 
// PARM=('ENVAR("_CEE_ENVFILE_S=DD:STDENV")/') 
//* 
//OMPCFG DD DISP=SHR,DSN=USER.Z25A.TCPPARMS(OMP) 
//STDENV DD * 
RESOLVER_CONFIG=//'ADCD.Z25A.TCPPARMS(TCPDATA)' 
OMPROUTE_DEBUG_FILE=/tmp/logs/omproute.debug 
OMPROUTE_IPV6_DEBUG_FILE=/tmp/logs/omprout6.debug 
OMPROUTE_DEBUG_FILE_CONTROL=1000,5 
//SYSPRINT DD SYSOUT=* 
//SYSOUT   DD SYSOUT=* 
//CEEDUMP  DD SYSOUT=*,DCB=(RECFM=FB,LRECL=132,BLKSIZE=132) 
//  PEND 

The OMPROUTE configuration is in //OMPCFG … USER.Z25A.TCPPARMS(OMP)

Setting up syslogd

If you are not running SYSLOGD, all of the console messages are prefixed with

BPXF024I (TCPIP) Oct 6 10:11:10 omproute 67174435 : …

When running with SYSLOGD, the default syslogd configuration is to write errors to a file (for example into /var/log/2022/10/06/errors) , and throws away the remainder. I added the following to my syslogd configuration and restarted syslogd.

*.OMPROUTE..* /var/log/%Y/%m/%d/omproute

The messages were then put into the file, and were like

Oct 6 11:52:19 S0W1 omproute[67174443]: EZZ7800I OMPROUTE starting
Oct 6 11:52:24 S0W1 omproute[67174443]: EZZ7817I Using defined OSPF protocol 89
Oct 6 11:52:24 S0W1 omproute[67174443]: EZZ7817I Using defined OSPF protocol 89
Oct 6 11:52:24 S0W1 omproute[67174443]: EZZ7838I Using configuration file: dd:OMPCFG=USER.Z25A.TCPPARMS(OMP)

After OMPROUTE started, In this file I had 19 messages. There were 6 message on the job log.

Configuring OMPROUTE

I had a TCPIP configuration with

INTERFACE IFPORTCP6 
  DEFINE IPAQENET6 
  CHPIDTYPE OSD 
  PORTNAME PORTCP 
  INTFID 7:7:7:7 
  IPADDR FE00::66:7:7:7 

DEVICE PORTB MPCIPA 
 LINK ETH2 IPAQENET PORTB 
HOME 172.26.1.9      ETH2 
                                                 
DEVICE PORTA MPCIPA 
 LINK ETH1 IPAQENET PORTA 
HOME 10.1.1.2        ETH1 
                                                 

In my configuration file I had

OSPF_INTERFACE 
       Attaches_to_Area =0.0.0.0 
       IP_Address=10.1.1.2 
       Subnet_mask=255.255.255.0 
       NAME=ETH1 
       ; 
RIP_INTERFACE 
       IP_Address=10.1.1.2 
       Subnet_mask=255.255.255.0 
       NAME=PORTA 
       ; 
IPv6_OSPF 
       RouterID = 7.7.7.7
       ; 
IPV6_OSPF_INTERFACE 
       Attaches_to_Area =0.0.0.0 
       NAME=IFPORTCP6 
       ; 
IPv6_RIP_Interface 
       Name = IFPORTCP6
       ; 

At startup I got messages in the job log

EZZ7871I NO MATCHING INTERFACE STATEMENTS FOR 172.26.1.20 (EZASAMEMVS)                        
EZZ8171I OMPROUTE IPV4 OSPF IS USING ASSIGNED ROUTER ID 10.1.1.2 FROM ETH1 INTERFACE          
EZZ8171I OMPROUTE IPV6 OSPF IS USING CONFIGURED ROUTER ID 7.7.7.7 FROM IPV6_OSPF STATEMENT     

To resolve the problem with EZASAMEMVS I added

 Interface 
      IP_Address=172.26.1.20 
      Name=EZASAMEMVS 
      subnet_mask=255.255.255.0 
      ; 

Which basically says this interface is not involved in any dynamic routing.

Once this was resolved the messages in the syslogd file were

EZZ7800I OMPROUTE starting
EZZ7817I Using defined OSPF protocol 89
EZZ7817I Using defined OSPF protocol 89
EZZ7838I Using configuration file: dd:OMPCFG=USER.Z25A.TCPPARMS(OMP)
EZZ7883I Processing interface from stack, address 10.1.1.2, name ETH1, index 5, flags 441
EZZ7883I Processing interface from stack, address 172.26.1.20, name EZASAMEMVS, index 2, flags ffff8c50
EZZ7871I No matching interface statements for 172.26.1.20 (EZASAMEMVS)
EZZ7977I Processing IPv6 interface from stack, address fe00::66:7:7:7, name IFPORTCP6, index 51, flags 1, flags2 0
EZZ7977I Processing IPv6 interface from stack, address fe80::7:7:7:7, name IFPORTCP6, index 51, flags 1, flags2 2
EZZ7882I Processing static route from stack, destination 9.114.209.0, Mask 255.255.255.0, gateway 0.0.0.0 , table EZBMAIN
EZZ7882I Processing static route from stack, destination 0.0.0.0, Mask 0.0.0.0, gateway 10.1.1.1 , table EZBMAIN
EZZ7882I Processing static route from stack, destination ::, prefixlen 0, gateway fe80::8c46:faff:fe86:1721 , table EZBMAIN
EZZ8023I The RIP routing protocol is Enabled
EZZ8036I The IPv6 RIP routing protocol is Enabled
EZZ8171I OMPROUTE IPv4 OSPF is using assigned router ID 10.1.1.2 from ETH1 interface
EZZ7937I The IPv4 OSPF routing protocol is Enabled
EZZ8171I OMPROUTE IPv6 OSPF is using configured router ID 7.7.7.7 from IPV6_OSPF statement
EZZ7937I The IPv6 OSPF routing protocol is Enabled
EZZ7898I OMPROUTE Initialization Complete

Displaying information from OMPROUTE

On the console

You can use commands like

d tcpip,,omp,ospf,list,all
d tcpip,,omp,ipv6ospf,all
d tcpip,,omprtoute,rttable
f omproute,ospf,list,all
f omproute,rttable
f omproute,rt6table
f omproute,IPV6OSPF,if
f omproute,IPV6OSPF,list,all
f omproute,IPV6OSPF,statistics
f omproute,IPV6rip,list,all
f omproute,ospf,nbrs

All of which will list all information about the OSPF configuration.

There are three types of resource you an display

  • OSPF – d tcpip,,omp,ospf,list,all or d tcpip,,omp,IPV6ospf,list,all
  • RIP – d tcpip,,omp,rip,list,all or d tcpip,,omp,IPV6rip,list,all
  • Other interface – d tcpip,,omp,generic,all or d tcpip,,omp,generic6,all

From netstat

Once OMPROUTE was active NETSTAT ROUTE gave me

IPv4 Destinations
Destination        Gateway         Flags    Refcnt     Interface
-----------        -------         -----    ------     ---------
Default            10.1.1.1        UGS      0000000000 ETH1
9.114.209.0/24     0.0.0.0         US       0000000000 ETH1
10.1.1.0/24        0.0.0.0         UO       0000000000 ETH1
10.1.1.2/32        0.0.0.0         UH       0000000000 ETH1
127.0.0.1/32       0.0.0.0         UH       0000000002 LOOPBACK
172.26.1.20/32     0.0.0.0         H        0000000000 EZAZCX
172.26.1.20/32     0.0.0.0         H        0000000000 EZASAMEMVS
IPv6 Destinations
DestIP:   Default
  Gw:     fe80::8c46:faff:fe86:1721
  Intf:   IFPORTCP6         Refcnt:  0000000000
  Flgs:   UGS               MTU:     1492
DestIP:   ::1/128
  Gw:     ::
  Intf:   LOOPBACK6         Refcnt:  0000000000
  Flgs:   UH                MTU:     65535
DestIP:   fe00::66:7:7:7/128
  Gw:     ::
  Intf:   IFPORTCP6         Refcnt:  0000000000
  Flgs:   UH                MTU:     9000
DestIP:   fe80::7:7:7:7/128
  Gw:     ::
  Intf:   IFPORTCP6         Refcnt:  0000000000
  Flgs:   UH                MTU:     9000

Where key fields are

  • Flgs UO. U=The interface is UP, and The route was created by OSPF (includes OSPF external routes).