One minute mvs: catalogs and datasets.

In days of old, when 64KB was a lot of real storage, to reference a data set you had to specify the data set name and the DASD volume the data set was on. DSN=MY.JCL,VOL=SER=USER00

After this, the idea of a catalog was developed. Just like the catalog in a library, it told you where things were stored. When you created a data set, and specified DISP=(NEW,CATLG), the data set name and volume were stored in the catalog. When you wanted to use a data set, and did not specify the volume, then the catalog was used to find the volume for the data set.

As systems grew and the number of data sets grew, the catalog grew and quickly became difficult to manage. For example if you deleted data sets, the entry was logically removed from the catalog, resulting in gaps in the catalog.

After this a multi level catalog was developed. You have one master catalog. You can have many user catalogs. You define an alias in the master catalog saying for data sets starting with a specific high level qualifier, use that user catalog.

When a userid was created, most system programmers would also create an alias pointing to a user catalog. They may define a user catalog for each user, or a user catalog could be shared by many aliases.

The catalogs are managed by the VSAM component of z/OS.

Entity naming

PDS and sequential files are referred to as datasets. VSAM provides simple database objects,

  • Relative Record ( where you say get me the n’th record)
  • Key sequence. You define a primary index, and you can an index on different columns using an ALTERNATIVE INDEX. 

VSAM uses the term cluster to what you use in you JCL or application. A cluster has a data component, and zero or more index components.

Moving systems.

I have been running on a self contained ADCD system at z/OS level 2.4. I have recently installed a self contained system at z/OS 2.5. How do I get my data sets into the new system?
You can import connect a user catalog into a new (master) catalog, and define an alias in the new master catalog pointing to the user catalog.

When I did this I could then see my COLIN.* data sets. To be able to use the data sets, I need the volumes to be attached to the z/OS system.

Useful IDCAMS commands

In batch you use the IDCAMS program (IDC = prefix for VSAM, AMS is for Access Management Services!)

If you do not specify a catalog, it defaults to the master catalog.

Create a user catalog

//IBMDF  JOB 1,MSGCLASS=H                                   
//S1  EXEC PGM=IDCAMS                                       
//SYSPRINT DD SYSOUT=*                                      
//SYSIN DD *                                                
 DEFINE USERCATALOG -                                       
      ( NAME('A4USR1.ICFCAT') -                             
        MEGABYTES(15 15) -                                  
        VOLUME(A4USR1) -                                    
        ICFCATALOG  -                                       
        FREESPACE(10 10) -                                  
        STRNO(3  ) ) -                                      
        DATA( CONTROLINTERVALSIZE(4096) -                   
              BUFND(4) ) -                                  
        INDEX(BUFNI(4) )                                    
/*

List an entry

You can list information about an entry, such as a data set, or a catalog using the LISTCAT command

LISTCAT ENT(COLIN.USERS) ALL

Listing aliases

You can use the IDCAMS command LISTCAT with alias

LISTCAT ALIAS

which gives a one line per entry list of all of the aliases

ALIAS --------- ADCDA       
ALIAS --------- ADCDB       
ALIAS --------- ADCDC       
ALIAS --------- ADCDD       
...

LISTCAT ALIAS ALL

gives 

ALIAS --------- ADCDA                                                       
     ...          
     ENCRYPTIONDATA                                                         
       DATA SET ENCRYPTION-----(NO)                                         
     ASSOCIATIONS                                                           
       USERCAT--USERCAT.Z24C.USER                                           
ALIAS --------- ADCDB                                                       
    ...      
     ENCRYPTIONDATA                                                         
       DATA SET ENCRYPTION-----(NO)                                         
     ASSOCIATIONS
       USERCAT--USERCAT.Z24C.USER

So we can see that the the alias ADCDA maps to user catalog USERCAT.Z24C.USER

Listing a catalog

The command

LISTCAT CATALOG(USERCAT.Z24C.USER)

gives

CLUSTER ------- 00000000000000000000000000000000000000000000       
   DATA ------- USERCAT.Z24C.USER                                  
   INDEX ------ USERCAT.Z24C.USER.CATINDEX                         
NONVSAM ------- ADCDA.S0W1.ISPF.ISPPROF                            
NONVSAM ------- ADCDA.S0W1.SPFLOG1.LIST                            
NONVSAM ------- ADCDB.S0W1.ISPF.ISPPROF                            
...                           
CLUSTER ------- SYS1.VVDS.VC4CFG1                                  
   DATA ------- SYS1.VVDS.VC4CFG1

Which shows there is a data component of the catalog called USERCAT.Z24C.USER, and there is index component called USERCAT.Z24C.USER.CATINDEX.

The catalog has a data component (USERCAT.Z24C.USER) and an index component USERCAT.Z24C.USER.CATINDEX.

Within the catalog are entries for data sets such as ADCDA.S0W1.ISPF.ISPPROF, and system (DFDSS) dataset SYS1.VVDS.VC4CFG1 - which contains information what is on the SMS DASD volume C4CFG1.

One minute networking: TCP buffer sizes

When data flows over a TCPIP connection there are several factors which control the rate at which data can be sent. You can influence some of these factors.

Data is sent as packets typically of size about 1440 bytes – because old hardware could only support this. You could use larger packets, but you may hit a router which chops it into smaller blocks.

The basic TCPIP flow

Consider a Client Server connection. The client application wants to send some data to a server application

  • The client uses send() to put some data into a TCPIP buffer and returns.
  • TCPIP sends some data (a packet) from this buffer, sets a timer and waits.
  • The server receives the data, end sends back an ACK saying so far I have received this many bytes from you.
  • The application on the server does a receive (if there is no data, the application is suspended until data arrives). If there is enough data to satisfy the receive, the application returns, otherwise it is suspended.
  • At the client end, when TCPIP has received the ACK. It no longer needs the data which has just been acknowledged. It can send more data.
  • If no ACK was received and the timer has timed out, TCPIP resend the data.

There are several parts to this:

  • Putting things into the pipe – the send buffer
  • The pipe
  • Getting things from the pipe, the receive buffer

The send buffer

  • TCPIP has a buffer for its use.
  • The application
    • An application does a send() and passes data to TCPIP.
    • If there is space in the TCPIP buffer, the data is moved into the buffer, and the application returns.
    • If there is not enough space for all of the data, enough data is moved to fill the buffer, and the application waits until more space is available in the buffer.
    • When all of the data has been passed to the TCPIP buffer, the application returns, and can do more application work.
  • TCPIP
    • TCPIP takes a chunk of the buffer (a packet) , sends it over the network, and sets a timer.
    • It can then process another chunk of data, and send it over the network, so there are multiple packets in flight.
    • When the far end has passed the data to the application, it sends the ACK back.
    • The local end, when it has received the ACK for a chunk of data, knows the data has been received by TCPIP at the remote end, it no longer needs to keep a copy of the data, and frees up the space on the buffer.

How big a buffer is needed to get good throughput?

The data is held in the TCPIP buffers; waiting to be sent plus the round trip time; from when the data was put into the TCPIP buffer, to getting the ACK back. This could be 10s of milliseconds. Multiple packets can be in-flight (perhaps 10s or 100s) which improves the throughput. So send 10 packets; wait, when the first ACK is received, send another packet etc., so there are always 10 packets in flight.

If the buffer is too small the application has to wait. Increasing the send buffer size will increase throughput up to a point (when the application does not have to wait) after this point making it larger may make no difference.

As more data is in flight, the connection needs a bigger send buffer.

An application can set the send buffer size using the SETSOCKOPT call. If this is not used, then there will be a TCPIP default send buffer size. On z/OS this is the system wide TCPCONFIG TCPSENDBFRSIZE …. parameter.

The default used to default to 16KB, and currently is typically 64KB. There is a TCPIP enhancement which says if the send buffer size is larger than 64KB, then TCPIP can dynamically increase it if it will improve performance. See Outbound Right Sizing(ORS).

Note: If you change the system wide send buffer size (TCPCONFIG TCPSENDBFRSIZE on z/OS), this will affect all applications that do not set the size using SETSOCKOPT. You should test this before putting it into production because it may affect many applications.

The receive buffer

At the receiving end, TCPIP has a buffer. Data from the network is put into this buffer. After the data has been put into the buffer, TCPIP sends back an ACK with three fields saying

  • so far I’ve received this many bytes from you
  • I’ve sent you this many bytes
  • my buffer has space for this many bytes

An application does a receive to get the data, if there is insufficient data to satisfy the receive, the application can wait, or return just the data in the buffers, depending on the options.

If the receive buffer is full, any incoming data will be thrown away. If the application does receive the data, then does lots of processing on the data, followed by receive more data etc, the receive buffer may fill up. Some applications receive the data, give the data to a subtask to process, immediately do another receive, and so try to keep the receive buffer empty.

If the amount of arriving data is larger than the free space in the buffer, TCPIP will return “no space left in the buffer” as part of an ACK. The sender then knows to wait. When the application receives the data, and makes space, “x bytes are available in the buffer” is sent as part of the ACK, and the sender can start sending data again. This “space available” is known as the Window Size, and helps regulate the flow of data.

If you think about this for several minutes, you will realise that there is a time lag between the receive available buffer size going to zero, and the sender receiving the ACK saying no space in receive buffer. Any in-flight packets may get thrown away, or the end application may get all the data from the buffer. The “no space left in receive buffer” tells the sender to stop sending data until there is space in the buffer, and the sender may then reduce the amount of in-flight data.

Having a zero sized window means there is a problem that the application is not getting the data from the buffer fast enough.

How big a receive buffer is needed to get good throughput?

If the buffer is too small the application has to wait, and packets may be thrown away.

An application can set the receive buffer size using the SETSOCKOPT call. If this is not used, then there will be a TCPIP default receive buffer size. On z/OS this is the TCPCONFIG TCPRCVBFRSIZE …. parameter.

The maximum receive buffer size is specified in TCPMAXRCVBUFRSIZE.

If the receive buffer size is greater than 64B, then a performance enhancement called Dynamic Right Sizing(DRS) can come into action which automatically increases the buffer size up to 2MB.

Inside the pipe

I have described the sender side filling the send buffer for the connection, and the application on the receiver side taking data from the connection’s receive buffer. I’ll look at the pipe in between.

Data is send across the network in packets. The packets are usually small – for example 1500 bytes for Ethernet. Some protocols support larger packet sizes. Data send within a z/OS can have 56KB packet sizes. The Maximum Segment Size (mss) is the maximum size of the user data in a packet.

If a packet is too large for a device, it may be cut into smaller chunks and then passed on – or the packet may just be dropped.

The simplest and slowest transmission is send one packet and wait for the ACK, then send another packet.

It is much more efficient to send multiple packets. For example send 10 packets, when the first ACK comes back (saying the first packet has been received), send the next packet and so on, so there are always 10 packets (or less) in the pipe.

The amount of data on the network is limited by the smaller of the send buffer size and the receive window size. This means you need both a big send buffer, and a big receive buffer to get maximum throughput.

The TCP window is the maximum number of bytes that can be sent before the ACK must be received. If the network is unreliable it is better to keep the window small to reduce the amount of data that needs to be resent after a missing ACK.

Where can I get more information?

I wrote a blog post about tuning MQ channels which gives additional information.

How do I display this buffer information?

On z/OS you can use

  • TSO NETSTAT CONFIG command reports the default receive buffer size, the default send buffer size, and the default maximum receive buffer size
  • TSO NETSTAT ALL (IPPORT nnnn where nnnn is the port number.
  • TCPMON on GITHUB to monitor the buffer and window sizes in near real time.

On Linux

You can use the command

  • ss -im -at ‘( dport = :21 )’ which displays information about connections with destination port of 21.
  • ss -im -at ‘( dst = 10.1.1.2 )’ which displays information about connections with destination ip address of 10.1.1.2

Is there more information available?

There is a lot of information on the web, but it is not usually easy to digest.

I thought this article was clear about the different buffers and windows.

How do I change the buffer sizes?

An application can change them using the SETSOCKOPT call see here options SO_RCVBUF and SO_SNDBUF

With some applications, they have a specific way of setting the buffer sizes

  • MQ for midrange RcvBuffSize etc
  • MQ on z/OS use +cpf RECOVER QMGR(TUNE CHINTCPRBDYNSZ nnnnn)
    +cpf RECOVER QMGR(TUNE CHINTCPSBDYNSZ nnnnn)
  • FTP on Linux -x option

Otherwise the system defaults are used.

Configuring frr on Linux

frr is a network router for Linux systems.

It works well, and has a lot of good documentation about all of the commands and options, but it does doesn’t have a “getting started” section. It is a bit like getting the wiring diagram for a car, when all you to do is get in and drive the car.

Below are some of the things I stumbled across lessons I learned.

Some later definitions override earlier definitions.

With

router ospf6
  ospf6 router-id 6.6.6.6  
  ospf6 router-id 6.6.6.7

The definition used is 6.6.6.7 because it overrides the earlier 6.6.6.6

interface eno1 
   ipv6 nd prefix 2001:db8:6099::/64
   ipv6 nd prefix 2008::/64

 
interface eno1
   description server2 ospf6
   ipv6 ospf6 instance-id 2

The description is description server2 ospf6 because it was the only one specified.

Both of the nd prefix values are used.

What is used where?

If you want to configure a daemon, you needs to specify parameters in different places.

For example, the ospf6 daemon

router ospf6
  ospf6 router-id 6.6.6.6
...  

interface enp0s31f6
  ipv6 ospf6 area 0.0.0.0

The router ospf6 configuration is for the daemon thread.

The information on the interface…. statement is for each interface. This means all of the configuration information for an interface is under the interface…. section.

To display the ospf6 configuration use

show running-config ospf6

It extracts the ospf6 interesting information from the configuration

Current configuration:
!
frr version 8.1
...
hostname Server
!
debug ospf6 zebra
debug ospf6 interface
!
interface eno1
 description server2 ospzz
 ipv6 ospf6 area 0.0.0.0
 ipv6 ospf6 instance-id 2
 ipv6 ospf6 network point-to-point
exit
!

router ospf6
 ospf6 router-id 6.6.1.1
 redistribute static
 redistribute ripng
exit
!
end

and ignored other information like

debug ospf zebra
debug ospf interface 

interface eno1 
   ipv6 nd prefix 2001:db8:6099::/64
   ipv6 nd prefix 2008::/64

The debug ospf… is ignored because this relates to ospf not to ospf6.

The ipv6 nd is ignored because this is not ospf6 related.

One minute networks: MAC address

A MAC address is a Media Access Control address. It has two parts, the manufacturer, and the manufacturer’s unique number. For example on my laptop I have an Ethernet socket. I can see from a wireshark trace that a packet is being broadcast with Ethernet address MicroStarInt_e9:31:2a, or 00 d8 61 e9 31 2a in hex. This was created by Micro Start international.

On a different machine the address is LCFCHeFE 36:f4:8a or 8c 16 45 36 f4 8a. This Ethernet adapter has been provided by LCFC Electronics technology, with serial number 36f48a.

Within an Ethernet switch, there are various broadcasts to devices on the switch, such as

ff02::1all nodes
ff02::2all routers
ff02::5all OSPF (Open Shortest Path First) routers

Using wireshark I can see a broadcast with code ff02::2 which is an IPV6 router Solicitation request from 8c:16:45:36:f4:8a. This is basically saying “have any routers been configured on this Ethernet network, if so, please tell me”. I can map the 8c:16:45:36:f4:8a back to the Ethernet adapter LCFCHeFE 36:f4:8a.

Wireshark has logic to map the Ethernet address prefix to the manufacturer, and convert 8c:16:45 to LCFCHeFE.

One minute networks: Switches, routers, hubs and IP addresses

This blog post is similar to my posts under “One Minute MVS” which aim to provide basic knowledge to understand a topic.

I struggled to understand some of the networking concepts. There is a lot of documentation on the internet, but it did not cover the basics.

The concepts below are based on Ethernet and physical connections. Other connection types such as wireless can be used just as well, but I find the Ethernet picture simple to understand.

What is a router?

A router is used

  • to convert from one network protocol to another protocol
  • to connect bits of network together.

Broadband router

I have broadband to my house, it comes from my telecom’s provider.

  • There is a broadband protocol for the connection to the telephone exchange, for example it connects using my house phone number, not IP address.
  • It converts from broadband to wireless and Ethernet protocols for my various devices around the house.
  • I can connect my laptop to the router, and connect to other devices around the house, on a different network.

What is a switch?

Think of a self contained office. An Ethernet switch is a box with physical sockets for plugging Ethernet cables into. Each person’s computer has a unique address (known as a MAC). Each computer is connected by an Ethernet cable to a physical port on the Ethernet switch. If your computer wants to send information to another computer in the office it sends a request to the Ethernet switch, saying send this information to this MAC address. The switch knows which physical port matches the MAC, and sends the data down the cable plugged into the physical port.

With the configuration described so far, a computer cannot get to the outside internet.

What is hub?

A hub is a very dumb switch, it sends the incoming data to all devices, it is not smart enough to work out which devices to send the data to. It is used when there are a small number of connections.

Does each device need a unique IP address?

Within a network or sub network, each device needs a unique IP address. My laptop has IP address 192.168.1.1 within my network. My neighbour’s laptop has the same IP address within her network. They are on totally separate networks and do not interact, and so they can each have the same IP address.

Each device connected to the internet needs a unique IP address, so the back-end systems can send the data to your device.

A router can be clever and make each device attached to the internet look like a unique device

  • A server has a fixed IP and port, so client applications can find it.
  • Each time a client machine starts, it can be given a different IP address, and when it connects to a server it can use a different port.
  • The server usually does not care what IP address and port the client uses.

If the IP the address of my laptop is 192.168.1.1, this is an internal address and cannot be used on the internet. My router has external address 7.7.7.1.

The router can do some clever mapping

  • If I try to connect from my laptop source address 192.168.1.1 port 100 to the outside world, the router can change the source address to 7.7.7.1 port 206, so looking like a port on the router.
  • If I try to connect from my laptop address 192.168.1.1 with a different port, 200, to the outside world, the router can change this address to 7.7.7.1 port 209
  • If I try to connect from my server with a different address 192.168.1.16 port 100 to the outside world, the router can change this address to 7.7.7.1 port 208.

As far as the internet is concerned requests have come from address 7.7.7.1 with three different ports. When the replies come back, the router maps them back to the internal addresses and ports. The ports numbers 206, 209 and 208 could have been any free port on the router. Tomorrow I may get different numbers.

What is a router – routing

In my house the broadband is on a cable. This is plugged into a router. My red Ethernet switch is also plugged into the router, and my blue Ethernet switch is plugged into another physical socket on the router.

When data from the internet arrives, there are routing rules which say

  • If the traffic is destined for the red switch, then send it to the red switch.
  • If the traffic is destined for the blue switch, then send it to the blue switch.
  • If the traffic is destined for any of the following list of addresses, send it on the broadband cable.
  • Otherwise send it on the broadband cable.

At the far end of the broadband cable is a telephone exchange. This is different sort of router which says if the traffic is for the phone number 01856…. then send to to my house.

The next level of detail

Each computer in the switch is given an IP address, such as 192.168.22.5, another might be 192.168.22.6, where the last number changes with the different computers.

The router has configuration information saying data for 192.168.22.* send it to the red switch.
The blue Ethernet switch has addresses 10.1.1.*

My router has 3 connections, one for broadband, one for the red switch, and one for the blue switch.

If you physically pick up my switch. All of the Ethernet cables coming out of it are part of a subnet. They all have a similar IP address (192.168.1.*), they are in the same subnet.

My router has the following definitions

  • The physical port to the router, has address 192.168.22.25
  • The physical port to the blue router has address 10.1.1.6
  • The physical port to my broadband router is address 192.168.1.222

The top part of these addresses are all different. They are different subnets.

Router routing

If we have a configuration like

Where internet traffic comes into A for the laptop

For sending data to the laptop with IP address 1.1.1.1, you can configure the network

  • On router A. Send traffic for 1.1.1.* to router B. If B is not available send traffic for 1.1.1.* to router C. This provides dual path, or a backup route.
  • On router B. Send traffic for 1.1.1.* to router D.
  • On router C. Send traffic for 1.1.1.* to router D.

Similarly you can configure router D to say to get to the internet you can go via B or C.

Addresses

You could have a configuration like

Where …

  • the red circle has a laptop connected directly to the router. All of the addresses begin with B. The router end of the connection has an address B.2, where B could be 10.1.1, or 192.168.1 etc
  • the blue circle has an Ethernet switch (SW1) attached to the router. All of the elements have an address A…. where A is different to B. The numbers after A are all different; a laptop with address A.3, a laptop with address A.6, the connection to the router has address A.2, and the router end of the connection has address A.8
  • the green circle is is similar to the blue circle, it uses switch SW2. The value of C is different to A and to B.
  • the yellow circle has the connection to the internet. I is the external address of the router and may be something like 9.8.7.23

Let A be 10.1.1 B be 10.2.2 and C be 10.3.3. I could pick any values as long as A, B and C are different, and different to any addresses on the internet.

Within the routing tables in the router I could have

  • All traffic for the address 10.2.2.9 (B.9), send down 10.2.2.2 (B.2). I could also say any traffic for address starting 10.2.2.* send down 10.2.2.2 (B2).
  • All traffic for the address 10.1.1.* , send down 10.1.1.2 (A.2). This is a range of addresses.
  • All traffic for the address 10.3.3.* , send down 10.3.3.8 (C.2). This is a range of addresses.

If I decide to change 10.2.2.* to 11.2.2.* I have to change all of the laptops and the switch within the blue subnet, and the router’s routing tables to reflect the change.

Technological advances

It used to be that a router and switch were hardware devices. These days many routers are just computers running Linux with lots of Ethernet ports.

One minute MVS: Networking subnets.

I’ve understood and used subnets, (in a hand waving way), but found it hard to write down what they are good for, and why we have them. There are many explanations on the web but they all seemed to describe how to use subnets, and not why we need them. Some of what I say below, may be strictly not true, but I hope it gives the right concepts.

  • You can use an Ethernet cable to join two machines. This is not very interesting.
  • You can have an Ethernet router. You plug the Ethernet cable from your machine to one of the ports on the Ethernet router.
  • The Ethernet router has devices attached to it with addresses such as 192.168.1.160, 192.168.1.24, 192.168.1.74. The router handles traffic for 192.168.1.* The IP address is 32 bits long, and the router is configured so that if the top 24 bits of an address are 192.168.1, then pass the traffic to the router. This is written as 192.168.1.0/24. The remaining 8 bits can be used for devices attached to the router, so almost 256 devices. (192.168.1.0 and 192.168.1.255 are reserved).
  • If you had a large building you could configure a router with address 192.169/16 and have 65,000+ devices attached to it. This may not be a good idea.
    • The router sends out management packets to all devices in the subnet saying, for example “does anyone have this IP address”. With many devices the router could spend all its time processing these management packets, and not handling user data
    • You may want to segregate different areas, so addresses 192.168.1.* is for the first floor, and 192.168.2.* is for the second floor. If you want to have a firewall for the first floor it is much easier configuring all traffic going to 192.168.1.* rather than for some machines within 192.168.* and so all users are using the firewall – which may not be what you want.
    • Each floor has a confidential printer. It is easier to configure the printer so that only machines with the same subnet address, IP address 192.168.1.* can send print files to the printer on 192.168.1.22, rather than filter out users on the second floor.
  • With IP V6 there are 128 bits available for subnetting. Mostly a subnet of /64 is used. I have an address 2a00:9999:8888:7777:a0cd:ec92:bceb:91ab/64 so 2a00:9999:8888:7777 is the address of my router (64 bits), and the device on the router is currently a0cd:ec92:bceb:91ab (64 bits).

Basic connectivity

Single point to point cable

My laptop is connected to my server by an Ethernet cable.

I’ve defined the address at each end 10.1.0.2/24 at the laptop and 10.1.0.3/24 at the server. I can ping between the two machines. When I changed the server to have 12.1.0.3/24 there was no connectivity – because they were in different subnets.

Wireless connection – IPV4

My system was configured automatically to have the laptop 192.168.1.222/24 and the server 192.168.1.222/24. These are the same subnet, so traffic goes from my laptop up the wireless connection to the wireless router, and to the server over the wireless connection.

Wireless connection – IPV6

My system was configured automatically to have the laptop a prefix (subnet) of fe80 and specific address c82d:b94c:21fa:3d1c with this this subnet. The server had prefix (subnet) fe80 and specific address c82d:b94c:21fa:3d1c.

The default routing is via device (the wireless router) with prefix (subnet) fe80 and address c82d:b94c:21fa:3d1. These are both “internal to the router” addresses.

Today my laptop also has IP address 2a00:9999:8888:7777:a0cd:ec92:bceb:91ab/64 and my server has address2a00:9999:8888:7777:605a:2d22:5daf:53d7/64. These can be used to contact sites on the internet, because they are external addresses.

Getting out of the subnet.

My server has a connection over virtual Ethernet to z/OS. The server end of this link has address 10.1.3.1/24. If I use wireless connection from my laptop to the server, I cannot easily access this link, because the wireless router does not know about 10.1.3.1 – and I have no way of configuring it.

On Linux I can configure the server to be a software router (radvd), and have a physical Ethernet cable to the it from my laptop. This way I can control the IP routing to and from the server.

You can also use a bridge … but that is an advanced topic.

How does my network interface get an IP address, and is a generated address ok to use?

This article has some good concepts about IP V6 addresses.

What addresses does an interface have?

An interface (think end of an Ethernet cable) typically has one IP V4 address which is usually manually assigned to it, and/or multiple IPV6 addresses.

An interface can have multiple IPV6 addresses – why?

You can explicitly define it

You can assign your own IP address(es) to the interface. You can do this so it has a global address, reachable from other networks.

Dynamic Host Configuration Protocol (DHCP)

If you are using a DHCP client, it communicates with a DHCP server which gives it configuration information, such as an IP address, and subnet, the client then configures the interface.

There has been a shortage of IP V4 addresses for many years. Consider an office which has a “drop-in area”. Rather than give each of the potential uses an IP address, the office is given as many IP addresses as there are desks in the drop-in area. This means perhaps 10 ip addresses are needed instead of 100. This is the advantage of DHCP.

For client devices this is fine. Your machine connects to the server and passes its IP address. The server does some work and sends the response back to the requester.

Tomorrow you may get a different IP address. It works. This means no IP address information is saved across different days. It is stateless.

A server needs either

  • a fixed IP address so clients can contact it,
  • a dynamic address, and an update to the DNS router to say “today megabank.com is on IP address 9.99.99.1”. It can take time to propagate an IP address across the worldwide DNS network.

IPv6 Stateless Address Auto-configuration (SLAAC)

The ideas behind DHCP have been extended in IPV6, the concepts are similar. Stateless Address Auto-configuration is well documented here

Within a network or router domain the system can generate an address, and it is only used within this domain, it could have a different address every time the interface is started. This is known as Stateless Address Autoconfiguration.

When an interface is started it generates an “internal use” address composed of FE00 + a mangled MAC address.

The interface then asks all devices on the local network, does anyone have this address FE00… This is for Duplicate Address Detection (DAD). There should be no reply. If there is a reply, then there is a duplicate address on the network (and the interface generates another address and repeats the process).

The interface then sends out a Router Solicitation request asking “Are there any routers out there?”. A router will then reply giving information. It will contain the “global IP prefix” such as 2001:db8::. which means to the outside world, my the address of the router is 2001:db8::/64. From this information plus the MAC address the interface can generate its IP address. The router also contains the address of the gateway (the address of the router with connections to the outside world) so traffic can now be routed externally.

This means you configure the router, and not the individual devices. If you have many devices this can save you a lot of work. If you change the router configuration, it is propagated to all the devices attached to it.

IPV6 privacy extensions will generate a “random address” (within the subnet). This is to prevent bad actors from monitoring traffic and using your IP address to monitor what sites you visit. By having a different IP address every day means they cannot correlate the traffic for a user.

Does it matter if my address is auto generated?

This is another of the “it depends” answers.

For machines that initiate work, it may not matter that the allocated IP address is different everyday or the same every day. Your IP address is passed with the request to the server. The server does the work, and sends the response back through the network. Assuming the network is configured properly the response will get back to you.

If your machine is a server machine, clients need to know the server’s address. If this changes your clients may need to use DNS to find the new address, and not use the dotted IP address. You may want to allocate a permanent IP address (within the router’s subnet).

Routers, firewalls and filters.

If your machines address is within the router’s subnet, traffic should be able to get to your router and so to your machine. If you change the subnet, traffic may not get to your router.

A firewall can be configured to allow traffic to and from specified addresses (and ports). If you use a different address, either at client or the server, the firewall may not let your packets through. Similarly with a network filter.

I was playing around with configuring IP V6 on my laptop, and the connection to z/OS failed. This was because I had been using one IP address, which I could see flowing to the back-end. When I tried some other configuration, there were more IP addresses allocated to the client, and a different IP address was used as the originator’s IP address in the ping request. The back-end server did not know how to route traffic back to this address, and so the return packets were thrown away and the ping timed out.

You need to be aware which addresses are used for your work. With some IP programs you can bind to a specific local IP address, and force the application to use a particular IP address. For example the Linux Ping command -I interface option.

Setting up for PING can be difficult!

  • Setting up ping within one machine is trivial
  • Setting up ping between two machines is relatively easy
  • Setting up ping with 3 machines can be hard to get right and to get working.

Most documentation describes how to set up PING between two machines, and does not mention anything more complex.

Ancient history (well 1980’s)

To understand modern TCPIP networks, it helps to know about the origins of Ethernet.

Originally with Ethernet there was a bus; a very long cable. You plugged your computer into this bus. Each computer on the bus had a unique 48 bit MAC address. You could send a request to another computer on this bus using their Ethernet address. You could send a request to ALL computers on this bus. These days this might be, send to all computers:”does anyone have this IP address…?”

An Ethernet bus is connected to an Ethernet router which can route packets to other routers.

Ethernet has evolved and instead of a long bus shared by all users, you have a switch device, and you plug the Ethernet cable from your computer to the switch. Conceptually it is the same, but with the switch you get much better performance and usability. You still have routers to get between different Ethernet environments. The request send to all computers:”does anyone have this IP address…?” goes to the switch, and the switch sends it to all plugged in computers.

When you start TCPIP, it configures the Ethernet hardware adapter with the addresses is should listen for.

Other terminology and concepts

  • A router is a networking device which forwards packets between different networks. Packets of data get sent from the originator through zero or more routers to the final destination.
  • A gateway has several meanings
    • It can connect different network types, for example act as a protocol translator, it can have a built-in firewall
    • It can route IP traffic between different IP subnets
    • It can loosely mean a router

Usually a router or gateway is a dedicated device or hardware.

You can have a computer act as a router or gateway; it takes data from one interface and routes it to a different interface. The computer can pass the data through a fire wall, or transform it, for example converting internal IP addresses to external IP addresses (NAT translation).

At a concept level, a computer’s network adapter is configured only to listen for packets which match the IP addresses of the interface, and ignore the rest.

If you want a computer to act as a router or gateway, you need to configure the network adapter to listen to all traffic, and to process the traffic. This is important when routing traffic from computer A through computer B to computer C.

A simple one hop ping, first time

I have set up my laptop to talk to a server over Ethernet.

I have configured my laptop using

sudo ip -6 addr add 2001:7::/64 dev enp0s31f6

This says for any IP V6 address starting with 2001:0007:0000:0000 then try sending it down the Ethernet cable with interface name enp0s31f6.

Ping a non existing address

If I try to ping an address say 2001:7::99 a packet is sent to all computers on the Ethernet bus

from myipAddress to everyone on the Ethernet bus, does anyone have IP address 2001:7::99?

There is no reply because no computer on the bus, has the address.

Ping the adjacent box

On the server, the IP address of the Ethernet cable is 2001:7::2.

If I ping this address, there are the following flows

From myipAddress (and myMAC) to everyone on the Ethernet bus, does anyone have IP address 2001:7::2?
From 2001:7::2 to myipAddress (MAC), yes I have that IP address. My Ethernet MAC address is …

My laptop puts the remote IP address and its MAC address into its neighbour cache, then issues the ping request.

The ping request looks in the neighbour cache find the IP address and MAC address and issues.

From myipAddress to 2001:7::2 at macAddress .. ping request.

The second and later times I issue a ping

The ping request looks in the neighbour cache find the IP address and MAC address and issues the ping

From myipAddress to 2001:7::2 at macAddress .. ping request.

So while the neighbour cache still has the IP of the target, and it’s MAC address, the ping can be routed to the next hop.

Setting up a multi hop ping

I have my laptop connected via a physical Ethernet cable to my server. The server is connected to z/OS via a virtual Ethernet connection. The IP address of the z/OS end of the virtual Ethernet cable is 2001:7::4.

The ping 2001:7::4 request on my laptop, does not work (as we saw above), because TCP asked everyone on the Ethernet bus if it has address 2001:7::4 – and no machine replies.

You need to define the link to the server machine as a gateway or router-like device which can handle IP addresses which are not on the Ethernet bus. You define it like

sudo ip -6 route add 2001:7::/64 via 2001:7::2

This says for any traffic with IP address 2001:0007:0000:0000:* send it via address 2001:7::2.

This requires 2001:7::2 to be known about, and so needs the following to be configured first

sudo ip -6 route add 2001:7::2 dev enp0s31f6

for TCPIP to know where to send the traffic to.

The route add 2001:7::2 dev enp0s31f6 command sends a broadcast on the Ethernet bus asking – does anyone have 2001:7::2. My server replies saying yes I have it. This is the same as for the ping above.

In summary

To send traffic on the same Ethernet bus you use

sudo ip -6 route add 2001:7::2 dev enp0s31f6

To route it via a router, switch, or computer acting as a router or switch you need.

sudo ip -6 route add 2001:7::/64 via 2001:7::2 or
sudo ip -6 route add 2001:7::/64 via 2001:7::2 <dev enp0s31f6>

The computer acting as a router or switch may need additional configuration. For example to allow it to route traffic from one Ethernet bus to another Ethernet bus you need to enable packet forwarding. On Linux to enable forwarding for all interfaces use

sysctl -w net.ipv6.conf.all.forwarding=1

On z/OS you use a static route such as

ROUTE 2001:7::/64 2001:7::3 JFPORTCP6

Where 2001:7::/64 is the range of addresses 2001:7::3 is (one of) the addresses of the interface at the remote end of the cabl, and JFPORTCP6 is the interface name. This is similar to the Linux route statement above.

You might need to set up the firewall. On my server I needed

sudo ufw route allow in on eno1 out on tap2

What IP Address does the sender have?

This is where is starts to get more complex.

Every network connection has at least one IP address.

With IP V6

  • Each interface gets an “internal” IP address such as fe80::9b07:33a1:aa30:e272
  • You can allocate an external address using the Linux command sudo ip -6 addr add 2001:7::1 dev enp0s31f6
  • If the interface has one external IP address configured, then this will be used.
  • If the interface has more than one external address configured then the first in the list may be used (not always).
  • If the interface does not have an external IP address, then TCPIP will find one from another interface, or allocate one for the duration of the request, such as 2a00:23c5:978f:9999:210a:1e9b:94a4:c8e. This address comes from my wireless connection

When my laptop is started, only the wireless connection has an IPV6 address. A ping request to 2001:7::4 had the origin IP address of 2a00:23c5:978f:9999:210a:1e9b:94a4:c8e which is the first address in the list for the wireless connection.

You can tell ping which address to use, for example

ping -I 2a00:23c5:978f:9999:cff1:dc13:4fc6:f21b 2007:1::4 (this fails because the server does now know to send the response to the requester)

I defined a new address for the interface using

sudo ip -6 addr rep 2002:7::1 dev enp0s31f6

I could issue ping -I 2002:7::1 2001:7::4, but this failed to get a response, because the back-end and intermediate nodes, did not know how to get the response back to 2002:7::1

Does this sender address matter?

Yes, because the remote end needs to have a definition to be able to send the response back.

I had my laptop connected to a Linux server over Ethernet, which in turn was connected to z/OS over a virtual Ethernet.

On z/OS I could see the ping request arrive, but it could not send the response back because it did not know how to get to 2a00:23c5:978f:9999….

I configured the laptop end of the Ethernet to give it an IP address 2001:7::1

sudo ip -6 addr add 2001:7::1/64 dev enp0s31f6

I configured the server to laptop to have an IP address of 2001:7::2

sudo ip -6 route add 2001:7::1/128 dev eno1

I configured the server to z/OS with an IP address and a route

sudo ip -6 addr add 2001:7::3/128 dev tap2
sudo ip -6 route add 2001:7::4/128 dev tap2

Now when I did the ping, the originator was 2001:7::1.

I configured the z/OS interface to send stuff back

INTERFACE JFPORTCP6 
    DEFINE IPAQENET6 
    CHPIDTYPE OSD 
    PORTNAME PORTC 
                                   
INTERFACE JFPORTCP6 
    ADDADDR 2001:DB8:8::9 
INTERFACE JFPORTCP6 
    ADDADDR 2001:DB8::9 
INTERFACE JFPORTCP6 
    ADDADDR 2001:7::4 
START JFPORTCP6

and the routing

BEGINRoutes 
;     Destination      Gateway                 LinkName  Size 
ROUTE 2001:7::/64      2001:7::3              JFPORTCP6   MTU 5000 
...
ENDRoutes

This says that all traffic with destination address 2001:0007:0000:000…. send to interface JFPORTCP6. This interface is connected to a gateway with the address of the remote end of the Ethernet (so on the server) of 2001:7::3.

The server machine needs to act as a router between the different Ethernet buses. You can display and configure this using

sysctl -a |grep forwarding
sudo sysctl -w net.ipv6.conf.all.forwarding=1

Packet forwarding on z/OS

By default the z/OS interface only listens for packets with one of the IP addresses of the interface. For z/OS to be able to be a router; accept all packets, and route them to other interfaces you need:

  • IPCONFIG DATAGRAMFWD in the TCP/IP Profile
  • PRIROUTER on the Interface definition . This configures the Ethernet adapter (hardware) so If a datagram is received at this device for an unknown IP address, the datagram is routed to this TCP/IP instance.

But normally if you are running z/OS you use a cheaper, physical router, rather than use the z/OS to do your routing. It might only be people like me who run z/OS on their laptop who want to try routing through z/OS.

Real time monitor for z/OS TCPIP interface statistics

Ive put up on Github a program which provides real time statistics for z/OS TCPIP interfaces.  For example

Interface       ,HH:MM:SS,IBytes,OBytes,IB/sec,OB/sec,,IUP   ,OUP   ,,...
TAP0            ,09:11:12,     0,     0,      ,      ,,     0,     0,,...  
TAP0            ,09:11:22,     0,     0,     0,     0,,     0,     0,,   
TAP0            ,09:11:32,     0,     0,     0,     0,,     0,     0,,  
TAP0            ,09:11:42,   348,   348,    34,    34,,     3,     3,,   
TAP0            ,09:11:52,     0,     0,     0,     0,,     0,     0,,  
TAP0            ,09:12:02,     0,     0,     0,     0,,     0,     0,, 
TAP0            ,09:12:12,     0,     0,     0,     0,,     0,     0,,  
TAP0            ,09:12:22,     0,     0,     0,     0,,     0,     0,,  
TAP0            ,09:12:32,     0,     0,     0,     0,,     0,     0,,  
TAP0            ,09:12:42,     0,     0,     0,     0,,     0,     0,,

Where the values are deltas from the previous time interval

  • IBytes is input bytes during that interval
  • OBytes is output bytes
  • IB/Sec is rate of input bytes
  • OB/Sec is rate of output byte
  • IUP is count of Input Unicast packets
  • OUP is count of Output Unicast packets.

You get one output file per interface.

The output file can be downloaded and used as input to a spread sheet, to graphically display the data.

I’m working on some code to display the global TCPIP stats, but I do not think they are very interesting.  If they are interesting to you, please let me know and I’ll add my code.

Using a 31 bit parameter list in a 64 bit C program.

You can use svc99() to create //DDNAME entries in a job. I used it to dynamically allocate

//COLIN DD SYSOUT=H

from within my C program.

You cannot use a 31 bit C program in a 64 bit C program

If you try to fetch and use a 31 bit C program in a 64 bit program you get

EDC5256S An AMODE64 application is attempting to fetch() an AMODE31 executable. (errno2=0xC4070068)

It is hard to make this to work, because of 64 bit parameters do not work in a 31 bit program. The save area’s are complex. Overall it is easier to just call a 64 bit program from a 64 bit program and do not try to use a 31 bit. You can do it, but you need some assembler glue code.

svc99 works in both 31 and 64 bit modes because at bind time

  • the 31 bit program includes a 31 bit version of svc99
  • the 64 bit program includes a 64 bit version of svc99.

The 64 bit program has come glue code to enable it to call the 31 bit program.

Using a 64 bit program with a 31 bit parameter list

Using svc99() was pretty easy from a 31 bit C program, but the documentation says The __S99parms structure must be in 31-bit addressable storage, which is a bit of a challenge.

For the 31 bit C program, the code is like

int main () { 
  SVC99char1(  sysoutClass, DALSYSOU,'A') 
  SVC99string(ddname,DALDDNAM,"COLIN") 
  struct __S99struc parmlist; 
  struct __S99rbx rbx = 
  { 
    .__S99EID  = "S99RBX", 
    .__S99EOPTS =0xC4,   // issue message before return, and use wto 
    .__S99EVER  =0x01};  // version 
  memset(&parmlist, 0, sizeof(parmlist)); 
  void *tp[3]= { /* array of text pointers */ 
      &sysoutClass, 
      &ddname,
      0}; 
  int  mask = 0x80000000; 
  memcpy(&p->tp[2],&mask,4);  //  make it a null address with high bit on
  parmlist.__S99RBLN = sizeof(parmlist); 
  parmlist.__S99VERB = 01; /* verb for dsname allocation */ 
  parmlist.__S99FLAG1 = 0x4000; /* do not use existing allocation */ 
  parmlist.__S99TXTPP = tp; /* pointer to pointer to text units */ 
  parmlist.__S99S99X  =&rbx; // pointer to extension 
  int rc; 
  rc = svc99(&parmlist);

Where

  • SVC99char1(..) and SVC99string(…) define the parameters for SVC 99
  • struct __S99rbx rbx defines the request block extension which indicates errors to be reported on the job log
  • void tp[3] is the array of parameter for svc 99. The value can be null. The last must have the top bit on, so mask is copied in.
  • parmlist is the svc99 parameter list which points to the other data: the definition, and the request block extension.

Getting it to work in a 64 bit C program.

You have to build a parameter list where all the parameters are in 31 bit storage.

Generate the 31 bit structure

I defined a structure to contain the 31 bit elements, then used ___malloc31 to allocate 31 bit storage for it.

struct pl31 { 
   struct __S99struc parmlist; 
   struct __S99rbx rbx; 
   //  31 bit equivilants of the SVC 99 parameters 
   struct sysoutClass   sysoutClass; 
   struct ddname        ddname     ; 
   void * __ptr32  tp[3];   /* array of text pointers */ 
} pl31; 

char * __ptr32 p31 =(char * __ptr32) __malloc31(sizeof(pl31));

The __ptr32 in char * __ptr32 p31= tells C this is a pointer to 31/32 bit storage.

Set up the parameter list

struct pl31 *  p ; 
p = (struct pl31   *  ) p31; // set up 31 bit pointer to data
memcpy(&p->rbx,&rbx,sizeof(rbx)); // copy from static to 31 bit area
memset(&p->parmlist, 0, sizeof(p->parmlist)); // initialise to 0 
p->parmlist.__S99RBLN = sizeof(p->parmlist); 
p->parmlist.__S99VERB = 01; // verb for dsname allocation 
p->parmlist.__S99VERB = 07; // display 
p->parmlist.__S99FLAG1 = 0x4000; // do not use existing allocation 
p->parmlist.__S99TXTPP =& p->tp ; // pointer to pointer to text units 
p->parmlist.__S99S99X  =& p->rbx ; // pointer to extension

Create the svc99 definitions in 31 bit storage

The macros SVC99string etc generate code like

struct ddname{ 
  short key; 
  short count; 
  short length; 
  char value[sizeof("COLIN   ")]; } 
ddname =  {0x0001,1,sizeof("COLIN   ")-1,"COLIN   "};

so within the 31 bit structure I could use

struct ddname ddname;

to allocate a structure the same shape as the original definition.

I then used a macro SVC99COPY(…); which copies the data from the original, static, definition into the 31 bit structure.

#define SVC99COPY(name) memcpy(&p->name,&name,sizeof(name));

Creating and initialising the rbx

Because the 31 bit storage is dynamically allocated, you cannot use structure initialisation like:

struct pl31 { 
  
    struct __S99struc parmlist; 
    struct __S99rbx rbx =
    {
       .__S99EID = "S99RBX",
       .__S99EOPTS =0xC4, // issue message before return, and use wto
       .__S99EVER =0x01   // version   
   };
   ...
} pl31;

I defined rbx in the mainline code, and copied it into the pl31 structure once it had been allocated.

Set up the pointer to the definitions

p->tp[0]  = &p-> ddname      ; 
p->tp[1]  = &p-> sysoutClass ; 
int  mask = 0x80000000; 
memcpy(&p->tp[2],&mask,4);

The array of tp[] has to be initialised at run time, rather than at compile time because the 31 bit storage is dynamically allocated.

Invoke the svc99()

rc = svc99(&p->parmlist);

So overall – not too difficult.

Making the code bimodal

If you want to make a function which can be used from 31 or 64 bit programs. You need to provide two versions of the code. One compiled as a 64 bit program, the other as a 31 bit program. You could have

  • myprog() for 31 bit programs and
  • myprog64() for 64 bit programs.

or you could specify 

#ifdef __LP64
   #pragma map (myprog, "myprog64")
#else
   #pragma map (myprog, "myprog31")
#endif
...
myprog()

At bind time you need to include the correct code, myprog64 or myprog31.

If could just use the one name, and have two object libraries, and specify the library in JCL, for example

//BIND.OBJLIB  DD DISP=SHR,DSN=COLIN.OBJLIB31 
//BIND.SYSIN DD * 
  INCLUDE OBJLIB(MYPROG) 
  NAME...

If you have used the wrong library at bind time, you may only find out a run time.

If you use the #pragma map to force it to use object names, you will find the problem at bind time, because myprog31 or myprog64 will not be found.