Defining a second TCPIP stack on z/OS on zPDT

I wanted a second TCPIP stack on my z/OS because I wanted to test it with MQWEB.   There is no good documentation in one place, there is good documentation hidden away, but not all in one place.
This took me about half a day to set up -including several IPLs , but I am on my own z/OS zPDT image so this was not a problem.  It take a while to understand the definitions – it is another one of “this point to that which points to something else…”.   You need to be able to copy a definition rather than use the books to create it from nothing.

I’ll describe setting up TCPIP2.

Overall I was surprised at how easy this bit was to set up.

The work breaks into

  • setting up the connectivity from Linux to z/OS
  • setting up the second TCP stack
    • Configure sys1.parmlib memmber and IPL
    • Define the new TCPIP procedure
    • Configure the new TCPIP configuration
    • Allowing people to use the TCPIP stack

Both of these need an IPL of z/OS, so you could do all of the customising and IPL afterwards at the end.

I’ll cover sharing an existing OSA adapter and setting up a new OSA adapter.

Sharing an existing OSA adapter.

Copy ADCD.Z24A.VTAMLST(OSATRL2) to USER.Z24A.VTAMLST(OSATRL2) and make the changes in bold

OSATRL1 VBUILD TYPE=TRL 00010000
OSATRL1E TRLE LNCTL=MPC,READ=(0400),WRITE=(0401), X00020007
               DATAPATH=(0402,0404,040,0406),     X00021013
               PORTNAME=PORTA,                    X00022004
               MPCLEVEL=QDIO                       00023005
*SATRL2E TRLE LNCTL=MPC,READ=(0404),WRITE=(0405),DATAPATH=(0406), X00024011
* PORTNAME=PORTB, X00025011
* MPCLEVEL=QDIO 00026011

I changed

  • DATAPATH=(0402) to DATAPATH=(0402,0404,0406)  – note every other address.    With 0402,0403 etc in the list, the second TCP failed to work, with messages like
    • EZZ4310I ERROR: CODE=80100040 REPORTED ON DEVICE PORTA. DIAGNOSTIC CODE: 03
    • EZZ4309I ATTEMPTING TO RECOVER DEVICE PORTA
    • IST1222I DATA DEVICE 0403 IS INOPERATIVE, NAME IS PORTA
    • IST1578I DEVICE INOP DETECTED FOR PORTA BY ISTTSCMA CODE = 104
  • Commented out/deleted the second TRLE definition

The zPDT devmap needs to have OSA definitions for these

name awsosa 0009 --path=A0 --pathtype=OSD --tunnel_intf=y # QDIO mode
device 400 osa osa --unitadd=0
device 401 osa osa --unitadd=1
device 402 osa osa --unitadd=2
device 403 osa osa --unitadd=3
device 404 osa osa --unitadd=4
device 405 osa osa --unitadd=5
device 406 osa osa --unitadd=6

I created a file USER.Z24A.TCPPARMS(T2OSA)

DEVICE PORTA  MPCIPA 
LINK ETH1  IPAQENET PORTA 
START PORTA 
HOME 10.1.1.3 ETH1 

and put

include USER.Z24A.TCPPARMS(T2OSA)

into my tcpip2 startup.

By putting the definitions in a PDS member, means I can use

V TCPIP,TCPIP2,OBEY,USER.Z24A.TCPPARMS(T2OSA)

to activate them.

I reipled the system to pick up VTAM changes.

Once I had stared TCPIP and TCPIP2 the command d net,id=OSATRL1E gave

D NET,ID=OSATRL1E
IST097I DISPLAY ACCEPTED
IST075I NAME = OSATRL1E, TYPE = TRLE 466
IST486I STATUS= ACTIV, DESIRED STATE= ACTIV
IST087I TYPE = LEASED , CONTROL = MPC , HPDT = YES
IST1954I TRL MAJOR NODE = OSATRL2
IST1715I MPCLEVEL = QDIO MPCUSAGE = SHARE
IST1716I PORTNAME = PORTA LINKNUM = 0 OSA CODE LEVEL = 7617
IST2337I CHPID TYPE = OSD CHPID = A0 PNETID = **NA**
IST1577I HEADER SIZE = 4096 DATA SIZE = 0 STORAGE = ***NA***
IST1221I WRITE DEV = 0401 STATUS = ACTIVE STATE = ONLINE
IST1577I HEADER SIZE = 4092 DATA SIZE = 0 STORAGE = ***NA***
IST1221I READ DEV = 0400 STATUS = ACTIVE STATE = ONLINE
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0403 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIP ULP INTERFACE = PORTA
...
IST1221I DATA DEV = 0404 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIP2 ULP INTERFACE = PORTA
IST2310I ACCELERATED ROUTING DISABLED
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0405 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0406 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------------------
IST1500I STATE TRACE = OFF

Setting up the connectivity from Linux to z/OS using a second OSA adapter

You need to set up an interface from Linux to z/OS via an Open Systems Adapter (OSA).

TCP/IP Interfaces are used to tunnel from Linux to z/OS.  These have names like tap0, tap1;  they tie up with z/OS paths and devices.  The Linux device drivers implement the QDIO protocol, a simpler and faster protocol than traditional z/OS channels.

Identify the path and devices to be used.

The zPDT find_io command gave me

 FIND_IO for "colin@colin-ThinkCentre-M920s" 

      I/face Cur           MAC      IPv4         IPv6 
Path  Name   State         Address  Address      Address 
----  ----   ---- -------- -------- -------       ----------------- ---------------- -------------- 
F0    eno1    UP, RUNNING 00:d8:... 10.1.0.3     fe80:...%eno1 
F1    wlxd..  UP, RUNNING d0:37:... 192.168.1.67 2a00:...6cab 
.  
A0   tap0     UP, RUNNING 9e:30:... 10.1.1.1     fe80:... %tap0 
A1   tap1     UP, RUNNING 7e:66:... 0.1.2.1      fe80:... %tap1 
A2   tap2   DOWN 02:a2:a2:a2:a2:a2  *            *

We can see from this the IP addresses being used;  channel paths A0, A1 are in use by tunneling; channel path A2 is available.

In the zPDT devmap I set up

[manager] # tap0 define network adapter (OSA) for communication with Linux
name awsosa 0009 --path=A0 --pathtype=OSD --tunnel_intf=y # QDIO mode
device 400 osa osa --unitadd=0
device 401 osa osa --unitadd=1
device 402 osa osa --unitadd=2

[manager] # tap1 define network adapter (OSA) for communication with Linux
name awsosa 0010 --path=A1 --pathtype=OSD --tunnel_intf=y --tunnel_ip=10.1.2.1 --tunnel_mask=255.255.255.0 # QDIO mode
device 408 osa osa --unitadd=0
device 409 osa osa --unitadd=1
device 40a osa osa --unitadd=2

Where the paths tie up with the output from the find_io.

Each connection needs 3 consecutive devices, for example 408,409,40a.

On z/OS use the command D U,CTC to find which devices are available.  I think (I am not sure) that the first device has to end in 0, or 8 .

I have

UNIT TYPE STATUS 
0400 OSA A-BSY 
0401 OSA A 
0402 OSA A-BSY 
0403 OSA OFFLINE 
0404 OSA OFFLINE 
0405 OSA OFFLINE 
0406 OSA OFFLINE 
0407 OSA OFFLINE 
0408 OSA A-BSY 
0409 OSA A 
040A OSA A-BSY

Once you have selected the OSA addresses to use, and configured the devmap file, you will need to restart zPDT with the updated devamp – but you need to customise z/OS and IPL – so do not IPL just yet.

Z/OS work for setting up the second TCP stack

Some basic terminology and concepts.

  • There is an network domain AF_INET which programmers use via sockets to communicate with the network.   (There is another network domain AF_UNIX for Unix programming).
  • You have to configure the domain, for example how many concurrent sessions it can support.
  • Originally you could have only one TCP stack in the environment.   This used an interface called INET.  This did not support more than one TCP/IP stacks.
  • A new interface was developed Common INET ( CINET). Conceptually this sits in front of TCP/IP and routes packets to the TCPIP subsystems.
  • To be able to use multiple stacks, CINET needs to be used instead of INET.
  • These are customised in SYS1.PARMLIB(BPXPRMxx).

Customise sys1.parmlib(BPXPRMxx) member

For example

FILESYSTYPE TYPE(INET) ENTRYPOINT(EZBPFINI) 

SUBFILESYSTYPE NAME(TCPIP) 
     TYPE(INET) 
     ENTRYPOINT(EZBPFINI) 

NETWORK DOMAINNAME(AF_INET) 
     DOMAINNUMBER(2) 
     MAXSOCKETS(64000) 
     TYPE(INET) 
     INADDRANYPORT(5555) 
     INADDRANYCOUNT(1000)

Change TYPE(INET) to TYPE(CINET) in 3 places, and change ENTRYPOINT(EZBPFINI) to ENTRYPOINT(BPXTCINT)

Add the new TCPIP address space

SUBFILESYSTYPE NAME(TCPIP2) 
     TYPE(CINET) 
     ENTRYPOINT(EZBPFINI)

This change needs a IPL to activate (or possibly a SETOMVS RESET=(xx).   I do not know what else the change from INET to CINET affects, so check with IBM before implementing it.

Define the TCPIP2 procedure

  • I defined a new profile in the STARTED class to map TCPIP2 to a userid.   I used the same userid as for TCPIP.
  • I copied the TCPIP procedure from TCPIP to TCPIP2.
  • The TCPIP procedure refers to TCP configuration,
    • //PROFILE  … DSN=SYS1.TCPPARMS(PROF) and
    • //SYSTCPD …. DSN=SYS1.TCPPARMS(TCPDATA).
  • Create your own copies of these, for example copy them to USER.TCPPARMS, and rename the members to PROF2, and TCPDATA2

Create VTAM definition for the tunnelling connection for the a second OSA adapter

 

If you are using a second OSA adapter, you need to create a VTAM member to map from the OSA device to a TCP/IP name using MPC.  This is Multi Protocol Channel, using protocol QDIO which is simpler and faster protocol than traditional z/OS channels.

Create a member in VTAMLST for a Transport Resource List major node, for example OSACOLIN.

----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
OSA5 VBUILD TYPE=TRL 
OSATRL5E TRLE LNCTL=MPC,READ=(0408),WRITE=(0409),DATAPATH=(040A),      X
               PORTNAME=PORTZ,                                         X
               MPCLEVEL=QDIO

Note the format, continuation ‘x’ in column 72, and continuation text in column 16.

You can use V NET,ACT,ID=OSACOLIN  to activate it. If you use D NET,IS=OSACP,E it should find it, and report it is active.

You can use D NET,TRL  to display the status of the  links.

Configure TCPPARMS(PROF2)

I used a copy of the TCPIP(PROF) as my starting configuration.

I commented out all of the lines between AUTOLOG and ENDAUTOLOG.

I went down to the DEVICE and BEGINROUTE section and used

DEVICE PORTZ MPCIPA
LINK ETHZ IPAQENET PORTZ
; end of link and device definitions
;
HOME 10.1.2.2 ETHZ
;
BEGINRoutes
;            Destination SubnetMask FirstHop LinkName Size
ROUTE        DEFAULT               10.1.2.1  ETHZ     MTU 1492
ENDRoutes
; start it when TCP/IP starts
START PORTZ

Where

  • DEVICE PORTZ MPCIPA  –  MPCIPA says this is an OSA QDIO, and uses the PORTZ definition.  PORTZ was defined above in the VTAMLST(OSACP).
  • LINK ETHZ IPAQENET PORTZ –  this create a LINK ETHZ associated with DEVICE PORTZ in the line above. It uses the interface type IPAQENET, which is for IP V4 and device OSA QDIO.   (There is IPAQENET6 for IP V6 for OSA QDIO).
  • HOME 10.1.2.2 ETHZ –  for traffic coming in over ETHZ (via PORTZ, and back to the tap1 which was defined with –tunnel_ip=10.1.2.1).   A ping 10.1.2.2 should come in over this interface.  For the first OSA adapter this had 10.1.1.2.
  •  BEGINRoutes
    ;                Destination SubnetMask FirstHop LinkName  Size
    ROUTE  DEFAULT                               10.1.2.1          ETHZ    MTU 1492
    ENDRoutes

    •  Any traffic going to 10.1.2.1 go via link ETHZ and use a packet size of 1492 bytes.
  • START PORTZ – get it working

Edit the TCPDATA

For sharing an OSA or using a new OSA, I edited the TCPDATA2 file and added

TCPIPJOBNAME TCPIP2
S0W1: HOSTNAME S0W1COL
DOMAINORIGIN COLIN.HOST.COM
DATASETPREFIX TCPIP
NSPORTADDR 53
RESOLVEVIA UDP
LOOKUP LOCAL
ALWAYSWTO YES

I dont know which of these are important.  I changed the bold lines, to match my name.

RACF profile changes

You have to set up a security  profile before an application can connect to TCPIP and listen on a socket.  MQWEB got EDC5112I Resource temporarily unavailable. (errno2=0x74610296)

rdefine SERVAUTH EZB.INITSTACK.*.TCPIP2  from(EZB.INITSTACK.*.TCPIP)

Using the model… above copies the permission from the base object.   You can allow more users using

permit EZB.INITSTACK.*.TCPIP2 class(SERVAUTH) id(START1) access(READ)

The “*” is for any system in the sysplex, so you could have EZB.INITSTACK.MVSA.TCPIP2 and allow access to TCPIP2 on system MVSA, but not from another MVS system.

You can protect TCPIP2 for example protect the NETSTAT command

RDEFINE SERVAUTH (EZB.NETSTAT.*.TCPIO2.*) UACC(NONE)
PERMIT (EZB.NETSTAT.*.TCPIP2.*) ACCESS(READ) CLASS(SERVAUTH) ID(TCPADMIN)
SETROPTS GENERIC(SERVAUTH) REFRESH 

Check it out

You can use the Linux netstat -i command to display the interfaces defined to Linux.  On my Linux  I got

colin@colin-ThinkCentre-M920s:/home/zPDT$ netstat -i 
Kernel Interface table
Iface     MTU   RX-OK ... Flg
eno1     1500   84758 ... BMRU
lo      65536  188855 ... LRU
tap0     1500       6 ... BMRU
tap1     1500      25 ... BMRU
wlxd0374 1500   10545 ... BMRU

z/OS commands

D TCPIP – displays the TCP address spaces in the LPAR

D tcpip,tcpip2,netstat,home gave
EZZ2500I NETSTAT CS V2R4 TCPIP2 540
HOME ADDRESS LIST:
ADDRESS LINK FLG
10.1.2.2 ETHZ P
127.0.0.1 LOOPBACK

Using TCPIP2 from Liberty web server

I added

_BPXK_SETIBMOPT_TRANSPORT=TCPIP2

to the server.env file, and restarted Liberty

I connected from my web browser to MQWEB using 10.1.2.2:9443, and got the messages

Your connection is not private
Attackers may be trying to steal your information from 10.1.2.2
NET:ERR_CERT_COMMON_NAME_INVALID

The NET:ERR_CERT_COMMON_NAME_INVALID message is because the certificate had a Subject Alternative Name of a different IP address 10.1.1.2.  It traffic flow was sent from 10.1.2.2.

This was what I expected.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s