Defining a second TCPIP stack on z/OS on zPDT

I wanted a second TCPIP stack on my z/OS because I wanted to test it with MQWEB.   There is no good documentation in one place, there is good documentation hidden away, but not all in one place.
This took me about half a day to set up -including several IPLs , but I am on my own z/OS zPDT image so this was not a problem.  It take a while to understand the definitions – it is another one of “this point to that which points to something else…”.   You need to be able to copy a definition rather than use the books to create it from nothing.

I’ll describe setting up TCPIP2.

Overall I was surprised at how easy this bit was to set up.

The work breaks into

  • setting up the connectivity from Linux to z/OS
  • setting up the second TCP stack
    • Configure sys1.parmlib memmber and IPL
    • Define the new TCPIP procedure
    • Configure the new TCPIP configuration
    • Allowing people to use the TCPIP stack

Both of these need an IPL of z/OS, so you could do all of the customising and IPL afterwards at the end.

I’ll cover sharing an existing OSA adapter and setting up a new OSA adapter.

Sharing an existing OSA adapter.

Copy ADCD.Z24A.VTAMLST(OSATRL2) to USER.Z24A.VTAMLST(OSATRL2) and make the changes in bold

OSATRL1 VBUILD TYPE=TRL 00010000
OSATRL1E TRLE LNCTL=MPC,READ=(0400),WRITE=(0401), X00020007
               DATAPATH=(0402,0404,040,0406),     X00021013
               PORTNAME=PORTA,                    X00022004
               MPCLEVEL=QDIO                       00023005
*SATRL2E TRLE LNCTL=MPC,READ=(0404),WRITE=(0405),DATAPATH=(0406), X00024011
* PORTNAME=PORTB, X00025011
* MPCLEVEL=QDIO 00026011

I changed

  • DATAPATH=(0402) to DATAPATH=(0402,0404,0406)  – note every other address.    With 0402,0403 etc in the list, the second TCP failed to work, with messages like
    • EZZ4310I ERROR: CODE=80100040 REPORTED ON DEVICE PORTA. DIAGNOSTIC CODE: 03
    • EZZ4309I ATTEMPTING TO RECOVER DEVICE PORTA
    • IST1222I DATA DEVICE 0403 IS INOPERATIVE, NAME IS PORTA
    • IST1578I DEVICE INOP DETECTED FOR PORTA BY ISTTSCMA CODE = 104
  • Commented out/deleted the second TRLE definition

The zPDT devmap needs to have OSA definitions for these

name awsosa 0009 --path=A0 --pathtype=OSD --tunnel_intf=y # QDIO mode
device 400 osa osa --unitadd=0
device 401 osa osa --unitadd=1
device 402 osa osa --unitadd=2
device 403 osa osa --unitadd=3
device 404 osa osa --unitadd=4
device 405 osa osa --unitadd=5
device 406 osa osa --unitadd=6

I created a file USER.Z24A.TCPPARMS(T2OSA)

DEVICE PORTA  MPCIPA 
LINK ETH1  IPAQENET PORTA 
START PORTA 
HOME 10.1.1.3 ETH1 

and put

include USER.Z24A.TCPPARMS(T2OSA)

into my tcpip2 startup.

By putting the definitions in a PDS member, means I can use

V TCPIP,TCPIP2,OBEY,USER.Z24A.TCPPARMS(T2OSA)

to activate them.

I reipled the system to pick up VTAM changes.

Once I had stared TCPIP and TCPIP2 the command d net,id=OSATRL1E gave

D NET,ID=OSATRL1E
IST097I DISPLAY ACCEPTED
IST075I NAME = OSATRL1E, TYPE = TRLE 466
IST486I STATUS= ACTIV, DESIRED STATE= ACTIV
IST087I TYPE = LEASED , CONTROL = MPC , HPDT = YES
IST1954I TRL MAJOR NODE = OSATRL2
IST1715I MPCLEVEL = QDIO MPCUSAGE = SHARE
IST1716I PORTNAME = PORTA LINKNUM = 0 OSA CODE LEVEL = 7617
IST2337I CHPID TYPE = OSD CHPID = A0 PNETID = **NA**
IST1577I HEADER SIZE = 4096 DATA SIZE = 0 STORAGE = ***NA***
IST1221I WRITE DEV = 0401 STATUS = ACTIVE STATE = ONLINE
IST1577I HEADER SIZE = 4092 DATA SIZE = 0 STORAGE = ***NA***
IST1221I READ DEV = 0400 STATUS = ACTIVE STATE = ONLINE
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0403 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIP ULP INTERFACE = PORTA
...
IST1221I DATA DEV = 0404 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIP2 ULP INTERFACE = PORTA
IST2310I ACCELERATED ROUTING DISABLED
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0405 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------------------
IST1221I DATA DEV = 0406 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------------------
IST1500I STATE TRACE = OFF

Setting up the connectivity from Linux to z/OS using a second OSA adapter

You need to set up an interface from Linux to z/OS via an Open Systems Adapter (OSA).

TCP/IP Interfaces are used to tunnel from Linux to z/OS.  These have names like tap0, tap1;  they tie up with z/OS paths and devices.  The Linux device drivers implement the QDIO protocol, a simpler and faster protocol than traditional z/OS channels.

Identify the path and devices to be used.

The zPDT find_io command gave me

 FIND_IO for "colin@colin-ThinkCentre-M920s" 

      I/face Cur           MAC      IPv4         IPv6 
Path  Name   State         Address  Address      Address 
----  ----   ---- -------- -------- -------       ----------------- ---------------- -------------- 
F0    eno1    UP, RUNNING 00:d8:... 10.1.0.3     fe80:...%eno1 
F1    wlxd..  UP, RUNNING d0:37:... 192.168.1.67 2a00:...6cab 
.  
A0   tap0     UP, RUNNING 9e:30:... 10.1.1.1     fe80:... %tap0 
A1   tap1     UP, RUNNING 7e:66:... 0.1.2.1      fe80:... %tap1 
A2   tap2   DOWN 02:a2:a2:a2:a2:a2  *            *

We can see from this the IP addresses being used;  channel paths A0, A1 are in use by tunneling; channel path A2 is available.

In the zPDT devmap I set up

[manager] # tap0 define network adapter (OSA) for communication with Linux
name awsosa 0009 --path=A0 --pathtype=OSD --tunnel_intf=y # QDIO mode
device 400 osa osa --unitadd=0
device 401 osa osa --unitadd=1
device 402 osa osa --unitadd=2

[manager] # tap1 define network adapter (OSA) for communication with Linux
name awsosa 0010 --path=A1 --pathtype=OSD --tunnel_intf=y --tunnel_ip=10.1.2.1 --tunnel_mask=255.255.255.0 # QDIO mode
device 408 osa osa --unitadd=0
device 409 osa osa --unitadd=1
device 40a osa osa --unitadd=2

Where the paths tie up with the output from the find_io.

Each connection needs 3 consecutive devices, for example 408,409,40a.

On z/OS use the command D U,CTC to find which devices are available.  I think (I am not sure) that the first device has to end in 0, or 8 .

I have

UNIT TYPE STATUS 
0400 OSA A-BSY 
0401 OSA A 
0402 OSA A-BSY 
0403 OSA OFFLINE 
0404 OSA OFFLINE 
0405 OSA OFFLINE 
0406 OSA OFFLINE 
0407 OSA OFFLINE 
0408 OSA A-BSY 
0409 OSA A 
040A OSA A-BSY

Once you have selected the OSA addresses to use, and configured the devmap file, you will need to restart zPDT with the updated devamp – but you need to customise z/OS and IPL – so do not IPL just yet.

Z/OS work for setting up the second TCP stack

Some basic terminology and concepts.

  • There is an network domain AF_INET which programmers use via sockets to communicate with the network.   (There is another network domain AF_UNIX for Unix programming).
  • You have to configure the domain, for example how many concurrent sessions it can support.
  • Originally you could have only one TCP stack in the environment.   This used an interface called INET.  This did not support more than one TCP/IP stacks.
  • A new interface was developed Common INET ( CINET). Conceptually this sits in front of TCP/IP and routes packets to the TCPIP subsystems.
  • To be able to use multiple stacks, CINET needs to be used instead of INET.
  • These are customised in SYS1.PARMLIB(BPXPRMxx).

Customise sys1.parmlib(BPXPRMxx) member

For example

FILESYSTYPE TYPE(INET) ENTRYPOINT(EZBPFINI) 

SUBFILESYSTYPE NAME(TCPIP) 
     TYPE(INET) 
     ENTRYPOINT(EZBPFINI) 

NETWORK DOMAINNAME(AF_INET) 
     DOMAINNUMBER(2) 
     MAXSOCKETS(64000) 
     TYPE(INET) 
     INADDRANYPORT(5555) 
     INADDRANYCOUNT(1000)

Change TYPE(INET) to TYPE(CINET) in 3 places, and change ENTRYPOINT(EZBPFINI) to ENTRYPOINT(BPXTCINT)

Add the new TCPIP address space

SUBFILESYSTYPE NAME(TCPIP2) 
     TYPE(CINET) 
     ENTRYPOINT(EZBPFINI)

This change needs a IPL to activate (or possibly a SETOMVS RESET=(xx).   I do not know what else the change from INET to CINET affects, so check with IBM before implementing it.

Define the TCPIP2 procedure

  • I defined a new profile in the STARTED class to map TCPIP2 to a userid.   I used the same userid as for TCPIP.
  • I copied the TCPIP procedure from TCPIP to TCPIP2.
  • The TCPIP procedure refers to TCP configuration,
    • //PROFILE  … DSN=SYS1.TCPPARMS(PROF) and
    • //SYSTCPD …. DSN=SYS1.TCPPARMS(TCPDATA).
  • Create your own copies of these, for example copy them to USER.TCPPARMS, and rename the members to PROF2, and TCPDATA2

Create VTAM definition for the tunnelling connection for the a second OSA adapter

 

If you are using a second OSA adapter, you need to create a VTAM member to map from the OSA device to a TCP/IP name using MPC.  This is Multi Protocol Channel, using protocol QDIO which is simpler and faster protocol than traditional z/OS channels.

Create a member in VTAMLST for a Transport Resource List major node, for example OSACOLIN.

----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
OSA5 VBUILD TYPE=TRL 
OSATRL5E TRLE LNCTL=MPC,READ=(0408),WRITE=(0409),DATAPATH=(040A),      X
               PORTNAME=PORTZ,                                         X
               MPCLEVEL=QDIO

Note the format, continuation ‘x’ in column 72, and continuation text in column 16.

You can use V NET,ACT,ID=OSACOLIN  to activate it. If you use D NET,IS=OSACP,E it should find it, and report it is active.

You can use D NET,TRL  to display the status of the  links.

Configure TCPPARMS(PROF2)

I used a copy of the TCPIP(PROF) as my starting configuration.

I commented out all of the lines between AUTOLOG and ENDAUTOLOG.

I went down to the DEVICE and BEGINROUTE section and used

DEVICE PORTZ MPCIPA
LINK ETHZ IPAQENET PORTZ
; end of link and device definitions
;
HOME 10.1.2.2 ETHZ
;
BEGINRoutes
;            Destination SubnetMask FirstHop LinkName Size
ROUTE        DEFAULT               10.1.2.1  ETHZ     MTU 1492
ENDRoutes
; start it when TCP/IP starts
START PORTZ

Where

  • DEVICE PORTZ MPCIPA  –  MPCIPA says this is an OSA QDIO, and uses the PORTZ definition.  PORTZ was defined above in the VTAMLST(OSACP).
  • LINK ETHZ IPAQENET PORTZ –  this create a LINK ETHZ associated with DEVICE PORTZ in the line above. It uses the interface type IPAQENET, which is for IP V4 and device OSA QDIO.   (There is IPAQENET6 for IP V6 for OSA QDIO).
  • HOME 10.1.2.2 ETHZ –  for traffic coming in over ETHZ (via PORTZ, and back to the tap1 which was defined with –tunnel_ip=10.1.2.1).   A ping 10.1.2.2 should come in over this interface.  For the first OSA adapter this had 10.1.1.2.
  •  BEGINRoutes
    ;                Destination SubnetMask FirstHop LinkName  Size
    ROUTE  DEFAULT                               10.1.2.1          ETHZ    MTU 1492
    ENDRoutes

    •  Any traffic going to 10.1.2.1 go via link ETHZ and use a packet size of 1492 bytes.
  • START PORTZ – get it working

Edit the TCPDATA

For sharing an OSA or using a new OSA, I edited the TCPDATA2 file and added

TCPIPJOBNAME TCPIP2
S0W1: HOSTNAME S0W1COL
DOMAINORIGIN COLIN.HOST.COM
DATASETPREFIX TCPIP
NSPORTADDR 53
RESOLVEVIA UDP
LOOKUP LOCAL
ALWAYSWTO YES

I dont know which of these are important.  I changed the bold lines, to match my name.

RACF profile changes

You have to set up a security  profile before an application can connect to TCPIP and listen on a socket.  MQWEB got EDC5112I Resource temporarily unavailable. (errno2=0x74610296)

rdefine SERVAUTH EZB.INITSTACK.*.TCPIP2  from(EZB.INITSTACK.*.TCPIP)

Using the model… above copies the permission from the base object.   You can allow more users using

permit EZB.INITSTACK.*.TCPIP2 class(SERVAUTH) id(START1) access(READ)

The “*” is for any system in the sysplex, so you could have EZB.INITSTACK.MVSA.TCPIP2 and allow access to TCPIP2 on system MVSA, but not from another MVS system.

You can protect TCPIP2 for example protect the NETSTAT command

RDEFINE SERVAUTH (EZB.NETSTAT.*.TCPIO2.*) UACC(NONE)
PERMIT (EZB.NETSTAT.*.TCPIP2.*) ACCESS(READ) CLASS(SERVAUTH) ID(TCPADMIN)
SETROPTS GENERIC(SERVAUTH) REFRESH 

Check it out

You can use the Linux netstat -i command to display the interfaces defined to Linux.  On my Linux  I got

colin@colin-ThinkCentre-M920s:/home/zPDT$ netstat -i 
Kernel Interface table
Iface     MTU   RX-OK ... Flg
eno1     1500   84758 ... BMRU
lo      65536  188855 ... LRU
tap0     1500       6 ... BMRU
tap1     1500      25 ... BMRU
wlxd0374 1500   10545 ... BMRU

z/OS commands

D TCPIP – displays the TCP address spaces in the LPAR

D tcpip,tcpip2,netstat,home gave
EZZ2500I NETSTAT CS V2R4 TCPIP2 540
HOME ADDRESS LIST:
ADDRESS LINK FLG
10.1.2.2 ETHZ P
127.0.0.1 LOOPBACK

Using TCPIP2 from Liberty web server

I added

_BPXK_SETIBMOPT_TRANSPORT=TCPIP2

to the server.env file, and restarted Liberty

I connected from my web browser to MQWEB using 10.1.2.2:9443, and got the messages

Your connection is not private
Attackers may be trying to steal your information from 10.1.2.2
NET:ERR_CERT_COMMON_NAME_INVALID

The NET:ERR_CERT_COMMON_NAME_INVALID message is because the certificate had a Subject Alternative Name of a different IP address 10.1.1.2.  It traffic flow was sent from 10.1.2.2.

This was what I expected.

ADCD Personalisation of z/OS for first time users

People like to customise how they work.  This blog is the z/OS personalisation I tend to do.  If you have other suggestions please let me know.

Running zPDT on Ubuntu

The recommendation is to run zPDT under a different userid ibmsys1.   To be able to switch between userids

  • Switch user from pull down, select ibmsys1
  • You can switch back to the your normal userid using  Ctrl-Alt-F2, and use Ctrl-Alt-F3 to switch to the ibmsys1 userid
  • I could not see how to cut and paste between userids so I have a file I write to from one userid, and read the file from  “the other side”
  • Post on virtual terminals. and how to switch without F keys.

Ive been running on my normal userid without any problems

Set time zone

Edit /etc/profile and set the TZ.  It defaults to EST5EDT.  I used BST for British Summer Time ( or Europe/London).

Screen size  using bigger screens.

  • Use x3270 -model 5  to get 132*27 screen size.  It works for the console and ISPF terminal .The -oversize 133×60 parameter should work.
  • Logon and use ISPF =0  ro set defaults.  Scroll down
    • To have command line at the top / Command line at bottom  remove the /
    •  Scroll down.  Screen format 3 1. Data 2. Std 3. Max 4. Part
    • Terminal Type 4 1. 3277 2. 3277A 3. 3278 4. 3278A

Other ISPF personalisation

  • Options
    _ Command line at bottom  remove the / to have the command line at the top
    / Tab to point-and-shoot fields so you can tap to column headers, press enter and sort by the columne
  • Member list options
    / Scroll member list
    / Allow empty member list
    / Allow empty member list (nomatch)
    / Empty member list for edit only
  • pfshow off remove the PFKEYS at the bottom
  • ISPF  keys set PF12 to retrieve not cancel
  • ISPF set scroll to CSR not PAGE in all applications
  • ISPF 3.4 use reflists list of the last 30 data sets used, or your own list
  • Setting the ISPF main panel.
    • Copy ADCD.Z24A.ISPPLIB(ISR@PRIM)  to USER.Z24A.ISPPLIB(MYMAIN).
    • Add extra content and comparisons at the bottom for example ISMF,’PGM(DGTFMD01) NEWAPPL(DGT)’ .
    • The following are already defined
      •  RACF,’PANEL(ICHP00)’
      • ISMF,’PGM(DGTFMD01) NEWAPPL(DGT)’
      • SMPE,’PGM(GIMSTART) PARM(&ZCMD) NOCHECK’
      • WLM,’CMD(%IWMARIN0)’
    • When you use the TSO Logon panel specify Command ===> ispf panel(MYMAIN)  

OMVS customising

Escape key to break into long running commands use the escape key.  Default is the cent key ¢

You can use x3270 to set a key to this value, in .x3270pro

x3270.keymap: mine
! Definition of the 'mine' keymap
x3270.keymap.mine: #override \
  <Key>Escape: Clear()\n\
  <Key>End: FieldEnd()\n\
  Ctrl<Key>Delete: EraseEOF()\n\
  Ctrl<Key>Right: NextWord()\n\
  Ctrl<Key>Left: PreviousWord()\n\
  Ctrl<Key>Up: Home()\n\
  <Key>Control_L: Reset()\n\
  <Key>Control_R: Reset()\n\
  <Key>Prior: PF(7)\n\
  <Key>Next: PF(8)\n\
  <Btn3Down>: PA(1)\n\
  Ctrl<Key>1: PA(1)\n\ 
! the next define Alt 4 and Ctrl \ as &cent.
  Alt<Key>4: String("\\x00a2")\n\
  Ctrl<Key>backslash: String("\\x00a2")

 

Note

  • the \\ escape characters.
  • lines end in \n\   which is new line + continutation
  • this also defines Ctrl 1 as PA1
  • String(“\\x00a2\n”) would type the cent symbol and press enter

Set userids OMVS information

create directory /u/adcd then use RACF command

ALTUSER ADCDA OMVS(HOME('/u/adcd') PROGRAM('/bin/sh'))
ALTUSER START1 OMVS( PROGRAM('/bin/sh'))

z/OS customisation

  • Set clock time zone.  Copy  ADCC.Z24A.PARMLIB(CLOCK00) to USER.Z24A.PARMLIB(CLOCK00) and edit it.  Note:  FEU.Z24A.PARMLIB(CLOCK00) is used by default.
    •  SET TIMEZONE=E.01.00 command
  • RMF not a good idea – sometimes abends (S0C6) other times locks up z/OS

 

The following come from FEU.Z24A.PARMLIB

IEASYM00
IEASYS00
IEASYSWS
CLOCK00
AUTORDT
IEACMD00
MPFLST00

 

GTF

Create user.z24a.proclib(GTF).   The text in bold is new text compared to SYS1.PROCLIB(GTF).  It deletes the data set before reallocating it.  It allows a userid to be specified.

//GTFNEW PROC M=GTFPARM,ID=SYS1 
//DELETE EXEC PGM=IEFBR14 
//IEFRDER DD DSNAME=&ID..TRACE,UNIT=SYSDA,SPACE=(TRK,20), *
// DISP=(MOD,DELETE) 
//IEFPROC EXEC PGM=AHLGTF,PARM='MODE=EXT,DEBUG=NO,TIME=YES', *
// TIME=1440,REGION=2880K 
//IEFRDER DD DSNAME=&ID..TRACE,UNIT=SYSDA,SPACE=(TRK,20), *
// DISP=(NEW,KEEP) 
//SYSLIB DD DSNAME=USER.Z24A.PROCLIB(&M),DISP=SHR

Create USER.Z24A.PROCLIB(GTFPARM)

TRACE=SYSM,USR,TRC,DSP,PCI,SRM

and any others you need for example  USER.Z24A.PROCLIB(GTFRACF)

TRACE=USRP
USR=(F44)
END

 

 

ADCD. Backups – why, when, how – whoops.

Having got my own personal z/OS running on my laptop, I now need to look after it.  When I worked for IBM there was a team of people who looked after the z/OS systems, including backups, security and applying fixes. Suddenly with my personal z/OS,  there are a lot of things I need to think about.  Today’s topic is backups.

On my Linux  machine I have backups being taken daily to an external hard drive.  I have a Linux on a USB in case I have problems with my main machine.  How do I do backups on z/OS?

What do I want to backup? Is the wrong question.

The real question should be What do I want to restore?  For example I can get a copy of the operating system from my original download files – or from IBM, but I need to be able to restore the files particular to me.  It is better to restore the total system rather than rebuild it, because of all the additional configuration you had to do (which you may not have record of).  The JCL I have written, the data in the database or MQ queue files, security profiles.

What situations do need to restore from?

It can range from

  • I messed up – I edited a file, and now it does not work.  I cannot undo the changes.  I deleted a file.  I want to go back to last week’s copy.
  • By accident you had two copies of a program updating a file – and corrupting it.
  • The database change you made cannot be undone – you added a new field, and now the record length is longer than the 4KB buffers.
  • There has been an I/O error on the disk (though this is rare).
  • I had my laptop stolen.
  • My 3 year old child used my hard drive as a toy and found it does not float on water.

You also need to ask how long do I have to recover?  If the answer is a week, then you can order a new hard drive, and wait a week for it to be delivered.  If you need it back within hours, you’ll have a spare disk just in case (or you did a make copies to this disk – so all you need to do is use it).

Setting up z/OS

As a rule, with ADCD you should not use any of the ADCD volume for your own data.  Create your own volumes and put your data on that.   Create a user catalog, and use alias’s from the master catalog for this user catalog.  If you have a new ADCD system you need to import the user catalog, and redefined the aliases.

Backup the USER.* data sets.   Do not change the ADCD.* or SYS1.* data sets.

Some of the subsystems, DB2, CICS and MQ have data files on the A4PRD* volumes.   This means you need to backup the volumes – and will be a challenge during migration.

When can I backup?

You should backup when the files are not being used.

  • You can edit a file, use tso xmit to make a copy of the PDS, then save the file you were editing.   That is OK. Using TSO XMIT while the file is being saved could cause a consistency problem.
  • You need to backup some files as logical files, so for example backup the MQ.PAGESET.   If this data set was spread across two disks, and you do an image copy of the first disk, followed by the image copy of the second disk, the data is likely to be inconsistent (if you restore you may not find out for a week after the restore!)  MQ  (and DB2) have logic to be able to recover when a logical dataset is restored.  Some systems have a quiesce capability which stop activity to the file, without stopping the subsystem.
  • Doing full volume backups should be done when the volume is not in use, either the z/OS is down, or the volume has been varied offline and removed from zPDT.  Shutting down may be better, so all the volumes are consistent together.  Sometimes there is data in buffers which has not been written to disk (lazy write), so you have to be careful.

You might try to backup only what has changed. This could be difficult.  Unless the disks/files are read only, there is a chance that a file has changed, or a file has been put on a disk.

How do I backup files?
PDS and sequential files.

You can use the TSO XMIT (TRANSMIT) command to take a file or library and create a file which is easy to transport.

To restore it you use TSO RECEIVE indsn(…) newname(abc…) so can have the current and restored versions with different names.   This allows you to process just one, or as many members as you want.

Files in USS

The file behind the filesystem is a VSAM file.

You could use unix commands like tar or pax to package up a file or directory.   The output can be a file in the file system or into a z/OS dataset.

You could use ADRDSSU to backup the whole file system – see the next topic.

Other files

Traditionally these files are backed up use the ADRDSSU or AMATERSE (or both) utilities which can backup the file, and any indexes etc that go with it.  The output can be a z/OS dataset, or DUMPed to tape.

Full volume backups

Shutdown z/OS  down cleanly, stop zPDT (to ensure buffers in Linux are flushed), and backup the linux files.   Restart z/OS.

Where do I backup to?

To recover from operator errors on “user files”, having the backup on z/OS may be enough.

To be able to recover from system problems, or disk problems, put the backups on a different file systems.  If my z/OS system is on the SSD on my laptop, have the files go to an external file system.  Some people will have their hard drives copied to another disk system, or even “off site”.

Getting backups out of z/OS

You can use FTP into TSO or USS to copy the files.  If you use pax output to a TSO file, you can ftp into TSO.  If you pax output into an unix file, FTP into USS.

You can also virtual tape, so ADRDSSU writes to a tape which maps to a file on the Linux file.

Having backed up the files what then?  Plan for a whoops.

  • It is worth checking that your backups restore, for example restore to a spare HDD, and try to boot from it.
  • It is also worth checking that you are backing up what you think you are backing up.  I know of one customer who was backing up the MQ pagesets, but did not change the backup job when they added more page sets.  I have been guilty or repeating a line and not changing the data set name, so data set A was backed up twice, and data set B was not backed up.
  • Determine how long it will take to restore disks, restart, and recover the file(s) of interest.  If this duration is too long – review your backup and restore procedures.

What next?

I asked about backup on the zPDT group forum and had lots of great comments.  Below is a summary of the comments.

  • Use of Clonezilla. This is a partition and disk imaging/cloning program similar to True Image® or Norton Ghost®. It helps you to do system deployment, bare metal backup and recovery.
  • Use ADRDSSU DUMP followed by AMATERSE to make the z/OS backups smaller.
  • Use of a Synology Network Addressed Storage for your backups.  Synology has comments like “Good for home users and small businesses”.
  • Use ADRDSSU to dump to a volume.  Vary volume offline, then backup the volume.
  • Do not use any of the AD-CD supplied volumes for your data. Create your own volume(s) and simply add them to the devmap for new releases. You need to have a usercatalog on your volume(s) and import it to subsequent releases. You can try to make ALIAS definitions carry forward; I usually just recatalog my datasets for each new release.
  • Use LVM snapshots. With the snapshot Linux grsync with an external drive
  • Use of Borg. The main goal of Borg is to provide an efficient and secure way to backup data. Borg cuts all data into chunks, builds a hash and if the hash is not yet known, the chunk is compressed and stored in a repository. Otherwise only a pointer is set for the chunk in the current archive. This saves a lot of time and disk space (after the initial backup) because only the changed parts of the z-disk images are compressed and stored into the archive.

How long will it take?

This depends on the media you are using, and how much data.  On my laptop copying an 8GB volume from HDD to SSD took about 4 minutes or about 30 MB/second. Compressing it may speed this up.

Some good JCL examples.

Thanks to James Alexander from Hostbridge for the following examples.

The user submits a tape job with an extra "mount" tape step:
//EXP       EXPORT SYMLIST=(DSNAME,UNIT,HLQ,VOL)                  
//*                                                              
//          SET HLQ=MYHLQ                                        
//          SET DSNAME=BACKUP.D999999.DFDSS                
//          SET UNIT=591                                          
//          SET VOL=J00001                                        
//*                                                              
//MOUNT     EXEC MOUNT,UNIT=&UNIT,DSNAME=&DSNAME,VOL=&VOL        
//*                                                              
//*  What follows is a standard DFDSS backup to tape. We compress
//*  it here so less disk space is used.                          
//*                                                              
//BACKUP    EXEC  PGM=ADRDSSU,REGION=0K                          
//SYSPRINT  DD  SYSOUT=*                                          
//TAPE1    DD  UNIT=&UNIT,VOL=SER=(&VOL),                        
//         DISP=NEW,DSN=&DSNAME,LABEL=(1,SL)                      
//SYSIN     DD    *,SYMBOLS=JCLONLY,DLM=$$                        
 DUMP DATASET(                              -                    
         INCLUDE(&HLQ..**           )       -                    
      )                                     -                    
      OUTDDNAME(TAPE1)                      -                    
      TOLERATE(ENQFAILURE)                  -                    
      OPTIMIZE(4)                           -                    
      COMPRESS                                                    
$$                                                                
//

The mount step executes AWSCMDX that runs a Linux script.   If the “DSNAME” tape file exists it mounts it;  if not it copies a tape template file and then mounts it.  A Linux job fires once an hour and syncs all of the files in the tape directory to AWS S3.  With this any user can run a tape job and get offsite backups,   Using the same methodology they can also do their own restores.

Here is the mount proc:

//MOUNT     PROC UNIT=590,DSNAME=BAD.DATASET.NAME,VOL=T00001
//* 
//X         EXPORT SYMLIST=(DSNAME,UNIT,VOL)   
//S         SET UNIT=&UNIT,DSNAME=&DSNAME,VOL=&VOL 
//* 
//M         EXEC PGM=AWSCMDX,PARMDD=MYPARMS
//SYSPRINT  DD   SYSOUT=*
//TAPE      DD   UNIT=(580,,DEFER),LABEL=(1,BLP),VOL=SER=123456,DSN=X
//MYPARMS   DD *,SYMBOLS=JCLONLY
./mountfile &UNIT &DSNAME &VOL 
/*

And here is the mountfile script in Linux:

#!/bin/bash
Unit=$1
Filename='/z/backup/tapes/'$2
Template='/z/backup/TapeTemplate'
echo 'Checking to see if the tape file exists'
if ! [ -e "$Filename" ]
then
    echo 'File does not exist copying template'
    cp $Template $Filename
fi 

echo 'Mounting '$Filename' on unit '$Unit
awsmount $Unit -m $Filename

Getting z/OS running on my Ubuntu laptop with zD&T and zPDT

Having downloaded and installed the zD&T, ZPDT and z/OS products on my laptop.  I was keen to start using it.   Here are part of my journey.

The  IBM ZPDT Guide and Reference redbook is excellent.

I used zD&T to install the products,  but it is missing a few things.

Check the environment

I used /usr/z1090/bin/z1090instcheck in section 4.1.84 of the red book to check the configuration.

I had

 UBUNTU kernel.core_pattern is |/usr/share/apport/apport which is BAD

I left it as it was, it is about what happens if there is a coredump.

Section 4.1.11 talks about checking /etc/sysctl.conf. You can use

sudo  /usr/z1090/bin/aws_sysctl

to set all of the parameters to value values.   It prompts before it changes any value, so is safe.

Creating the devmap

I used the perl script to create my devmap file.

Number of processors

You may want to set the number if processors less than the number of cores in your machine, so you can continue to run if zPDT is 100% busy.
Use the top command.  At the top it shows “load average: 5.3 3.7 4.1” this shows 3 numbers – this is 3 cores.

Set the number of cores using the

processors 1 # number of processors

statement.  Event with 1 processor, sometimes the laptop locked up, the cursor did not move,  or was about 5 seconds behind moving the mouse.

Storage usage

z/OS requires a minimum of 2GB to start (otherwise you get a message at IPL)

IAR057D LESS THAN 2GB OF REAL STORAGE IMPACTS SYSTEM AVAILABILITY

Using the top command gave

 

Use the free -g command to display the storage in your machine, so you do not over allocate the storage.

      total used free shared buff/cache available
Mem:     7     3    0      3          3         0
Swap:    0     0    0

The devmap had

memory 5984m  # define storage size for virtual host

When I stared zD&T  I got

AWSSTA146I Starting independent 1090 instance 'ibmsys1'
AWSEMI001T Insufficient Memory for 1090 to start.

I changed the memory to

memory 2G  # define storage size for virtual host

And I got into the IPL.

First IPL

I did a cold start, as per instructions.

I saw a message “waiting for vtam”.  So I reipled.

I used the command k s,del=n to prevent messages rolling off the top of the screen, and saw

CSV028I ABEND306-0C ISTCFCR2 and VTAM won’t start

I issued the commands

  • SETPROG APF,ADD,DSNAME=NET621.SCNMLNK1,VOLUME=A4PRD3
  • S VTAM

and the system started.

Edit ADCD.Z24A.PARMLIB(PROGAB)  and add  NET621.SCNMLNK1  A4PRD3, and do the same the the same data set on SARES1

Basic checks

The system worked as expected.

I could use x3270 localhost:3270 to get into z/OS using a 3274 “hardware controller” or x3270 10.1.1.2 via TCPIP.  This showed my IP network was basically working.

To get FTP working

Use the MVS command S  FTPD this takes a short time to start.

I installed vsftp on Ubuntu and started it

  • sudo apt install vsftpd
  • sudo service vsftpd start

I could FTP 10.1.1.2 and enter the userid and password.  When I issued any command such as ls, it hung.

I used this article on the Ubuntu firewall. The command  sudo ufw status verbose gave

Status: active

I disabled the firewall in linux

sudo ufw disable

and ftp worked.

The log showed me the activity sudo grep -i ufw /var/log/syslog |grep 10.1.1

Either of the commands give access through the firewall

  • sudo ufw allow from 10.1.1.2
  • sudo iptables -A INPUT -p tcp -s 10.1.1.2 -j ACCEPT

TCP error messages

I got

11.45.32 *EZZ9308E UNRESPONSIVE NAME SERVER DETECTED AT IP ADDRESS 9.26.4.6
11.45.32 EZZ9310I NAME SERVER 9.26.4.6
TOTAL NUMBER OF QUERIES SENT 2
TOTAL NUMBER OF FAILURES 2
PERCENTAGE 100%

This is because it is trying to use the DNS server .  The default is to use DNS then local. To fix this edit  ADCD.Z24A.TCPPARMS(TCPDATA)  insert the bold line

; LOOKUP statement 
; ================ 
; LOOKUP indicates the order of name and address resolution. DNS means 
; use the DNSs listed on the NSINTERADDR and NAMESERVER statements. 
; LOCAL means use the local host tables as appropriate for the 
; environment being used (UNIX System Services or Native MVS). 
; 
; LOOKUP DNS LOCAL  the default
LOOKUP LOCAL

Specifying LOCAL uses /etc/hosts which has

10.1.1.2 S0W1.CANLAB.IBM.COM S0W1 
127.0.0.1__ localhost localhost.localdomain localhost4 
___________ localhost4.localdomain4__________________ 
::1________ localhost localhost.localdomain localhost6 
___________ localhost6.localdomain6________________

Where the _ is really hex ’41’ !

I went into OMVS and issued

mv /etc/resolv.conf /etc/resolv.conf.old
touch /etc/resolv.conf
chmod 755 /etc/resolv.conf

ISPF primary menu

The initial ISPF menu is in

ADCD.Z24A.ISPPLIB(ISR@PRIM)

You can copy it to USER.Z24A.ISPPLIB and tailor it.  If you create a new member name – COLIN, use the command at logon ispf panel(COLIN)

The supplied ISR@PRIM has options which are not listed on the display RACF, ISMF, SMPE, WLM S (for SDSF).  This allows you to issue =S or =ISMF.

When you logon to TSO there is an ISPFLITE procedure with no optional products in the list, in case you have problems.

Unexpected messages

I got the following message many times at IPL  once for each disk.

IEC816I xxx VARY ONLINE – CU AUTHORIZATION FAILED SER=IBM-01024

A VARY ONLINE command attempted to validate the use of advanced features for the device.
The authorization failed.

I think this can be ignored.

RACF set up.

RACMAP command gets Abend System 684 rc 004

Copy ADCD.Z24A.PARMLIB(IKJTSO00) to USER.Z24A.PARMLIB(IKJTSO00)

Add RACMAP