Having got my own personal z/OS running on my laptop, I now need to look after it. When I worked for IBM there was a team of people who looked after the z/OS systems, including backups, security and applying fixes. Suddenly with my personal z/OS, there are a lot of things I need to think about. Today’s topic is backups.
On my Linux machine I have backups being taken daily to an external hard drive. I have a Linux on a USB in case I have problems with my main machine. How do I do backups on z/OS?
What do I want to backup? Is the wrong question.
The real question should be What do I want to restore? For example I can get a copy of the operating system from my original download files – or from IBM, but I need to be able to restore the files particular to me. It is better to restore the total system rather than rebuild it, because of all the additional configuration you had to do (which you may not have record of). The JCL I have written, the data in the database or MQ queue files, security profiles.
What situations do need to restore from?
It can range from
- I messed up – I edited a file, and now it does not work. I cannot undo the changes. I deleted a file. I want to go back to last week’s copy.
- By accident you had two copies of a program updating a file – and corrupting it.
- The database change you made cannot be undone – you added a new field, and now the record length is longer than the 4KB buffers.
- There has been an I/O error on the disk (though this is rare).
- I had my laptop stolen.
- My 3 year old child used my hard drive as a toy and found it does not float on water.
You also need to ask how long do I have to recover? If the answer is a week, then you can order a new hard drive, and wait a week for it to be delivered. If you need it back within hours, you’ll have a spare disk just in case (or you did a make copies to this disk – so all you need to do is use it).
Setting up z/OS
As a rule, with ADCD you should not use any of the ADCD volume for your own data. Create your own volumes and put your data on that. Create a user catalog, and use alias’s from the master catalog for this user catalog. If you have a new ADCD system you need to import the user catalog, and redefined the aliases.
Backup the USER.* data sets. Do not change the ADCD.* or SYS1.* data sets.
Some of the subsystems, DB2, CICS and MQ have data files on the A4PRD* volumes. This means you need to backup the volumes – and will be a challenge during migration.
When can I backup?
You should backup when the files are not being used.
- You can edit a file, use tso xmit to make a copy of the PDS, then save the file you were editing. That is OK. Using TSO XMIT while the file is being saved could cause a consistency problem.
- You need to backup some files as logical files, so for example backup the MQ.PAGESET. If this data set was spread across two disks, and you do an image copy of the first disk, followed by the image copy of the second disk, the data is likely to be inconsistent (if you restore you may not find out for a week after the restore!) MQ (and DB2) have logic to be able to recover when a logical dataset is restored. Some systems have a quiesce capability which stop activity to the file, without stopping the subsystem.
- Doing full volume backups should be done when the volume is not in use, either the z/OS is down, or the volume has been varied offline and removed from zPDT. Shutting down may be better, so all the volumes are consistent together. Sometimes there is data in buffers which has not been written to disk (lazy write), so you have to be careful.
You might try to backup only what has changed. This could be difficult. Unless the disks/files are read only, there is a chance that a file has changed, or a file has been put on a disk.
How do I backup files?
PDS and sequential files.
You can use the TSO XMIT (TRANSMIT) command to take a file or library and create a file which is easy to transport.
To restore it you use TSO RECEIVE indsn(…) newname(abc…) so can have the current and restored versions with different names. This allows you to process just one, or as many members as you want.
Files in USS
The file behind the filesystem is a VSAM file.
You could use unix commands like tar or pax to package up a file or directory. The output can be a file in the file system or into a z/OS dataset.
You could use ADRDSSU to backup the whole file system – see the next topic.
Other files
Traditionally these files are backed up use the ADRDSSU or AMATERSE (or both) utilities which can backup the file, and any indexes etc that go with it. The output can be a z/OS dataset, or DUMPed to tape.
Full volume backups
Shutdown z/OS down cleanly, stop zPDT (to ensure buffers in Linux are flushed), and backup the linux files. Restart z/OS.
Where do I backup to?
To recover from operator errors on “user files”, having the backup on z/OS may be enough.
To be able to recover from system problems, or disk problems, put the backups on a different file systems. If my z/OS system is on the SSD on my laptop, have the files go to an external file system. Some people will have their hard drives copied to another disk system, or even “off site”.
Getting backups out of z/OS
You can use FTP into TSO or USS to copy the files. If you use pax output to a TSO file, you can ftp into TSO. If you pax output into an unix file, FTP into USS.
You can also virtual tape, so ADRDSSU writes to a tape which maps to a file on the Linux file.
Having backed up the files what then? Plan for a whoops.
- It is worth checking that your backups restore, for example restore to a spare HDD, and try to boot from it.
- It is also worth checking that you are backing up what you think you are backing up. I know of one customer who was backing up the MQ pagesets, but did not change the backup job when they added more page sets. I have been guilty or repeating a line and not changing the data set name, so data set A was backed up twice, and data set B was not backed up.
- Determine how long it will take to restore disks, restart, and recover the file(s) of interest. If this duration is too long – review your backup and restore procedures.
What next?
I asked about backup on the zPDT group forum and had lots of great comments. Below is a summary of the comments.
- Use of Clonezilla. This is a partition and disk imaging/cloning program similar to True Image® or Norton Ghost®. It helps you to do system deployment, bare metal backup and recovery.
- Use ADRDSSU DUMP followed by AMATERSE to make the z/OS backups smaller.
- Use of a Synology Network Addressed Storage for your backups. Synology has comments like “Good for home users and small businesses”.
- Use ADRDSSU to dump to a volume. Vary volume offline, then backup the volume.
- Do not use any of the AD-CD supplied volumes for your data. Create your own volume(s) and simply add them to the devmap for new releases. You need to have a usercatalog on your volume(s) and import it to subsequent releases. You can try to make ALIAS definitions carry forward; I usually just recatalog my datasets for each new release.
- Use LVM snapshots. With the snapshot Linux grsync with an external drive
- Use of Borg. The main goal of Borg is to provide an efficient and secure way to backup data. Borg cuts all data into chunks, builds a hash and if the hash is not yet known, the chunk is compressed and stored in a repository. Otherwise only a pointer is set for the chunk in the current archive. This saves a lot of time and disk space (after the initial backup) because only the changed parts of the z-disk images are compressed and stored into the archive.
How long will it take?
This depends on the media you are using, and how much data. On my laptop copying an 8GB volume from HDD to SSD took about 4 minutes or about 30 MB/second. Compressing it may speed this up.
Some good JCL examples.
Thanks to James Alexander from Hostbridge for the following examples.
The user submits a tape job with an extra "mount" tape step:
//EXP EXPORT SYMLIST=(DSNAME,UNIT,HLQ,VOL)
//*
// SET HLQ=MYHLQ
// SET DSNAME=BACKUP.D999999.DFDSS
// SET UNIT=591
// SET VOL=J00001
//*
//MOUNT EXEC MOUNT,UNIT=&UNIT,DSNAME=&DSNAME,VOL=&VOL
//*
//* What follows is a standard DFDSS backup to tape. We compress
//* it here so less disk space is used.
//*
//BACKUP EXEC PGM=ADRDSSU,REGION=0K
//SYSPRINT DD SYSOUT=*
//TAPE1 DD UNIT=&UNIT,VOL=SER=(&VOL),
// DISP=NEW,DSN=&DSNAME,LABEL=(1,SL)
//SYSIN DD *,SYMBOLS=JCLONLY,DLM=$
DUMP DATASET( -
INCLUDE(&HLQ..** ) -
) -
OUTDDNAME(TAPE1) -
TOLERATE(ENQFAILURE) -
OPTIMIZE(4) -
COMPRESS
$
//
The mount step executes AWSCMDX that runs a Linux script. If the “DSNAME” tape file exists it mounts it; if not it copies a tape template file and then mounts it. A Linux job fires once an hour and syncs all of the files in the tape directory to AWS S3. With this any user can run a tape job and get offsite backups, Using the same methodology they can also do their own restores.
Here is the mount proc:
//MOUNT PROC UNIT=590,DSNAME=BAD.DATASET.NAME,VOL=T00001
//*
//X EXPORT SYMLIST=(DSNAME,UNIT,VOL)
//S SET UNIT=&UNIT,DSNAME=&DSNAME,VOL=&VOL
//*
//M EXEC PGM=AWSCMDX,PARMDD=MYPARMS
//SYSPRINT DD SYSOUT=*
//TAPE DD UNIT=(580,,DEFER),LABEL=(1,BLP),VOL=SER=123456,DSN=X
//MYPARMS DD *,SYMBOLS=JCLONLY
./mountfile &UNIT &DSNAME &VOL
/*
And here is the mountfile script in Linux:
#!/bin/bash
Unit=$1
Filename='/z/backup/tapes/'$2
Template='/z/backup/TapeTemplate'
echo 'Checking to see if the tape file exists'
if ! [ -e "$Filename" ]
then
echo 'File does not exist copying template'
cp $Template $Filename
fi
echo 'Mounting '$Filename' on unit '$Unit
awsmount $Unit -m $Filename
ColinPaice 