How do I diff on Z/OS with Unix files and directories

I wanted to compare the contents of two directories in Unix System Services on z/OS before I merged them. This took me some time to do because the documented is lacking.

With ISPF you can use SUPERC (3.13), give it two PDSs and it shows you the differences.

On Unix there is the diff command. This can compare individual files, or directories. It can display just the changes, or the changes in context.

File a

AOnly 
line1
Aline2
line3

File b

line1 
Bline2
line3
BOnly
BOnly2

Using ispf edit compare to show differences

You can use diff to show the differences in two files, but it is not easy to understand. ISPF EDIT has the compare facility. If you know two files are different you can use

  • oedit /etc/zexpl/rseapi.env
  • use the primary command compare
    • enter the fully qualified name /u/ibmuser/zexpl/rseapi.env,, in the “Name . . . . .” field. if you specify +/filename the + means the same directory.
    • press pf3 and it will show the differences

Because I tend to remove comments to make it easier to see the content, I tend to use

  • oview /etc/ssh/sshd_config you get into ISPF edit, but no changes are saved.
  • Comments start with a #. x all;f ‘#’ all 1 1; f ‘ ‘ 1 1 all;del all nx removes comment lines and blank lines
  • reset to show the hidden lines
  • compare I then specify my version of the file, and see the changes.

  • Lines like ====== TrustedUserCAKeys /etc/ssh/user_ca_key.pub are from my copy.
  • Lines in green with a line number are in both files 000006 Subsystem sftp /usr/lib/ssh/sftp-server
  • Lines like .OAAAA UseDNS yes are from the base file

Update your version, make a copy of the original, then replace the original with your version.

diff -c1 a b – show the files and the changes in context

The line prefix for input file going to output file

  • – to be removed
  • ! to be changed
  • + to be added

The command diff -c1 a b gives

*** a Tue Aug 27 02:48:56 2024              
--- b Tue Aug 27 02:50:41 2024
***************
*** 1,4 ****
- AOnly
line1
! Aline2
line3
--- 1,5 ----
line1
! Bline2
line3
+ BOnly
+ BOnly2
  • *** a Tue Aug 27 02:48:56 2024 the first file name, and last changed date
  • — b Tue Aug 27 02:50:41 2024 the second file name, and last changed date
  • *** 1,4 **** the*** show it is file 1, lines 1 to 4
  • – AOnly this line is in file a is not in file b, so would need to be removed(-) from file a
  • ! Aline2 this line is in file b – but different
  • — 1,5 —- this is file b, lines 1 to 5
  • ! Bline2 this line is also in file a – but different
  • + BOnly this line is in file b and and was additional(+) to file a

When one file exists but is empty you get output like

*** /etc/resolv.conf Wed Mar  6 11:54:50 2024                         
--- /u/ibmuser/temp/resolv.conf Thu Dec 7 05:40:24 2023
***************
*** 0 ****
--- 1,2 ----
+ nameserver 127.0.0.1
+ TCPIPJOBNAME TCPIP

which follows the rules I explained above. *** 0 **** shows the content after line 0 is empty, because the next line is — 1,2 —- from the other file.

diff a b – show just the changes

gives

1d0             
< AOnly

3c2
< Aline2
---
> Bline2

4a4,5
> BOnly
> BOnly2

The output can be split into sections. The first line of each section is like

  • 1d0 the first line of a needs to be deleted from b, line 0
  • 3c2 line 3 of a is changed from line 2 of b
  • 4a4,5 lines 4,5 of b need to be added to a

The < and > tell you which file the data came from

When data is changed it gives the lines

  • < content of file a
  • output divider
  • > content of file b

When the data is in file a and not file b

  • < contents of file a

When the data is in file b and not file a

  • > contents of file b

diff -s dir1 dir2 compare the directory contents

If you specify -s, or just specify two directories, it compares the directory content.

You can use

diff -c1 dir1 dir2 

the -c1 to display the contents (how I like it).

With the directory entries you get records like

Only in /u/ibmuser/temp: test.tar 
Common subdirectories: /etc/wbem and /u/ibmuser/temp/wbem
Only in /etc: yylex.c
diff -c1 /etc/hosts /u/ibmuser/temp/hosts
*** /etc/hosts Wed Mar 6 11:06:55 2024
--- /u/ibmuser/temp/hosts Tue Feb 28 12:43:07 2023

You can find which files are missing from /etc , by using grep ‘Only in /u/ibmuser/temp’ on the output.

It shows the command used for the individual files, and the output

diff -c1 /etc/hosts /u/ibmuser/temp/hosts
*** /etc/hosts Wed Mar 6 11:06:55 2024
--- /u/ibmuser/temp/hosts Tue Feb 28 12:43:07 2023
...

diff -s -r dir1 dir2 compare the directory contents

The -r option displays the data recursively.

Using a VSAM file from another system.

I have been working with two levels of ADCD z/OS system, Z24C and Z25D. I want to be able to use VSAM files from the z24C level system on the z25D system.

With non VSAM files, it is easy. I can define an alias for a high level qualifier such as my userid COLIN which points to the user catalog with my data sets in it. It is a bit harder with VSAM files, especially where there is a file with the same name of both systems (such as CSF.CSFCKDS).

A VSAM PATH is an alias for VSAM files.

Conceptually the first step is

//IBMDEFP JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEF PATH -
(NAME(COLINQ.AUT420.AUT420D.CSI.Z24C) -
PATHENTRY( AUT420.AUT420D.CSI ) -
) -
CATALOG( USERCAT.Z24C.PRODS )
/*

File AUT420.AUT420D.CSI is in catalog USERCAT.Z24C.PRODS

The above JCL will create a name COLINQ.AUT420.AUT420D.CSI.Z24C which points to the file AUT420.AUT420D.CSI in catalog USERCAT.Z24C.PRODS. The entry COLINQ.AUT420.AUT420D.CSI.Z24C is put in the same catalog.

If you use ISPF 3.4 it will not find the dataset.

Create an alias for the High Level Qualifier

//IBMUSERT JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE ALIAS (NAME(COLINQ) RELATE('USERCAT.Z24C.PRODS'))
/*

The above JCL will create an alias COLINQ, and says to find any datasets beginning with COLINQ go and look in catalog USERCAT.Z24C.PRODS.

To import the catalog into the current system

//IBMIMPC JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//CAT DD DISP=SHR,DSN=ADCD.LIB.JCL,VOL=SER=C4SYS1,
// UNIT=3390
//SYSPRINT DD SYSOUT=A
//SYSIN DD *
IMPORT -
OBJECTS -
((USERCAT.Z24C.PRODS -
VOLUME(C4SYS1) -
DEVICETYPE(3390))) -
CONNECT -
CATALOG(CATALOG.Z25D.MASTER)
/*
//

The above JCL says import the catalog USERCAT.Z24C.PRODS on volume(C4SYS1), type (3390) into the (master) catalog CATALOG.Z25D.MASTER.

If the system needs to find USERCAT.Z24C.PRODS, it has enough information to be able to find it.

What you actually do

Now that you understand the process, the process you should follow is

  • Import the catalog into the current system.
  • Define an High Level Qualifier alias to point to the catalog. I might pick COLIN4C ( for the z24C system).
  • Create a path using COLIN4C as the high level qualifier of the data set, for each VSAM file.

You should then be able to see your data set in ISPF 3.4

To access the Z24C /u ZFS files system on the Z25D system I used

 IMPORT - 
OBJECTS -
((CATALOG.Z24C.MASTER -
VOLUME(C4SYS1) -
DEVICETYPE(3390))) -
CONNECT -
CATALOG(CATALOG.Z25D.MASTER)

DEFINE ALIAS (NAME(Z24CMAST) RELATE('CATALOG.Z24C.MASTER'))

DEFINE PATH -
(NAME(Z24CMAST.ZFS.USERS ) -
PATHENTRY( ZFS.USERS )) -
CATALOG(CATALOG.Z24C.MASTER)

In Unix I created a directory

mkdir /u/old

The mounted the file system in ISPF option 6 TSO

mount filesystem('Z24CMAST.ZFS.USERS') mountpoint('/u/old') type(ZFS)  
mode(read)

I could then access the files from /u/old/…

NO? Migrating an ADCD z/OS release:RACF

This post is a very quick post in response to a question about migrating your RACF data to a different system. I will update it when I have more time.

This is one of a series of posts about migrating to a newer level of ADCD.

This covers RACF.

A typical z/OS migration is to take your existing system and upgrade it. With ADCD the operating system is replaced and you have to move to it, taking your data with you.

You need to think about your RACF definitions, and migrates your specific data into the newer RACF database. This will include userid, profiles, permissions and digital certificates.

Some of the changes are good practice and clean up. For example

Instead of a definition CICSTS55.* in class(STARTED), the profile is CICSTS.*.* in class(STARTED). Using the scripts described in this blog you can update your old system before you move to the new system. 

I have put files into a github repository.

Migrating certificates and keyrings

The JCL EXPCERT runs a rexx exec which create commands to export your certificates, imports them (on the new system), and recreates the keyrings (on the new system).

The exec issues the RACDCERT command to list all the keyrings for a userid, and processes that output.

//IBMCERT  JOB 1,MSGCLASS=H 
// EXPORT SYMLIST=(*)
//ADCDA EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//OUTPUT DD DISP=SHR,DSN=COLIN.MIG.DATA(EXPCERTD)
//IMPORT DD DISP=SHR,DSN=COLIN.MIG.DATA(IMPCERTD)
//RING DD DISP=SHR,DSN=COLIN.MIG.DATA(RINGD)
//SYSEXEC DD DISP=SHR,DSN=USER.Z24C.CLIST
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
%LRING COLIN.CERT.EXPORT START1 PASS4ME4
/*
//S2 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSIN DD DISP=SHR,DSN=COLIN.MIG.DATA(EXPCERTD)
//SYSTSPRT DD SYSOUT=*
/*  

The parameters are

  • the exported certificates high level qualifier. It will create data sets like COLIN.CERT.EXPORT.C14 . Once you have imported the certificates you need to delete these data sets.
  • the userid that owns the keyrings
  • the password for the exported certificates

The output data sets are

OUTPUT: this has data like

RACDCERT EXPORT(LABEL('AdcdShrCA')) -
CERTAUTH DSN('COLIN.CERT.EXPORT.C8')-
FORMAT(PKCS12DER) password('&PASSWORD')

IMPORT: This has data like

RACDCERT ADD('COLIN.CERT.EXPORT.C8') CERTAUTH -
TRUST withlabel('AdcdShrCA') PASSWORD('&PASSWORD')

RING: this has data like

RACDCERT ADDRING(RING) ID(START1)
RACDCERT ID(START1) CONNECT(RING(RING) –
  CERTAUTH DEFAULT usage(PERSONAL ) –
  LABEL(‘AdcdShrCA’) )

Migrating userids and profiles

RACF provides a DBSYNC rexx program to take two unloaded RACF databases and generate the RACF command statements to show the differences and help you convert from one to the other. One file will have “define resource XYZ” another file will have “delete resource XYZ”.

In theory you can just run the appropriate files and merge your old definitions into the new database. I have a lot of junk in my old database and wanted to be more selective as to what I copied across, and understand the differences.

I used the DBSYNC job to create the various files. I changed the DCB to DCB=(RECFM=VB,LRECL=600,BLKSIZE=6400)

The output of ALTFILE1 looks like

/*70:339:286*/ "altgroup PKIGRP   omvs( gid(0000990031))" 
/*75:373:315*/ "altuser ADCDA owner(IBMUSER ) dfltgrp(TEST ) noadsp n....
"noseclabel"

I use ISPF View on the data set, so I do not accidentally change the output.

I run the Rexx exec RACFONEL (in user.*.clist). This creates one line in the file for each logical line, and sorts the file alphabetically. For example

altuser ADCDA owner(IBMUSER  ) dfltgrp(TEST     ) noa..... noseclabel

Depending on what I am doing, I select a subset of the records for example, for my userid and use those records.

The Rexx exec RACSPLIT does the following

  • Cosmetic edits to make fields align – for example change ‘ group’ to ‘group’, changes revoke to resume to make it easier to compare records
  • Splits each line so it fits more into columns 1-80. Sometimes the lines are longer because the definition requires a long string.
  • It appends the records into the save file

Example output

altuser ZWESIUSR owner(IBMUSER  ) - 
dfltgrp(ZWEADMIN ) -
noadsp -
nospecial -
nooperations -
nogrpacc -
name(zzzzz) -
nodata -
noseclabel
altuser ZWESVUSR omvs( uid(0000990016 ) home('/apps/zowe/v10/home/z.....

If I am happy with the output, delete the top records in the file, and save it.

Do the same for the old and new RACF DBSYNC files and then use ISPF edit compare function to show the difference.

Once you know what the difference are you can run the updates for the changes you want to make to your new database.

Which of my ADCD disks should I move to my SSD device?

I’m working on moving to a newer version of ADCD, but I do not have enough space for all of the ADCD disks, on my SSD drive, so I am using an external USB device. Which of my new files should I move off the USB drive onto my SSD device for best performance?

Background

How much free space do I have on my disk?

The command

df -P /home/zPDT

gave

Filesystem   1024-blocks     Used Available Capacity Mounted on
/dev/nvme0n1p5 382985776 339351984 24105908      94% /home/zPDT

This shows there is not much free space. What is using all of the space?

ls -lSr

the -S is sort by size largest first, the -r is reverse sort, so the largest comes last.

This showed me lots of old ADCD files which I could delete. After I deleted them, df -P showed the disk was only 69% full.

zPDT “disks”

Each device as seen by zPDT is a process. For example

$ps -ef |grep 5079
colin 5079 4792 0 10:21 ? 00:00:00 awsckd --dev=0A94 --cunbr=0001

So process with pid 5079 is running a program awsckd passing in the device number 0A94

Linux statistics

You can access Linux statistics under the /proc tree.

less /proc/5079/io

gave

rchar: 251198496
wchar: 79167416
syscr: 4525
syscw: 1403
read_bytes: 78671872
write_bytes: 78655488
cancelled_write_bytes: 0

rchar: characters read

The number of bytes which this task has caused to be read from storage. This is simply the sum of bytes which this process passed to read(2) and similar system calls. It includes things such as terminal I/O and is unaffected by whether or not actual physical disk I/O was required (the read might have been satisfied from pagecache).

wchar: characters written

The number of bytes which this task has caused, or shall cause to be written to disk. Similar caveats apply here as with rchar.

read_bytes: bytes read

Attempt to count the number of bytes which this process really did cause to be fetched from the
storage layer. This is accurate for block-backed filesystems.

write_bytes: bytes written

Attempt to count the number of bytes which this process caused to be sent to the storage layer.

How to find the hot files

Use the Linux command

grep read_bytes -r /proc/*/io |sort -k2,2 -g

This finds the read_bytes for each process. It then sorts numerically (-g) and displays the output. For example

/proc/5088/io:read_bytes: 55910400
/proc/5078/io:read_bytes: 61440000
/proc/5091/io:read_bytes: 72916992
/proc/5079/io:read_bytes: 78671872
/proc/5076/io:read_bytes: 138698752
/proc/5074/io:read_bytes: 321728512

You can then display the process information

ps -ef |grep 5074

Which gave

… awsckd –dev=0A80 –cunbr=0001

From the devmap ( or z/OS) device 0A80 is C4RES1.

The disks with the most read activity were (in decreasing order) C4RES1, C4SYS1, C4PAGA, USER02, C4CFG1, C4USS1