Category: Catalog

How difficult is it to delete a data set – it can be harder than you think.

I was using a volume on one zD&T system, and used the same volume on the next version of zD&T. Some of the datasets are SMS managed, and were cataloged in the first system’s master catalog.

Now I’m on the latest level of zD&T, I am trying to delete unwanted data sets, and this was a challenge, as all of the usual methods did not work.

I can use ISPF 3.4 and specify the volume USER05. There are some old datasets, such as SYS1.S0W1.Z24C.DMP00001. These are not cataloged. They are only visible in ISPF 3.4 if you specify the volume. If you try to browse the data set it gives data set not cataloged.

I failed to delete the dataset using the usual techniques….

From ISPF

If you try to delete it from ISPF 3.4 it says “Data set not cataloged”,

From JCL

For example

//IBMDEL   JOB 1,MSGCLASS=H 
//TSO     EXEC PGM=IEFBR14 
//DDS      DD  DSN=SYS1.S0W1.Z24C.DMP00001,VOL=SER=USER05, 
//             DISP=(OLD,DELETE) 
/*

This fails with the error message

IEFA107I IBMDEL TSO DDS – DATA SET SYS1.S0W1.Z24C.DMP00001 NOT FOUND

From IDCAMS

//IBMDEL2  JOB 
//STEP1    EXEC  PGM=IDCAMS 
//DD1      DD    VOL=SER=USER05,UNIT=3390,DISP=OLD 
//*            DSN=SYS1.S0W1.Z24C.DMP00003 
//SYSPRINT DD    SYSOUT=A
//SYSIN   DD * 
     DELETE    - 
           SYS1.S0W1.Z24C.DMP00001      - 
           FILE (DD1) - 
           PURGE 
/*

Fails with

IDC3012I ENTRY SYS1.S0W1.Z24C.DMP00001 NOT FOUND
IDC3009I ** VSAM CATALOG RETURN CODE IS 8 – REASON CODE IS IGG0CLEG-42

The problem…

The dataset is on an SMS managed volume. All data sets managed by SMS have to be cataloged. The catalog that owns the dataset is on a non existant system – the old master catalog from the system from 3 years ago. The outcome is that there is no catalog available to locate the data set.

On the SMS volume USER05 is the SMS dataset SYS1.VVDS.VUSER05. This is like a catalog for the volume, and contains information about all the data sets on the SMS volume. You can use IDCAMS to print this data set (in hex).

The solution…

You need to recatalog it – then delete it

recatalog

//IBMRECAT JOB 
//RECATSMS EXEC PGM=IDCAMS 
//SYSPRINT DD   SYSOUT=* 
//SYSIN    DD   * 
  DEFINE NONVSAM(NAME(SYS1.S0W1.Z24C.DMP00001   ) - 
    OWNER(COLIN ) - 
    VOLUMES(USER05)        - 
    DEVT(3390  )    - 
    RECATALOG) 
  LISTCAT ENT(SYS1.S0W1.Z24C.DMP00001) ALL 
/*

This gave 

  DEFINE NONVSAM(NAME(SYS1.S0W1.Z24C.DMP00001   ) -                                           
    OWNER(COLIN ) -                                                                           
    VOLUMES(USER05)        -                                                                  
    DEVT(3390  )    -                                                                         
    RECATALOG)                                                                                
IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0                                             
                                                                                                      
  LISTCAT ENT(SYS1.S0W1.Z24C.DMP00001) ALL                                                    
NONVSAM ------- SYS1.S0W1.Z24C.DMP00001                                                               
     IN-CAT --- CATALOG.Z31B.MASTER                                                                   
     HISTORY                                                                                          
       DATASET-OWNER------COLIN     CREATION--------2025.296                                          
       RELEASE----------------2     EXPIRATION------0000.000                                          
       ACCOUNT-INFO-----------------------------------(NULL)                                          
     SMSDATA                                                                                          
       STORAGECLASS -----SCBASE     MANAGEMENTCLASS---(NULL)                                          
       DATACLASS --------(NULL)     LBACKUP ---0000.000.0000                                          
     ENCRYPTIONDATA                                                                                   
       DATA SET ENCRYPTION-----(NO)                                                                   
     VOLUMES                                                                                          
       VOLSER------------USER05     DEVTYPE------X'3010200F'     FSEQN------------------0             
     ASSOCIATIONS--------(NULL)                                                                       
     ATTRIBUTES

Having done this, I can now browse SYS1.S0W1.Z24C.DMP00001 in ISPF without specifying a volume, so proving it is now cataloged.

Delete it

If I now try to delete it I get

IEC331I 050-088(,USER05),COLIN,ISPFPROC,VCMP,IGG0CLE4
IEC614I SCRATCH FAILED – RC 008, DIAGNOSTIC INFORMATION IS (040942D1),
ISPFPROC,USER05,SYS1.S0W1.Z24C.DMP00001

The codes for IEC331I are given in IEC3009I. If you search for “code 50” then page down till you get code 88, it says

Explanation: A VVR or NVR with the correct component name was found, but the catalog name did not match. On a delete request, the BCS record will be deleted, but the VVR or NVR and the format 1 DSCB will not be scratched. There is no SFI data.

I had to use

//IBMDEL2  JOB 
//STEP1    EXEC  PGM=IDCAMS 
//DD1      DD    VOL=SER=USER05,UNIT=3390,DISP=OLD 
//SYSPRINT DD    SYSOUT=A 
//SYSIN   DD * 
  DELETE  SYS1.S0W1.Z24C.DMP00001    NVR FILE(DD1) 
/*

Which says delete the data set (and remove it from the VVDS dataset).
This job was successful, and the data set was no longer on the volume.

Sometimes I had to remove the NVR from the delete statement.
My overall JCL was 

//IBMDEL   JOB  1 
//E1 EXPORT SYMLIST=* 
// SET NAME='IZUSVR.JVM.IZUSVR1.D250809.T080903.X001' 
// SET VOL='USER06' 
//RECATSMS EXEC PGM=IDCAMS 
//DD1      DD    VOL=SER=&VOL,UNIT=3390,DISP=OLD 
//SYSPRINT DD   SYSOUT=* 
//SYSIN    DD   *,SYMBOLS=JCLONLY 
                                                                 
  DEFINE NONVSAM(NAME(&NAME.) - 
    OWNER(COLIN ) - 
    VOLUMES(&VOL.)        - 
    DEVT(3390  )    - 
    RECATALOG) 
                                                                 
  LISTCAT ENT(&NAME. ) ALL 

  DELETE  &NAME.                     NVR FILE(DD1) 

  DELETE  &NAME.                         FILE(DD1) 
/*

Migrating an ADCD z/OS release: user data sets, user catalogs and High Level Qualifier

Start here:Migrating an ADCD z/OS release to the next release.

I have covered datasets, aliases and catalogs in One minute mvs: catalogs and datasets.

If your master catalog is a mess

If you have your personal data sets in the master catalog instead of a user catalog, see Here’s another nice mess I’ve gotten into!

You could start again on the new system and do it properly.

Note down which volumes your data sets are on. On the new system define a user catalog and alias, and use ISPF 3.4, to list the contents of the volume and use the C local command to catalog the data set.

I found the easiest way of doing VSAM datasets is a backup and restore

If you have a user catalog for your user data sets

You need to get a list of the user aliases in the master catalog on the old system, and the user catalogs the aliases refer to, before trying to import them into the new system. I could not find a way to list the master catalog from another system. See here for example JCL

To access your data sets in a user catalog you need to import the catalog into the master catalog on the new system, and define the aliases for your data to the catalog.

On your new system, you will not have immediate access to your data sets, so you either need

  • to access a dataset, and specify the volser information,
  • or you need to have a copy of the JCL to import a user catalog, and define alias, stored in a note pad area outside of z/OS. Edit a member such as USER.*.PROCLIB(TEMP) and paste the import catalog JCL. When that has worked, paste in the add alias to your datasets ( COLIN.*). Once you have done this, you can use ISPF 3.4 and use COLIN.* to locate your data sets.

Once you have your list of aliases and user catalogs, you can IPL the new system and import the user catalogs, and define the aliases.

Product data sets

There is usually an alias in the master catalog for a program product. The alias may have the product release as part of the name. If you have customised any of the product owned datasets, you may need to copy the data off.

One minute mvs: catalogs and datasets.

In days of old, when 64KB was a lot of real storage, to reference a data set you had to specify the data set name and the DASD volume the data set was on. DSN=MY.JCL,VOL=SER=USER00

After this, the idea of a catalog was developed. Just like the catalog in a library, it told you where things were stored. When you created a data set, and specified DISP=(NEW,CATLG), the data set name and volume were stored in the catalog. When you wanted to use a data set, and did not specify the volume, then the catalog was used to find the volume for the data set.

As systems grew and the number of data sets grew, the catalog grew and quickly became difficult to manage. For example if you deleted data sets, the entry was logically removed from the catalog, resulting in gaps in the catalog.

After this a multi level catalog was developed. You have one master catalog. You can have many user catalogs. You define an alias in the master catalog saying for data sets starting with a specific high level qualifier, use that user catalog.

When a userid was created, most system programmers would also create an alias pointing to a user catalog. They may define a user catalog for each user, or a user catalog could be shared by many aliases.

The catalogs are managed by the VSAM component of z/OS.

Entity naming

PDS and sequential files are referred to as datasets. VSAM provides simple database objects,

  • Relative Record ( where you say get me the n’th record)
  • Key sequence. You define a primary index, and you can an index on different columns using an ALTERNATIVE INDEX. 

VSAM uses the term cluster to what you use in you JCL or application. A cluster has a data component, and zero or more index components.

Moving systems.

I have been running on a self contained ADCD system at z/OS level 2.4. I have recently installed a self contained system at z/OS 2.5. How do I get my data sets into the new system?
You can import connect a user catalog into a new (master) catalog, and define an alias in the new master catalog pointing to the user catalog.

When I did this I could then see my COLIN.* data sets. To be able to use the data sets, I need the volumes to be attached to the z/OS system.

Useful IDCAMS commands

In batch you use the IDCAMS program (IDC = prefix for VSAM, AMS is for Access Management Services!)

If you do not specify a catalog, it defaults to the master catalog.

Create a user catalog

//IBMDF  JOB 1,MSGCLASS=H                                   
//S1  EXEC PGM=IDCAMS                                       
//SYSPRINT DD SYSOUT=*                                      
//SYSIN DD *                                                
 DEFINE USERCATALOG -                                       
      ( NAME('A4USR1.ICFCAT') -                             
        MEGABYTES(15 15) -                                  
        VOLUME(A4USR1) -                                    
        ICFCATALOG  -                                       
        FREESPACE(10 10) -                                  
        STRNO(3  ) ) -                                      
        DATA( CONTROLINTERVALSIZE(4096) -                   
              BUFND(4) ) -                                  
        INDEX(BUFNI(4) )                                    
/*

Creating an alias to use the catalog

The JCL below creates two aliases in the master catalog. They both point to a catalog called A4USR1.ICFCAT (which is in the master catalog)

//IBMUSERT JOB 1,MSGCLASS=H                                     
//S1  EXEC PGM=IDCAMS,REGION=0M                                 
//SYSPRINT DD SYSOUT=*                                          
//SYSIN DD *                                                    
   DEFINE ALIAS (NAME(BACKUP) RELATE('A4USR1.ICFCAT') )         
   DEFINE ALIAS (NAME(COLIN ) RELATE('A4USR1.ICFCAT') )         
/*

To import an existing user catalog into a master catalog

//IBMUSERT JOB 1,MSGCLASS=H                                   
//S1  EXEC PGM=IDCAMS,REGION=0M                               
//SYSPRINT DD SYSOUT=*                                        
//SYSIN DD *                                                  
   IMPORT CONNECT -                                           
      OBJECTS -                                               
        (('A4USR1.ICFCAT' VOLUME(A4USR1) DEVICETYPE(3390)  -  
        ))                                                    
/*

From the information provided, the master catalog knows the name, and location of the user catalog.

List an entry

You can list information about an entry, such as a data set, or a catalog using the LISTCAT command

LISTCAT ENT(COLIN.USERS) ALL

Listing aliases

You can use the IDCAMS command LISTCAT with alias

LISTCAT ALIAS

which gives a one line per entry list of all of the aliases

ALIAS --------- ADCDA       
ALIAS --------- ADCDB       
ALIAS --------- ADCDC       
ALIAS --------- ADCDD       
...

LISTCAT ALIAS ALL

gives 

ALIAS --------- ADCDA                                                       
     ...          
     ENCRYPTIONDATA                                                         
       DATA SET ENCRYPTION-----(NO)                                         
     ASSOCIATIONS                                                           
       USERCAT--USERCAT.Z24C.USER                                           
ALIAS --------- ADCDB                                                       
    ...      
     ENCRYPTIONDATA                                                         
       DATA SET ENCRYPTION-----(NO)                                         
     ASSOCIATIONS
       USERCAT--USERCAT.Z24C.USER

So we can see that the the alias ADCDA maps to user catalog USERCAT.Z24C.USER

Listing a catalog

The command

LISTCAT CATALOG(USERCAT.Z24C.USER)

gives

CLUSTER ------- 00000000000000000000000000000000000000000000       
   DATA ------- USERCAT.Z24C.USER                                  
   INDEX ------ USERCAT.Z24C.USER.CATINDEX                         
NONVSAM ------- ADCDA.S0W1.ISPF.ISPPROF                            
NONVSAM ------- ADCDA.S0W1.SPFLOG1.LIST                            
NONVSAM ------- ADCDB.S0W1.ISPF.ISPPROF                            
...                           
CLUSTER ------- SYS1.VVDS.VC4CFG1                                  
   DATA ------- SYS1.VVDS.VC4CFG1

Which shows there is a data component of the catalog called USERCAT.Z24C.USER, and there is index component called USERCAT.Z24C.USER.CATINDEX.

The catalog has a data component (USERCAT.Z24C.USER) and an index component USERCAT.Z24C.USER.CATINDEX.

Within the catalog are entries for data sets such as ADCDA.S0W1.ISPF.ISPPROF, and system (DFDSS) dataset SYS1.VVDS.VC4CFG1 - which contains information what is on the SMS DASD volume C4CFG1.

Here’s another nice mess I’ve gotten into! My master catalog.

Or “How to clean up the master catalog when you have filled it up with junk”. Looking at my z/OS system,  I was reminded of my grandfathers garage/workshop where the tools were carefully hung up on walls, the chisels were carefully stored a a cupboard to keep them sharp etc.   He had boxes of screws, different sizes and types in different boxes.   My father had a shed with a big box of tools.  In the box were his chisels, hammers, saws etc..  He had a big jar of “Screws – miscellaneous – to be sorted”.    The z/OS master catalog should be like my grandfather’s garage, but I had made it like my father’s shed.

Well, what a mess I found!   This blog post describes some of the things I had to do to clean it up and follow best practices.

In days of old, well before PCs were invented, all data sets were cataloged in the master catalog.  Once you got 10s of thousands of data sets on z/OS, the time to search the catalog for a dataset name increased, and the catalogs got larger and harder to manage.  They solved this about 40 years ago by developing the User Catalog  – a catalog for User entries.
Instead of 10,000 entries for my COLIN.* data sets, there should be an Alias COLIN in the Master Catalog which points to another catalog which could be just for me, or can be shared by other users. This means that even if I have 1 million datasets in the user catalog, the access time for system datasets is not affected.  What I expected to see in the master catalog is the system datasets, and aliases for the userids.  I had over 400 entries for COLIN.* datasets, 500 BACKUP.COLIN.* datasets, 2000, MQ.ARCHIVE.* datasets etc.  What a mess!

Steps to solve this.

Prevention is better than cure.

I’ve more recently created Keeping people out of the master catalog, which should help you prevent a mess in your catalog.

You can use the RACF Option PROTECTALL.  This says a userid needs a RACF profile before it can create a dataset.  This means each userid (and group) needs a profile  like ‘COLIN.*’, and give the userid access to this profile.  Once you have done this for all userids, you can use the RACF command SETROPTS PROTECTALL(WARNING) to enable this support.   This will allow users to create datasets, when there is no profile, but produces a warning message on the operator console – so you can fix it.  An authorised person can use SETROPTS NOPROTECTALL to turn this off.  Once you have this running with no warnings you can use the command SETROPTS PROTECTALL to make it live – without warnings, and you will live happily ever after, or at least till the next problem.

Action:

  1. Whenever you create a userid (or a high level qualifier) you need to create the RACF dataset profile for the userid (high level qualifier)
  2. You also need to set up an ALIAS for the new userid(high level qualifier) to point to a User Catalog.

How bad is the problem?

You can use IDCAMS to print the contents of a catalog

//S1 EXEC PGM=IDCAMS 
//SYSPRINT DD SYSOUT=* 
//SYSIN DD * 
LISTCAT CATALOG(CATALOG.Z24A.MASTER) NAME 
/*

This has output like

NONVSAM ------- BACKUP.USER.Z24A.VTAMLST.D201210 
NONVSAM ------- BACKUP.USER.Z24A.VTAMLST.D201222 
NONVSAM ------- BACKUP.USER.Z24A.VTAMLST.D201224 
ALIAS --------- BAQ300

This says there are datasets BACKUP… which should not be in the catalog.
There is an Alias BAQ300 which points to a user catalog.   This is what I expect.

The IDCAMS command

LISTCAT ALIAS CATALOG(CATALOG.Z24A.MASTER) ALL

list all of the aliases in the catalog, for example

ALIAS --------- BAQ300 
  ...  
  ASSOCIATIONS 
    USERCAT--USERCAT.Z24A.PRODS

This shows for high level qualifier BAQ3000, go and look in the user catalog  USERCAT.Z24A.PRODS.

Moving the entries out of the Master Catalog

The steps to move the COLIN.* entries out of the Master Catalog are

  1. Create a User Catalog
  2. Create an ALIAS COLIN2 which points to this User Catalog. 
  3. Rename COLIN…. to COLIN2….
  4. Create an ALIAS COLIN for all new data sets.
  5. Rename COLIN2… to COLIN…
  6. Delete the ALIAS COLIN2.

Create a user catalog

Use IDCAMS to create a user catalog

 DEFINE USERCATALOG - 
   ( NAME('A4USR1.ICFCAT') - 
   MEGABYTES(15 15) - 
   VOLUME(A4USR1) - 
   ICFCATALOG - 
   FREESPACE(10 10) - 
   STRNO(3 ) ) - 
   DATA( CONTROLINTERVALSIZE(4096) - 
       BUFND(4) ) - 
   INDEX(BUFNI(4) )

To list what is in a user catalog

Use a similar IDCAMS command to list the master catalog 

LISTCAT ALIAS CATALOG(A4USR1.ICFCAT) ALL

Create an alias for COLIN2

 DEFINE ALIAS (NAME(COLIN2) RELATE('A4USR1.ICFCAT') )

Get the COLIN.* entries from the Master Catalog into the User Catalog

This was a bit of a challenge as I could not see how to do a global  rename.

You can rename non VSAM dataset either using ISPF 3.4 or use the TSO rename command in batch.

The problem occurs with the VSAM data sets.   When I tried to use the IDCAMS rename, I got an error code IGG0CLE6-122 which says I tried to do a rename, which would cause a change of catalog.

The only way I found of doing it was to copy the datasets to a new High Level Qualifier, and delete the originals.   Fortunately DFDSS has a utility which can do this for you.

//S1 EXEC PGM=ADRDSSU,REGION=0M PARM='TYPRUN=NORUN' 
//SYSPRINT DD SYSOUT=* 
//SYSIN DD * 
  COPY - 
     DATASET(INCLUDE(COLIN.** )) - 
     DELETE -     
     RENUNC(COLIN2) 
/*

Most of the data sets were “renamed” to COLIN2… but I had a ZFS which was in use, and some dataset aliases.  I used

  •  the TSO command unmount filesystem(‘COLIN.ZCONNECT.BETA.ZFS’)
  • the  IDCAMS command DELETE COLIN.SCEERUN ALIAS for each of the aliases.

and reran the copy job.   This time it renamed the ZFS.  The renaming steps are

  • Check there are no datasets with the HLQ COLIN.
  • Define an alias for COLIN in the master catalog to point to a user catalog.
  • Rerun the copy job to copy from COLIN2 back to COLIN.
  • Mount the file system.
  • Redefine the alias to data sets (eg COLIN.SCEERUN).
  • Delete the alias for COLIN2.

To be super efficient, and like my grandfather, I could have upgraded the SMS ACS routines to force data sets to have the “correct” storage class, data class, or management class.  The job output showed  “Data set COLIN2.STOQ.CPY has been allocated with newname  COLIN.STOQ.CPY using STORCLAS SCBASE,  no DATACLAS, and no MGMTCLAS“.  These classes were OK for me, but may not be for a multi-user z/OS system.

One last thing, don’t forget to add the new user catalog to your list of datasets to backup.