Where’s my dump?

I had been working on a bug and getting out a System Dump (SDUMP). Then, after a fix, I stopped getting dumps, and just got a message with a return code indicating an abend had occurred. So where was my dump?

z/OS has Dump Elimination and Analysis DAE. This keeps a note of the dumps being taken, and can be configured to say “If you get the same abend many times, don’t bother taking a dump”. This stops you getting many identical dumps, and filling your disk storage.

In IPCS option 3 UTILITY – Perform utility functions, 5 DAE – Process DAE data, you can display the contents of the DAE information. On my system the data set is SYS1.DAE.

Command ===>                                                  Scroll ===> PAGE
Enter an Action Code next to an entry.
Enter / next to an entry to choose from a list of Action Codes.

Dataset: 'SYS1.DAE'
Dumps since last DAE Display: 0 Total Dumps suppressed: 214
Events since last DAE Display: 0 Suppression rate: 81%

A Last Last Total Date of Symptom String information:
C Date System Events Dump Abend Reason Module CSECT
_ 05/01/26 S0W1 7 04/28/26 S0EC6 055B0718 BPXINPVT BPXFSMNT
_ 05/01/26 S0W1 66 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD
_ 05/01/26 S0W1 10 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD
s 05/01/26 S0W1 53 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD

Selecting a record with S gave

                           Date      Time       System Name                    
Last (most recent) Event: 05/01/26 19:02:16 S0W1
Dump Taken: 04/28/26 18:44:59 S0W1

Symptoms used for Dump Suppression:
MVS RETAIN
Key Key Symptom Data Explanation
MOD/ RIDS/ BPXINPVT LOAD MODULE NAME
CSECT/ RIDS/ BPXVOTHD ASSEMBLY MODULE CSECT NAME
PIDS/ PIDS/ 5752SCPX1 PRODUCT/COMPONENT IDENTIFIER
AB/S AB/S 00C4 ABEND CODE-SYSTEM
REXN/ RIDS/ BPXMIPCE RECOVERY ROUTINE CSECT NAME
FI/ VALU/H 7542A0704742A0708BB917 FAILING INSTRUCTION AREA
REGS/ REGS/ C07DC REG/PSW DIFFERENCE
HRC1/ PRCS/ 00000004 ABEND REASON CODE
SUB1/ VALU/C OPENMVS COMPONENT SUBFUNCTION

This information is called a Symptom String. It provides a very short summary of the problem. You can search the internet with this information, to see if the problem has been found before.

For me, this symptom string, there has been 66 occurrences of it in my system (see the first display).

DAE can be configured to say if you get the same symptom string, do not take a dump, as is is most probably the same problem. This means your disks are not fully of identical system dumps.

Similar dumps

In

Command ===>                                                  Scroll ===> PAGE
Enter an Action Code next to an entry.
Enter / next to an entry to choose from a list of Action Codes.

Dataset: 'SYS1.DAE'
Dumps since last DAE Display: 0 Total Dumps suppressed: 214
Events since last DAE Display: 0 Suppression rate: 81%

A Last Last Total Date of Symptom String information:
C Date System Events Dump Abend Reason Module CSECT
_ 05/01/26 S0W1 7 04/28/26 S0EC6 055B0718 BPXINPVT BPXFSMNT
_ 05/01/26 S0W1 66 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD
_ 05/01/26 S0W1 10 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD
_ 05/01/26 S0W1 53 04/28/26 S00C4 00000004 BPXINPVT BPXVOTHD

There is a record for S0EC6 055B0718, and several records for S00C4 00000004. The detailed symptom string for these S00C4 abends are similar but different. The abend occurred at a different place, so the symptom string is slightly different.

Configuring DAE

You configure DAE through parmlib members ADYSETXX

On my system member ADYSET01 stops DAE, so I get a dump for every abend.

In member ADYSET00 I have

 SVCDUMP(MATCH,SUPPRESSALL,UPDATE,NOTIFY(3,30)),   

Where

  • MATCH: Specifies that DAE is to compare the symptoms from the current memory dump to those that have already been recorded in the DAE data set. (Coding MATCH does not indicate that DAE suppresses duplicate memory dumps or update the DAE data set.)
  • SUPPRESSALL: Specifies that duplicate memory dumps are to be suppressed when all criteria for matching and suppressing memory dumps are met except for the VRADAE key.
  • Update: Specifies that the DAE data set is to be updated with the results of matching.
  • Notify: This sends an internal signal to system applications listening, that a dump has occurred. The default is three memory dumps in 30 minutes for a particular symptom string.

How am I meant to know what’s happened?

As well as DAE processing, a record is written to EREP/LOGREC. EREP is a repository for hardware errors, and software errors.

For example, formatting the dataset, gave me

TYPE:  SOFTWARE RECORD      REPORT:  SOFTWARE EDIT REPORT           DAY.YEAR      
(PROGRAM INTERRUPT) REPORT DATE: 121.26
...
SEARCH ARGUMENT ABSTRACT
PIDS/5752SCPX1 RIDS/BPXINPVT#L RIDS/BPXVOTHD AB/S00C4 PRCS/00000004 REGS/C082C
RIDS/BPXMIPCE#R

SYMPTOM DESCRIPTION
------- -----------
PIDS/5752SCPX1 PROGRAM ID: 5752SCPX1
RIDS/BPXINPVT#L LOAD MODULE NAME: BPXINPVT
RIDS/BPXVOTHD CSECT NAME: BPXVOTHD
AB/S00C4 SYSTEM ABEND CODE: 00C4
PRCS/00000004 ABEND REASON CODE: 00000004
REGS/C082C REGISTER/PSW DIFFERENCE FOR R0C:-082C
RIDS/BPXMIPCE#R RECOVERY ROUTINE CSECT NAME: BPXMIPCE
...
TIME OF ERROR INFORMATION
PSW: 07047001 80000000 00000000 20E1F2F4
INSTRUCTION LENGTH: 04 INTERRUPT CODE: 0004
FAILING INSTRUCTION TEXT: 00175045 00005049 01B0E368
TRANSLATION EXCEPTION ADDRESS: 00000000_01B6A404
BREAKING EVENT ADDRESS: 00000000_20E1F280

AR/GR 0-1 FFF00001/00000000_00000097 00000000/00000051_01B6AC90
AR/GR 2-3 00000001/00000000_0239A000 00000000/00000051_02FCFA00
...

HOME ASID: 0010 PRIMARY ASID: 0010 SECONDARY ASID: 0056
PKM: 8040 AX: 0001 EAX: 0000

RECOVERY ROUTINE ACTION
THE RECOVERY ROUTINE RETRIED TO ADDRESS 20E1F7DA.
THE REQUESTED SVC DUMP WAS NOT TAKEN. THE DUMP WAS SUPPRESSED BY DAE.
NO LOCKS WERE REQUESTED TO BE FREED.
THE SDWA WAS REQUESTED TO BE FREED BEFORE RETRY.

The information is a superset of the information in DAE. You get the registers at the point of failure, and what recovery action was taken. (Programs can ignore some abends, or pass the decision to a program higher up the stack).

In this case I can see there was no dump taken because of DAE.

The information in the logrec record was enough for me to debug the program. I did not need an SDUMP.

Getting the z/OS standard image to work.

I had a lot of challenges getting the zD&T replacement (the standard image) to work. I do not know if this is the same image that every one else gets, or if mine was “a special” one. Some things did not work, somethings were not best practice.

I also wanted to use the data from my previous systems, my PDS, my Unix file systems, my keyrings etc. I also wanted to future proof migrating to the next code drop from IBM.

Summary of actions

Below is a list of the things I did to get the system up and working as I expected it to look.

I’ve grouped them by area. In the main body of the document, they are in the approximate order of execution

Problems

Customising

Before you start using it

Missing download and setting the IBMUSER password

The files on the IBM Passport Advantage site are incomplete. The documentation tells you do download the ZDTDPY volume, edit it to set the password, and boot the system. This download was not on the download site.

I had an old z/OS system I could use and so could reset the password. See Help, I cannot logon to my z/OS system

It is not difficult to write a C program to prompt for a password on the z/OS console – and have the value not displayed. See How do I enter a password on the z/OS console for my program?

I needed to configure my laptop

The IP address of z/OS is 172.26.1.2. I configured my startup script to have

#  define route from my laptop to my server
sudo ip route add 10.1.1.0/24 via 10.1.0.3

# define route from my laptop to z/OS on the server
sudo ip route add 172.26.1.0/24 via 10.1.0.3

# connect to the server
ssh -X colin@10.1.0.3

The sample devmap is wrong, it does not provide 3270’s

The system would not IPL with a console, the output came out printer like.

I edited the devmap file:

  • add the 3270port 3270. Without this you do not get any 3270’s defined, and you get a line printer like console.
  • Change the processors to match what your server has. My dongle has support for 3 CPUs – but I can define 3-1 ZIIPs
  • Check the memory to match you configuration
  • I removed the IPL and put it in my shell script.

The top of my devmap is

[system]
processors 5 cp cp cp ziip ziip # number of processors
memory 10G
system_name VS01

3270port 3270 # port number for TN3270 connections

First IPL

The IPL parameters are configured to start up lots of subsystems I didn’t want to use I used

ipl DE27 parm DE28AU

Logon with userid IBMUSER and the password you configured.

Configure parmlib

The sys0.iplparm points to loadxx members. You need a load member with USER.PARMLIB in it (such as LOADAU)

  • Copy LOADAU to LOADCP
  • In LOADCP change SYSPARM to SYSPARM AU,CP
  • In user.parmlib create IEASYSCP (CP matching the CP above)
  • add your parameters, such as OMVS=(CP) for IP(V6) support

You can IPL with this

ipl DE27 parm DE28CP

If this fails to IPL, to back to parm DE28AU

Add userids

The userid IBMUSER is all powerful. I prefer using a less powerful userid.

I wanted to save my userids from the previous system. I wrote code makeRACF. With this you run a program on your old system, and specify a userid or dataset prefix. The program then generates the RACF commands to recreate the userid or resource.

Because the TSO configuration is different from the previous zD&T systems I had to configure the definitions

TSO (ACCTNUM ('ACCT001') - 
COMMAND ('ex ''colin.zlogon.clist''') -
PROC (PROC001 ) -
SIZE (2096128) -
MAXSIZE (2096128) -
USERDATA (0000) -
UNIT (3390))

I created some definitions to my userid COLIN access to resources on the new system

ALTUSER COLIN SPECIAL AUDITOR OPERATIONS                             
ALTUSER COLIN DFLTGRP(SYS1)
ALTUSER COLIN tso( ACCTNUM('ACCT001') -
CoMMAND('ISPF') PROC(PROC001 ) -
SIZE(2096128) MAXSIZE(2096128) USERDATA(0000) UNIT(SYSDA))
permit ACCT001 class(ACCTNUM) id(COLIN ) access(READ )
permit ACCT# class(ACCTNUM) id(COLIN ) access(READ )
permit JCL class(TSOAUTH)id(COLIN) access(REAd)
permit CONSOLE class(TSOAUTH)id(COLIN) access(REAd)
permit PROC001 class(TSOPROC)id(COLIN) access(REAd)
permit ISPFPROC class(TSOPROC)id(COLIN) access(REAd)
setropts raclist(TSOAUTH) refresh
setropts raclist(ACCTNUM) refresh
setropts raclist(TSOPROC) refresh
PERMIT *.** CLASS(JESSPOOL) ID(COLIN) ACCESS(ALTER)
SETR RACLIST(JESSPOOL) REFRESH

I created JCL for these definitions, so for system refreshes, I just rerun the jobs.

I did not use the UNIX facility to allocate a (random) UID. I specified the UID I had before.

On the ZFS with my files, I needed the uid of the file owner to be the same as my users uid. If z/OS allocated me a uid, I would not be able to access my files. I would then have to go through and change the owner of them.

I recreated userids COLIN and START1.

You need to see what groups the userids belonged to, and you may need to recreate the groups, or just us what are already defined.

Import user catalogs

I have user catalogs for all of my data sets. These catalogs need to be imported into the master catalog

//IBMUSERT JOB 1,MSGCLASS=H                                           
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
IMPORT CONNECT -
OBJECTS -
(('A4USR1.ICFCAT' VOLUME(A4USR1) DEVICETYPE(3390) -
))
/*

How do you find out what user catalogs you have?

On the old system.

//IBMUSERT JOB 1,MSGCLASS=H                               
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//DD1 DD DISP=SHR,VOL=SER=D5SYS1,UNIT=3390
//SYSIN DD *
LISTCAT CATALOG(CATALOG.Z31B.MASTER) USERCATALOG ALL FILE(DD1)
/*

or

//IBMUSERT JOB 1,MSGCLASS=H                               
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LISTCAT USERCATALOG ALL
/*

The ALL Parameter also lists the Aliases using the user catalog.

Define aliases to map high level qualifier to a user catalog.

Once you have imported the catalogs you can define the data set aliases

//IBMUSERT JOB 1,MSGCLASS=H                                     
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE ALIAS (NAME(BACKUP) RELATE('A4USR1.ICFCAT') )
DEFINE ALIAS (NAME(COLIN ) RELATE('A4USR1.ICFCAT') )
/*

What aliases do I have?

//IBMUSERT JOB 1,MSGCLASS=H                                         
//S1 EXEC PGM=IDCAMS,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LISTCAT ALIAS
/*

Define user directories

For each file system, and each user, they need a directory defined. The standard image uses automount to create a ZFS for each entry in /u. If my RACF profile says my home directory is /u/colin, it will create a ZFS for this. If /u/colin already existed, it will not be used.

I created entries in /usr. If you try to make a directory, it may fail because /usr is mounted read only.

From userid IBMUSER to into Unix ( TSO OMVS)

  • mkdir /usr/colin
  • If this fails use
    • chmount -w /usr … do your work … chmount -r /usr
  • mkdir /usr/tmp
  • mkdir /usr/tmp/zowet
  • mkdir /usr/tmp/zowet/colin Userid COLIN has HOME= /usr/tmp/zowet/colin
  • You may need to change the ownership of the directory
    • chown -R colin:sys1 mkdir /usr/tmp/zowet/colin
  • If you need to…
    • chmount -r /usr

Mount the ZFS

Once I had imported the catalogs, and defined the aliases I could try to mount my ZFS

I created USER.PARMLIB(BPXPRMUS) and added

MOUNT FILESYSTEM('COLIN.ZFS.ZOWE.TEMP') TYPE(ZFS) 
MOUNTPOINT('/usr/tmp/zowet') MODE(RDWR)
PARM('AGGRGROW') AUTOMOVE

MOUNT FILESYSTEM('COLIN.ZOPEN.ZFS') TYPE(ZFS)
MOUNTPOINT('/usr/zopen') MODE(RDWR)
PARM('AGGRGROW') AUTOMOVE

The activate using the operator command

T OMVS=(US)

I was then able to logon to my newly created userid COLIN, and go into z/OS Unix.

Configure VTAM to give you greater than 80 *24 screen size

VTAM configuration

  • Edit SYS1.VTAMLST(EXLOCAL). You have to edit this because NET does not have USER.VTAMLST configured.
  • Create EXLOCALO from it (for backup)
  • change DLOGMOD to have value D4B32XX3,
  • create an entry for CUADDR=061,

Clean up

  • Edit SYS1.VTAMLST(ATCCON00). Remove IVPLCLI,IVPLCLT because the devices they reference do not exist.

If your changes to EXLOCAL do not work, use

V NET,ACT,ID=EXLOCALO

TCPIP

The provided TCPIP procedure creates certificates etc and does other work. It creates a new CA certificate, I need to export it, and sent it to my machines which connect in, and to all my keyrings.

I want to use the set up I have, so my TCPIP V6 Procedure is

USER.PROCLIB(TCPIP)

//TCPIP    PROC PARMS='CTRACE(CTIEZB00)' 
//TCPIP EXEC PGM=EZBTCPIP,PARM='&PARMS',REGION=0M,TIME=1440
//SYSPRINT DD SYSOUT=*
//SYSERR DD SYSOUT=*
//SYSERROR DD SYSOUT=*
//ERRORFIL DD SYSOUT=*
//SYSDEBUG DD SYSOUT=*
//PROFILE DD DISP=SHR,DSN=USER.TCPPARMS(PROFILE)
//SYSTCPD DD DISP=SHR,DSN=TCPIP.TCPPARMS(TCPDATA)

Support TCPIP V6

The TCPIP as shipped does not support IP V6. You need to create a BPXPRMxx member and activate it

You need a BPXPRMv6 member in user.parmlib

FILESYSTYPE TYPE(CINET) 
ENTRYPOINT(BPXTCINT)
SUBFILESYSTYPE NAME(TCPIP)
TYPE(CINET)
ENTRYPOINT(EZBPFINI)
DEFAULT
NETWORK DOMAINNAME(AF_INET6)
DOMAINNUMBER(19)
MAXSOCKETS(50000)
TYPE(CINET)

In USER.PARMLIB(IEASYSXX) use BPX=(V6).

In USER.TCPPARMS(PROFILE) I have

...
DATASETPREFIX TCPIP

TCPCONFIG TTLS
INCLUDE USER.TCPPARMS(IPV6)

...

and the IPV6 member has

IPCONFIG6 MULTIPATH

TCPIP Resolver

See Why is the wrong TCPIP Resolver proc being used?

Create USER.PROCLIB(RESOLVER)

//* TCPIP RESOLVER - COLINS 
//*
//RESOLVER PROC PARMS=CTRACE(CTIRES00)
//*
//EZBREINI EXEC PGM=EZBREINI,REGION=0M,TIME=1440,
// PARM=('&PARMS',
// 'ENVAR("RESOLVER_TRACE=/var/log/resolver"/')
//SETUP DD DISP=SHR,DSN=COLIN.TCPPARMS(GBLRESOL),FREE=CLOSE
//SYSTCPT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//*

Create COLIN.TCPPARMS(GBLRESOL) with

Copy

  DEFAULTTCPIPDATA('COLIN.TCPPARMS(GBLTDATA)') 
GLOBALTCPIPDATA('COLIN.TCPPARMS(RESOLVE)')
# GLOBALTCPIPDATA(/etc/resolv.conf)
;
# -----------------------------------------------------------------
# Default zPDT Linux Base to z/OS Tunnel (Stand-Alone)
# -----------------------------------------------------------------
;
# GLOBALIPNODES(/etc/hosts)
GLOBALIPNODES('COLIN.TCPPARMS(HOSTS)')
....

Change the operator console PFKeys (to add the shutdown command)

See How do I change the PFKeys on the console?

Copy SYS1.PARMLIB(PFKTAB00) to USER.PARMLIB

  • Change PF12 to be PFK(12) CMD(“%NETV SHUTSYS”) CON(Y)
  • Change any other keys you fancy.

Fix SMF

Message HSF0066W

I was getting

HSF0066W Required exit IEFU86 for SMF subsystem SYS not enabled. Some data may be missing in SDSF event log.

Copy SYS1.PARMLIB(SMFPRM00) to USER.PARMLIB(SMFPRM00) add IEFU86

SYS(EXITS(IEFACTRT,IEFUJI,IEFU83,IEFU84,IEFU85,IEFUJV,IEFU86)), 
SUBSYS(STC,EXITS(IEFU83,IEFU84,IEFU85,IEFU29,IEFU86),
INTERVAL(SMF,SYNC))

Message IEE391A

IEE391A SMF ENTER DUMP FOR DATA SET ON VOLSER OPEVS1 DSN=SYS1.VS01.MAN1

Create in USER.PROCLIUB(SMFCLEAR)

//SMFCLEAR PROC MAN='SYS1.VS01.MAN1' 
//*
//* CLEAR SMF DATASET -
//* SMF MAN DATASET DUMPED REQUIRED MESSAGE RECIEVED
//* OR ISSUE SWITCH, I SMF, THEN THAT FILE WILL BE CLEARED
//*
//*
//DUMP1 EXEC PGM=IFASMFDP,REGION=1M
//INDD1 DD DSN=&MAN,DISP=SHR
//DUMPOUT DD DUMMY
//SYSPRINT DD SYSOUT=D
//SYSIN DD *
INDD(INDD1,OPTIONS(CLEAR))

Then you can use either of

S SMFCLEAR
S SMFCLEAR,MAN='SYS1.VS01.MAN2'

SETSMF command not enabled

I could not issue

setsmf recording(logstream)
setsmf recording(dataset)

setsmf recording(logstream)

commands, to be able to change the SMF LOGSTREAM.

I created USER.PARMLIB(SMFPRM00) from SYS1.PARMLIB and added AUTHSETSMF – it defaults to NONE.

...
DEFAULTLSNAME(IFASMF.VS01.DATA)
AUTHSETSMF
NOPROMPT,
REC(PERM),
,,,

Configure SYSLOGD

Define the started task

//IBMUSERT JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
RDEFINE STARTED SYSLOGD.* STDATA(USER(IBMUSER))
SETR RACLIST(STARTED) REFRESH

SYSLOGD is used by many Unix processes for recording errors and other information.

Create USER.PROCLIB(SYSLOGD)

//SYSLOGD PROC 
//* Licensed Materials - Property of IBM *
//* "Restricted Materials of IBM" *
//* 5650-ZOS *
//* Copyright IBM Corp. 1992, 2013 *
//* Read parms from /etc/syslog.conf
//CONFHFS EXEC PGM=SYSLOGD,REGION=4096K,TIME=NOLIMIT,
// PARM='ENVAR(''CEE_ENVFILE_S=DD:STDENV'')/-c -i '
//STDENV DD DISP=SHR,DSN=USER.PROCLIB(SYSLOGDD)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSERR DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//CEEDUMP DD SYSOUT=*

Create USER.PROCLIB(SYSLOGDD)

I have

*.INETD*.*.*       /var/log/inetd.log 
auth.* /var/log/auth.log
mail.* /var/log//mail -F 640 -D 770
local1.err /var/log/local1.log
*.err /var/log/errors.log
# *.CPAGENT.*.* /var/log/CPAGENT.log
*.CPATTLS.*.* /var/log/CPATTLS
*.TTLS*.*.* /var/log/TTLS.log
*.Pagent.*.* /var/log/Pagent.log
*.TCPIP.*.debug /var/log/TCPIPdebug.log
*.TCPIP.*.warning /var/log/TCPIP.log
*.TCPIP.*.err /var/log/TCPIPerr.log
*.TCPIP.*.info /var/log/TCPIPinfo.log
*.SYSLOGD*.*.* /var/log/syslogd.log
*.TN3270*.*.* /var/log/tn3270.log
*.SSHD*.*.* /var/log/SSHD.log
*.FTPD*.*.* /var/log/FTPD.log
daemon.debug /var/log/SSHDdebug.log
*.TCPIP.*.none;
*.err /var/log/errors
*.CPAGENT.*.* /var/log/CPAGENT.log
*.TRMD1.*.info /var/log/TRMD1I.log
*.DMD.*.* /var/log/DMD.log

You can use this as a basis and change it as needed.

Configure SSHD

The SSHD procedure in SYS1.PROCLIB, invokes a shell script which then spawns the SSHD code with a name like SSHD.

I use

//SSHD    PROC 
//SSHD EXEC PGM=BPXBATCH,REGION=0M,TIME=NOLIMIT,
// PARM='PGM /usr/sbin/sshd -f /etc/ssh/sshd_config '
//* PARM='PGM /bin/sh -c /etc/ssh/sshd.sh'
//* STDIN AND STDOUT ARE BOTH DEFAULTED TO /dev/null
//STDERR DD PATH='/tmp/sshd.stderr',PATHOPTS=(OWRONLY,OCREAT,OAPPEND),
// PATHMODE=(SIRWXU)
//STDOUT DD PATH='/tmp/sshd.stdout',PATHOPTS=(OWRONLY,OCREAT,OAPPEND),
// PATHMODE=(SIRWXU)

Note that SSHD uses the /etc/ssh/sshd_config directory. You should back this up regulary. When IBM replaces the image this would be replace

After thought, I could always use -f /usr/colin/ssh/

Edit the configuration file and add either userids or groups.

# Allow specific user IDs 
AllowUsers IBMUSER COLIN
AllowGroups SYS1 IZUADMIN

If you add groups and not userids, you need to connect the userid to a group.

RACF changes

You can issue some racf commands on the z/OS console for example

<RVARY

To change this character (<) copy SYS1.PARMLIB(IEFSSN00) to USER.PARMLIB(IEFSSN01) and change the entry for RACF.

Define IXGLOGR as a started task to eliminate security messages

//IBMIXL  JOB 1,MSGCLASS=H 
//STEPNAME EXEC PGM=IKJEFT01
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
* IXGLOGR EXISTS AS A GROUP - IT REALLY SHOULD BE A USERID
ADDUSER IXGLUSER
CONNECT IXGLUSER GROUP(IXGLOGR)
RDEFINE STARTED IXGLOGR.* STDATA(USER(IXGLUSER) +
PRIVILEGED(YES) TRUSTED(YES) GROUP(IXGLOGR ))
SETROPTS RACLIST(STARTED) REFRESH
/*

Add missing groups

To protect datasets such as SYS1.*, the high level qualifier should be defined as a group. I have JCL ADDGROUP

//IBMKEY2 JOB 1,MSGCLASS=H 
//STEPNAME EXEC PGM=IKJEFT01
//SYSTSIN DD *
ADDGROUP SYS1
ADDGROUP ADCD
ADDGROUP AOK
ADDGROUP AOP
ADDGROUP AZD
ADDGROUP BBL
ADDGROUP CATALOG
ADDGROUP CBC
ADDGROUP CEE
ADDGROUP CFZ
ADDGROUP CNJ
ADDGROUP COUPLE
ADDGROUP CSD
ADDGROUP CSF
ADDGROUP CSQARC1
ADDGROUP CSQARC2
ADDGROUP EUVF
ADDGROUP FEU
ADDGROUP FEU1
ADDGROUP FFST
ADDGROUP GDDM
ADDGROUP GIM
ADDGROUP GLD
ADDGROUP GSK
ADDGROUP GTFNO
ADDGROUP GTFRACF
ADDGROUP HAP
ADDGROUP HLA
*DDGROUP IBMUSER
ADDGROUP ICQ
ADDGROUP IOE
ADDGROUP ISF
ADDGROUP ISP
ADDGROUP IXGLOGR
ADDGROUP IZU
ADDGROUP IZUSVR
ADDGROUP NETVIEW
ADDGROUP NFS
ADDGROUP SMPE
ADDGROUP STCJMON
ADDGROUP SYT1
ADDGROUP TCPIP
ADDGROUP USER
ADDGROUP ZFS
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=* g

Protect system data sets

You should define profiles for all datasets. I had a a member ADDSD.

//IBMKEY2 JOB 1,MSGCLASS=H 
//STEPNAME EXEC PGM=IKJEFT01
//SYSTSIN DD *
ADDSD 'SYS1.**' GENERIC UACC(READ) WARNING
PERMIT 'SYS1.**' ACCESS(ALTER) ID(SYS1)
ADDSD 'ADCD.**' GENERIC UACC(READ) WARNING
ADDSD 'AOK.**' GENERIC UACC(READ) WARNING
ADDSD 'AOP.**' GENERIC UACC(READ) WARNING
ADDSD 'AZD.**' GENERIC UACC(READ) WARNING
ADDSD 'BBL.**' GENERIC UACC(READ) WARNING
ADDSD 'CATALOG.**' GENERIC UACC(READ) WARNING
ADDSD 'CBC.**' GENERIC UACC(READ) WARNING
ADDSD 'CEE.**' GENERIC UACC(READ) WARNING
ADDSD 'CFZ.**' GENERIC UACC(READ) WARNING
ADDSD 'CNJ.**' GENERIC UACC(READ) WARNING
ADDSD 'COUPLE.**' GENERIC UACC(READ) WARNING
ADDSD 'CSD.**' GENERIC UACC(READ) WARNING
ADDSD 'CSF.**' GENERIC UACC(READ) WARNING
ADDSD 'CSQARC1.**' GENERIC UACC(READ) WARNING
ADDSD 'CSQARC2.**' GENERIC UACC(READ) WARNING
ADDSD 'EUVF.**' GENERIC UACC(READ) WARNING
ADDSD 'FEU.**' GENERIC UACC(READ) WARNING
ADDSD 'FEU1.**' GENERIC UACC(READ) WARNING
ADDSD 'FFST.**' GENERIC UACC(READ) WARNING
ADDSD 'GDDM.**' GENERIC UACC(READ) WARNING
ADDSD 'GIM.**' GENERIC UACC(READ) WARNING
ADDSD 'GLD.**' GENERIC UACC(READ) WARNING
ADDSD 'GSK.**' GENERIC UACC(READ) WARNING
ADDSD 'GTFNO.**' GENERIC UACC(READ) WARNING
ADDSD 'GTFRACF.**' GENERIC UACC(READ) WARNING
ADDSD 'HAP.**' GENERIC UACC(READ) WARNING
ADDSD 'HLA.**' GENERIC UACC(READ) WARNING
ADDSD 'IBMUSER.**' GENERIC UACC(READ) WARNING
ADDSD 'ICQ.**' GENERIC UACC(READ) WARNING
ADDSD 'IOE.**' GENERIC UACC(READ) WARNING
ADDSD 'ISF.**' GENERIC UACC(READ) WARNING
ADDSD 'ISP.**' GENERIC UACC(READ) WARNING
ADDSD 'IXGLOGR.**' GENERIC UACC(READ) WARNING
ADDSD 'IZU.**' GENERIC UACC(READ) WARNING
ADDSD 'IZUSVR.**' GENERIC UACC(READ) WARNING
ADDSD 'NETVIEW.**' GENERIC UACC(READ) WARNING
ADDSD 'NFS.**' GENERIC UACC(READ) WARNING
ADDSD 'SMPE.**' GENERIC UACC(READ) WARNING
ADDSD 'STCJMON.**' GENERIC UACC(READ) WARNING
ADDSD 'SYT1.**' GENERIC UACC(READ) WARNING
ADDSD 'TCPIP.**' GENERIC UACC(READ) WARNING
ADDSD 'USER.**' GENERIC UACC(READ) WARNING
ADDSD 'ZFS.**' GENERIC UACC(READ) WARNING
PERMIT 'ADCD.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'AOK.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'AOP.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'AZD.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'BBL.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CATALOG.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CBC.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CEE.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CFZ.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CNJ.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'COUPLE.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CSD.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CSF.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CSQARC1.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'CSQARC2.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'EUVF.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'FEU.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'FEU1.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'FFST.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GDDM.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GIM.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GLD.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GSK.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GTFNO.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'GTFRACF.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'HAP.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'HLA.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'IBMUSER.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'ICQ.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'IOE.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'ISF.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'ISP.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'IXGLOGR.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'IZU.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'IZUSVR.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'NETVIEW.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'NFS.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'SMPE.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'STCJMON.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'SYT1.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'TCPIP.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'USER.**' ACCESS(ALTER) ID(SYS1)
PERMIT 'ZFS.**' ACCESS(ALTER) ID(SYS1)
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*

You should have started tasks defined

Started tasks should be defined, so end users cannot just create a Started Task, and run it as an authorised task.

Define a default

//IBMKEY2 JOB 1,MSGCLASS=H 
//STEPNAME EXEC PGM=IKJEFT01
//SYSTSIN DD *
RDEFINE STARTED *.* OWNER(IBMUSER) +
DATA('GENERIC DEFINITION - COLIN') +
STDATA( USER(=MEMBER)
SETROPTS RACLIST(STARTED) REFRESH

Export certificates and keyrings

I have member EXPRING. this runs the LRING Rexx which creates a data set containing certificates used by the userid. They are stored in &USERID..CERTS.START1

//IBMKEYR JOB 1,MSGCLASS=H 
//STEPNAME EXEC PGM=IKJEFT01,PARM='LRING START1',REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSEXEC DD DISP=SHR,DSN=USER.Z31B.CLIST
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD DUMMY

Import certificates and keyrings

Use the Rexx program to export certificates from your old system, and import them into the new system.

Import

//IBMKEYR JOB 1,MSGCLASS=H
//STEPNAME EXEC PGM=IKJEFT01,PARM='IRING COLIN.CERTS.START1',
// REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSEXEC DD DISP=SHR,DSN=USER.Z31B.CLIST,UNIT=3390,VOL=SER=B3CFG1
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD DUMMY

RRS Error messages

At the start up of RRS I got

ATR132I RRS LOGSTREAM CONNECT HAS FAILED FOR
– OPTIONAL LOGSTREAM ATR.VS01.ARCHIVE.
– RC=00000008, RSN=0000080B

See here

Issue the operator command

SETRRS ARCHIVELOGGING,DISABLE 

ICSF

ICSF

I have my own (I)CSF data sets, containing my keys. My Started Task JCL is the same as the default, but with a different member

//CSF  PROC PRM=CP 
//CSF EXEC PGM=CSFINIT,PARM=&PRM,REGION=0M,TIME=1440,MEMLIMIT=NOLIMIT

You could use the provided JCL, then stop and restart CSF

P CSF
S CSF,PRM=CP

The PARM=CP points to a member CSFPRMCP in USER.PARMLIB. Mine has

CKDSN(COLIN.SCSFCKDS) 
PKDSN(COLIN.SCSFPKDS)
TKDSN(COLIN.SCSFTKDS) qq
DOMAIN(0)
SSM(YES)
KEYARCHMSG(YES)

If this member does not exist you get an abend

IEF764I CSF CSF IEFPARM CSFMIOPD HCR77F0 PARMLIB READ FAILED - MEMBER CSFPRMCP NOT FOUND.        
CSFO0016 ERROR OCCURRED OPENING OPTIONS FILE. MEMBER CSFPRMCP IN DDNAME IEFPARM RC=12 RS=1.
...
DUMP TITLE=COMPON=CSF...ABEND=S018F,REASON=0000001B

Logrec EREP

When LOGREC fills up, you get a retained message on the console.

You can print the contents of the EREP file using

//PRINTLOG JOB (ACCT),'PRINT LOGREC',CLASS=A,MSGCLASS=H           
//STEP EXEC PGM=IFCEREP1,PARM='CARD'
//SERLOG DD DISP=SHR,DSN=VSPROV.VS01.LOGREC
//DIRECTWK DD UNIT=SYSDA,SPACE=(CYL,10,,CONTIG)
//EREPPT DD SYSOUT=A,DCB=BLKSIZE=133
//TOURIST DD SYSOUT=A,DCB=BLKSIZE=133
//ZERLOG DD SYSOUT=A,DCB=BLKSIZE=133
//SYSIN DD *
PRINT=PS
ACC=N
ZERO=Y
ENDPARM
/*

and clear it using a procedure in USER.PROCLIB(CLLOGREC)

//CLLOGREC  PROC 
//STEP1 EXEC PGM=IFCDIP00
//SERERDS DD DISP=OLD,DSN=VSPROV.VS01.LOGREC,
// UNIT=3390,VOL=SER=OPEVS1

and the started task definition in STLOGREC

//IBMUSERT JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
RDEFINE STARTED CLLOGREC.* STDATA(USER(IBMUSER))
RDEFINE STARTED PRLOGREC.* STDATA(USER(IBMUSER))
SETR RACLIST(STARTED) REFRESH

JAVA message and SMFLIM

I was getting messages

JVMJ9VM015W Initialization error for library j9shr29(11): JVMJ9VM009E J9VMDllMain failed
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

See JVMJ9VM015W Initialization error for library j9shr29(11): JVMJ9VM009E J9VMDllMain failed

I created USER.PARMLIB(SMFLIM00) with

REGION JOBNAME(JCACHER) JOBMSG(ISSUE) MAXSHARE(80000) 

SMF logstream data sets are too small

Im getting message allocating data sets like IXGLOGR.IFASMF.VS01.DATA… every couple of minutes.

The log stream is define with

LOGSTREAM NAME(IFASMF.VS01.DATA) STRUCTNAME() LS_DATACLAS()
LS_MGMTCLAS() LS_STORCLAS() HLQ(IXGLOGR) MODEL(NO) LS_SIZE(500)
STG_MGMTCLAS() STG_STORCLAS() STG_DATACLAS() STG_SIZE(500)
LOWOFFLOAD(0) HIGHOFFLOAD(80) STG_DUPLEX(YES) DUPLEXMODE(UNCOND)
RMNAME() DESCRIPTION() RETPD(2) AUTODELETE(YES) OFFLOADRECALL(YES)
DASDONLY(YES) DIAG(NO) LOGGERDUPLEX() EHLQ(NO_EHLQ) GROUP()
MAXBUFSIZE(65532)

Where LS_SIZE(500) is 45 tracks.

I used JCL

//IBMLOG JOB 1,MSGCLASS=H                                   
//LOGDEF EXEC PGM=IXCMIAPU,REGION=4M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DATA TYPE(LOGR) REPORT(YES)
UPDATE LOGSTREAM NAME(IFASMF.VS01.DATA)
LS_SIZE(15000)
/*

I could not run this, because the log stream was in use.

I had to create an USER.PARMLIB(SMFPRMNO) with

ACTIVE, 
DSNAME(SYS1.VS01.MAN1,
SYS1.VS01.MAN2),
RECORDING(DATASET)
DEFAULTLSNAME(IFASMF.VS01.DATA)
...

Then use

T SMF=NO

The run the command to update the logstream

and then switch back

T SMF=00

What’s next?

That’s as far as I’ve got. As I do more I’ll update this document.