I’ve been looking into the SMF Real Time, where an application program can get records directly from SMF, and not have to post-process SMF datasets or log streams. To use the real time support, SMF needs to use log streams.
What is SMF?
SMF is System Management Facility. z/OS and the subsystems can write data to SMF for post processing. Typical records are audit and accounting records from z/OS, RACF or CICS, changes to SMS, and changes to resources. Each product has one or more SMF record-type numbers allocated to it. Within each SMF record type you can have sub-types, for example the z/OS SMF 30 record has a sub-type for job start, another sub-type for job step end, and another sub-type for job end.
Display SMF options
The command
d smf
gave
NAME VOLSER SIZE(BLKS) %FULL STATUS
P-SYS1.S0W1.MAN1 B3SYS1 7200 0 ALTERNATE
S-SYS1.S0W1.MAN3 USER04 72000 1 ACTIVE
showing the dataset are being used, and giving information about the datasets
The command
d smf,o
displays all of the SMF options, and where they came from – for example a parmlib member, or from the SETSMF command.
IEE967I 08.44.41 SMF PARAMETERS 489
MEMBER = SMFPRM00
...
SYNCVAL(00) -- DEFAULT
DUMPABND(RETRY) -- DEFAULT
INMEM(IFASMF.COLIN,TYPE(30,42),RESSIZMAX(0128M)) -- PARMLIB
SUBSYS(STC,NOTYPE(14:19,62:69,99)) -- SYS
...
STATUS(010000) -- PARMLIB
INTVAL(01) -- PARMLIB
MAXDORM(0001) -- PARMLIB
REC(PERM) -- PARMLIB
NOPROMPT -- PARMLIB
DSNAME(SYS1.S0W1.MAN3) -- PARMLIB
DSNAME(SYS1.S0W1.MAN1) -- PARMLIB
ACTIVE -- PARMLIB
The old way of recording SMF data
SMF had set of datasets it would use in turn. Typically these were named like SYS1.MANX, SYS1.MANY, or SYS1.PROD.MAN2 etc.. When the active dataset filled up, SMF would switch to the next empty dataset. You (or automation) then runs a job to either copy the records to another dataset, or post process the records; and then clear the dataset for reuse.
As computers got bigger, more work was done, more records were written and writing records to disk could not keep up.
Logstreams is the way forward.
A log stream is a stream of data which can be written to a Coupling Facility(CF) structure, or to a dataset on disk. Typically writing to a CF is faster than writing to disk.
With MANx datasets, all records were written to one dataset. With logstreams, you can configure SMF have multiple logstreams and you configure which record type(s) go to which log stream. This means you can have CICS records going to the “CICS log stream”, and RACF records going to the “RACF logstream”, and the remainder going to a default log stream.
Having multiple logstreams means data can be written to many log streams concurrently, and so avoids the bottleneck of writing to a MANx dataset.
Setting up security profiles
It took me several attempts to configure the security profiles.
Be able to define and delete logstreams
//IBMUSER1 JOB 1,MSGCLASS=H
//KEYCERTS EXEC PGM=IKJEFT01
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
RDEFINE FACILITY RESOURCE(MVSADMIN.LOGR) UACC(NONE)
permit MVSADMIN.LOGR class(FACILITY) -
access(control) ID(SYS1)
setr raclist(facility) refresh
Define individual logstreams
RDEFINE LOGSTRM IFASMF.** UACC(NONE)
PERMIT IFASMF.** class(LOGSTRM ) -
access(ALTER ) ID(SYS1)
setr raclist(logstrm ) refresh
Giving SMF access to the logstreams
RDEFINE FACILITY IFA.IFASMF.* UACC(READ)
setr raclist(facility) refresh
Setting up logstreams
You need to set up at least one log stream. It is easy to define more and change the SMF configuation.
I used the define logstream command
//IBMLOG JOB 1,MSGCLASS=H
//LOGDEF EXEC PGM=IXCMIAPU,REGION=4M
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DATA TYPE(LOGR) REPORT(YES)
DELETE LOGSTREAM NAME(IFASMF.DEFAULT)
DEFINE LOGSTREAM NAME(IFASMF.DEFAULT)
DESCRIPTION(SMF_LOGSTREAM)
MODEL(NO)
DASDONLY(YES)
STG_SIZE(65532)
LS_SIZE(15000)
HLQ(IXGLOGR)
HIGHOFFLOAD(80)
LOWOFFLOAD(0)
AUTODELETE(YES) /* DELETE OPTION */
OFFLOADRECALL(NO)
MAXBUFSIZE(65532)
DIAG(NO)
RETPD(1) /* DELETE 1 DAYS */
//
I also define a log stream IFASMF.COLIN
With the HLQ(IXGLOGR) definition, behind the logstreams were data sets like
Dataset Volume
IXGLOGR.IFASMF.COLIN.ADCDPL *VSAM*
IXGLOGR.IFASMF.COLIN.ADCDPL.DATA USER05
IXGLOGR.IFASMF.COLIN.A0000000 *VSAM*
IXGLOGR.IFASMF.COLIN.A0000000.DATA USER04
Configure SMF
I created a member SMFPRMLS in a user.parmlib
ACTIVE /* ACTIVE SMF RECORDING */
DSNAME(SYS1.&SYSNAME..MAN1,
SYS1.&SYSNAME..MAN3)
RECORDING(LOGSTREAM)
NOPROMPT /* DO NOT PROMPT OPERATOR */
REC(PERM) /* TYPE 17 PERM RECORDS ONLY */
MAXDORM(0001) /* WRITE IDLE BUFFER AFTER 1 SEC */
INTVAL(01) /* EVEY MINUTE */
STATUS(010000) /* WRITE SMF STATS AFTER 1 HOUR */
JWT(0400) /* 522 AFTER 30 MINUTES */
SID(&SYSNAME(1:4))
LISTDSN /* LIST DATA SET STATUS AT IPL */
DEFAULTLSNAME(IFASMF.DEFAULT)
LSNAME(IFASMF.COLIN,TYPE(30,42))
AUTHSETSMF
SYS(NOTYPE(14:19,62:69,99),EXITS(IEFU83,IEFU84,IEFACTRT,
IEFUSI,IEFUJI,IEFU29),NOINTERVAL,NODETAIL)
SUBSYS(STC,EXITS(IEFU29,IEFU83,IEFU84,IEFUJP,IEFUSO))
INMEM(IFASMF.COLI2,RESSIZMAX(128M),TYPE(30,42))
I activated it using the command
t smf=ls
When this failed, because my log stream definitions were not correct, the SMF collection defaulted to using the specified SYS1.MANx datasets.
The important bits of the SMFPRMxx file are
- RECORDING(LOGSTREAM) – use logstreams rather than datasets
- LSNAME(IFASMF.COLIN,TYPE(30,42)) for record types 30 and 42 write them to this log stream
- DEFAULTLSNAME(IFASMF.DEFAULT) If there is no LSNAME for a record type – then write them to this log stream
You can issue setsmf commands to override the existing definition.
Processing SMF records
For SMF datasets
For the Use JCL like
// SET SMFPDS=SYS1.S0W1.MAN1
// SET SMFSDS=SYS1.S0W1.MAN3
//SMFDUMP EXEC PGM=IFASMFDP
//DUMPINA DD DSN=&SMFPDS,DISP=SHR,AMP=('BUFSP=65536')
//DUMPINB DD DSN=&SMFSDS,DISP=SHR,AMP=('BUFSP=65536')
//DUMPOUT DD DISP=(NEW,CATLG),DSN=&RMF,SPACE=(CYL,(10,10))
//* DCB=(LRECL=32760,RECFM=VBS)
//* DCB=(BLKSIZE=0,LRECL=32760,RECFM=VBS)
//*UMPOUT DD DISP=SHR,DSN=IBMUSER.RMF,SPACE=(CYL,(1,1))
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
INDD(DUMPINA,OPTIONS(DUMP))
INDD(DUMPINB,OPTIONS(DUMP))
OUTDD(DUMPOUT,TYPE(42,80,30))
RELATIVEDATE(BYDAY,0,1)
START(0000)
END(2300)
/*
This processes records within the specified time range in the datasets.
For log streams
Use JCL like the following – using PGM=IFASMFDL
//IBMSMFL JOB 1,MSGCLASS=H
//* DUMP THE SMF DATASETS
// SET SMF=IBMUSER.SMF
//*
//S1 EXEC PGM=IEFBR14
//DUMPOUT DD DISP=(MOD,DELETE),DSN=&SMF,SPACE=(CYL,(1,1))
//*
//SMFDUMP EXEC PGM=IFASMFDL,REGION=0M
//DUMPOUT DD DISP=(NEW,CATLG),DSN=&SMF,SPACE=(CYL,(10,10))
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LSNAME(IFASMF.COLIN,OPTIONS(DUMP))
OUTDD(DUMPOUT,TYPE(30))
RELATIVEDATE(BYDAY,0,1)
START(0000)
END(2300)
/*
//
When you specify a date range, it will read not only the active log stream datasets, but any archive ones it created, and which are available.
Display SMF
With logstream the D SMF command gave
LOGSTREAM NAME BUFFERS STATUS
A-IFASMF.DEFAULT 774 CONNECTED
A-IFASMF.COLIN 584 CONNECTED
A-IFASMF.INMEM 0 IN-MEMORY
ColinPaice 