Tag: SMF Real time

Accessing SMF Real Time data.

The traditional way of processing SMF data (product statistics, and audit information), is to post-process SMF datasets. This might be done hourly, or daily, (or more frequently). This means there is a delay between the records being created, and being available for processing.

With SMF Real Time, you can connect an application program to SMF, and get records from the SMF buffers, as the records are created.

Configuring SMF

SMF needs to be in logstream mode. See Many is so last year – logstreams is the way to go.

You need to configure SMF to generate the records. See the SMFPRMxx parameter in parmlib.

I created an entry dynamically using

setsmf inmem(IFASMF.INMEM,RESSIZMAX(128M),TYPE(30,42))

Note: 128M is the smallest buffer size.

The IBM documentation Defining in-memory resources covers various topics.

Displaying information

The command

D SMF

gave me

IFA714I 11.46.50 SMF STATUS 101                
      LOGSTREAM NAME               BUFFERS        STATUS      
      A-IFASMF.DEFAULT                      0       CONNECTED   
      A-IFASMF.COLIN                        0       CONNECTED   
      A-IFASMF.INMEM                  4826066       IN-MEMORY

The command

 D SMF,M

Gave showed my Real time, in Memory resource in use

d smf,m                                                   
IFA714I 11.48.15 SMF STATUS 109                   
IN MEMORY CONNECTIONS                                                
  Resource: IFASMF.INMEM                                             
  Con#: 0001 Connect Time: 2026.019 10:07:20                         
  ASID: 004B                                                         
  Con#: 0002 Connect Time: 2026.019 11:48:10                         
  ASID: 0049

The Application Programming Interface.

The API is pretty easy to use. I based my C application on the IBM example.

I called my program from Python, so that was an extra challenge.

Query

You can issue the query API request. This returns the name of the INMEM definitions available to you, and the SMF record types in the definition.

Capture the data

You need to issue

  • connect, passing the name of the INMEM definition. It returns a 16 byte token. Once the connect has completed successfully, SMF will capture the data in a buffer.
  • get, passing the token. You can specify a flag saying blocking – so the thread waits until data is available. You do not get records from before the connect.
  • If there is too much data for your application to process – or your application is slow to process the data, SMF will wrap the data, and so lose records. The application will get return code IFAINMMissedData (Meaning: Records were skipped due to buffer re-use—that is, wrapping of the data in the in-memory resource. In this case, the output buffer might not contain a valid record.) You should reissue the get.
  • disconnect, passing the token. The disconnect can be done on a different thread. If so, it notifies any thread in a blocking get request, which gets a return code IFAINMGetForcedOut.

Problems

The problems I originally had were that my SMF was not running in log stream mode.
Once I set this up, I could get data back.

I set up INMEM record for SMF 30 records, and although I submitted some batch jobs, I did not get any SMF 30 records in my program.
If I logged off TSO, I got a record. If I issued tso omvs from ISPF I got records.

I added 

SUBSYS(JES2)

to my SMFPRMLS member, and I got SMF 30 records for batch jobs.

I later changed this to be

SUBSYS(JES2,EXITS(IEFU29,IEFU83,IEFU84,IEFUJP,IEFUSO))

to be consistent wit the SUBSYS(STC… parameter)
I got SMF 30 records when logging on using SSH, from using TSO OMVS, or spawning a thread in OMVS to run in the background, for example ls &

It is curious that I do not have SUBSYS(TSO) defined – but I get entries for TSO usage.

It is OK, but…

The code works and generates records. One problem I have is how to stop my program running.

You could use a non blocking call, loop around getting records until you get no records available, then return, do an external wait, and then reloop. This puts the control in your application, but does use CPU as it loops periodically (every second perhaps) looking for records.

You could use a blocking call where the request waits until a record is available, or another thread issues the disconnect call. This means an extra programming challenge creating a thread for the blocking request to run off, and another thread to handle the disconnect request.

The first case, non blocking case, feels easier to code – but at the cost of higher CPU.