Re-entrant assembler macros in z/OS

In a C program on z/OS you can code assembler macros. See Putting assembler code inside a C program

I wanted to use a WTO/WTOR macro to put a message onto the operator console. This took a few hours to get working, firstly I needed to understand using Re-entrant macros, and secondly see Using Re-entrant assembler macros in C ASM().

Background to using re-entrant macros in assembler

Programming in assembler

You can write a macro like

   WTO  'Colins message'

This generates code

         WTO   'Colin'               

BRAS 1,LABEL1 Set register 1 to the following DC and jump to the label
LENDATA DC AL2(9)
DC B'0000000000000000'
DC C'Colin'
LABEL1 DS 0H
SVC 35

This works.

I can pass a string to the macro – a half word length followed by the data – like LENDATA above. This makes the WTO macro more complex.

You can pass a variable content message to WTO. The code following

  • creates data in the correct format (length followed by the data).
  • The WTO creates the data structure
  • The WTO updates the data structure from the data passed to the macro
* define the variable data, and point register 2 to it
BRAS 2,OVERTEXT
DATA DC AL2(5) Length of string
DC C'Colin' the data to display

* invoke WTO. (2) says the address of the data is in register 2
OVERTEXT DS 0H
WTO TEXT=(2)

* This generates...
CNOP 0,4
BRAS 1,TWOLABA branch around definition
* The WTO data structure
DC AL2(8) TEXT LENGTH
DC B'0000000000010000' MCSFLAGS
TEXTADDR DC AL4(0) MESSAGE TEXT ADDRESS
..
* The instructions to update the data structure from the passed data
WTOLABA DS 0H
LR 14,1 FIRST BYTE OF PARM LIST
SR 15,15 CLEAR REGISTER 15
AH 15,0(1,0) ADD LENGTH OF TEXT + 4
AR 14,15 FIRST BYTE AFTER TEXT
ST 2,4(0,1) STORE TEXT ADDR INTO PLIST
SVC 35 ISSUE SVC 35

Where

  • BRAS 2,OVERTEXT saves in register 2 the address of data, then branches to the label OVERTEXT
  • BRAS 1,WTOLABA saves in register 1 the address of the structure following the instruction, then branches to label WTOLABA
  • ST 2,4(0,1) this saves what register 2 points to – my data; and stores it at the front of the control block data in field TEXTADDR

This fails to execute because the whole program is Re-entrant (RENT) and so the whole program is read only. The program is trying to store register 2 into read only storage.

Solving the RENT problem.

The problem of trying to write into read only storage is solved splitting the above code into two parts and using thread read write storage.

  • define the structure for the constants in read only storage
  • copy the read only structure to thread read write storage
  • use the thread read write storage for the request.

Define the structure for the constants in read only storage

You use an extra parameter on the macro call

WTOS      WTO TEXT=(),MF=L  
ETWOS DS 0H

This generates code

WTOS     WTO   TEXT=(),MF=L                                       
WTOS DS 0F
DC AL2(8) TEXT LENGTH
DC B'0000000000010000' MCSFLAGS
DC AL4(0) MESSAGE TEXT ADDRESS
...
DC AL4(0) WSPARM ADDRESS
ETWOS DS 0H

This has created the data structure. The data is of length EWTOS-WTOS

Copy the read only structure to thread read write storage

In assembler you can use MVC or MVCL to copy from the read only data to thread read/write storage.

Use the thread read write storage for the request.

If the static part of the structure was copied to the block of storage userdata, the code below will issue the WTO request

WTOL   WTO TEXT=((2)),MF=(E,userdata)

This generates the code to update the structure in read only storage

         WTOR  TEXT=((2)),MF=(E,userdata)                                
LA 1,userdata LOAD PARAMETER REG
LR 14,1 FIRST BYTE OF PARM LIST
SR 15,15 CLEAR REG 15
AH 15,0(1,0) ADD LENGTH OF TEXT + 4
AR 14,15 ADDR OF BYTE AFTER TEXT
ST 2,4(1,0) MOVE TEXT INTO PARM LIST
OI 4(14),B'00000000' SET EXTENDED MCS FLAGS
OI 5(14),B'10000000' SET EXTENDED MCS FLAGS2
SVC 35 ISSUE SVC 35

Note:

Some z/OS components use MF=(S,name) to generate the Static structure with the specified name, others use MF=(L).

They all use MF=(E,name).

Moving to the z/OS standard image and onward

For vendors and people like me who used ZD&T or zPDT to run z/OS on an IBM provided emulator on Linux, moving to the new standard image is a challenge.

Below are my thoughts on how to make it easier to use the standard image.

What does migration mean?

The term migration may mean different things to different people.

  • “Production customers” have a z/OS image, and they refresh the products, while keeping userid, user datasets etc. the same. The products (from IBM and vendors) gradually changes over time, typically changing every 3-6 months. This process is well know, and has been used over many decades.
  • With the IBM standard image, IBM makes a new level of z/OS available, and you have to migrate userids, datasets etc into the image. Every 3-6 months there may be a new image available. Moving from one level of standard image to another level of standard image is new and not documented. It looks easy to do it wrong, and make migration hard. It may take time to migrate to the first standard image, but moving to later images should take no more than half an hour.

This blog post is to suggest ways of making it easy to set up the to use the standard image.

Moving to the first standard image may mean a lot of work, but if you do it the right way moving on should be easy.

Setting the direction

My recommendations are (I would welcome discussion on these topics).

A couple of years ago I wrote a series of blog post starting with Migrating an ADCD z/OS release to the next release. A lot of the information is still relevant. Below I’ve tried to refine it for the migration to the standard image.


Restrict what you put into the master catalog.

You can restrict what user put into the master catalog. For example, enforce every data set High level qualifier has a RACF profile, and only allow user catalog entries to be added to the catalog by general users.

See

Ensure you use a user catalog

If your datasets are in a user catalog, then to go to the next standard image, you just import the user catalog. If you’ve cataloged dataset in the master catalog, then these are not immediately transferable to a new system.

Use USER. datasets, not SYS1. datasets

You can configure z/OS so it uses parmlib and proclib datasets you specify. On the ZD&T there are USER.Z31B PROCLIB, PARMLIB, CLIST datasets etc. You can copy/use these on each new standard image.

If you have changed ADCD.* or SYS1.* datasets, you can use ISPF 3.4, then sort on the “changed” column to see members changed since you first used the system. Then move them to the USER.* dataset.

Create resources using JCL rather than issuing commands, or using the ISPF panels

Use JCL to issue commands in batch TSO, rather than issue the commands manually. For example with the standard image you may get one userid (IBMUSER), and you want to create more userids. Have a JCL member with the commands to create the additional userid commands.

Once created, you just submit the JCL for the follow-on standard image.

Have an ordering to the members in your migration dataset.

If you have to define a group before you create a userid which uses this group, then have members R1GROUP, R2USER1, or have multiple PDSEs, eg COLIN.DO1GROUP.JCL, COLIN.DO2USERS.JCL. where the members within a data set can be issued in any order.

OMVS file systems

I have multiple ZFS (file systems) which I mount on the z/OS image. If these are cataloged in the user catalog, they can be mounted on the new system and used.

You need to think about where to mount them. If the new image has been configured to use automount, this can cause problems. Automount is an OMVS facility which can create a ZFS and mount it for a user. You can allocate a ZFS on a per userid basis, so if one userid use lots of disk space, it does not affect other users. They just run out of space.

When automount is active on the /u directory, if I try to mount my file system on /u, for example /u/colinzfs, the mount will fail because /u/colinzfs is already allocated.

You need to use another directory perhaps /my to mount your ZFS on.

If user’s home directory is something like /my/colin, SSH certificates will be available on the new system, without having to set them up again.

Changing files in system file systems

Try to avoid changing the system file systems, for example /etc/ /var, /usr/

If you have changed the system file systems, see here to see which files have changes since you started using the current image, and move them to your own file system.

Userids and OMVS

You can use the RACF autoid facility which allocates a UID for the userid. This means you do not need to mange the list of UIDs. This makes life easier for an administrator, but harder for a standard image user.

If you use the autoid on the current system you may get an UID such as 990021. On the newer image, your userid may be given a difference UID – depending on the order and number of requests made. Having a different UID can cause problems when using your ZFS. For example the files for my userid COLIN have owner with UID 990021. On the newer system, I may get UID 990033. As this UID is different to 990021, I will not have access to my files.

You should consider explicitly allocating a UID which stays with the user.

If you want to extract RACF profiles from the current system. See the extract program. This will create the RACF command needed to define the profiles. You can specify userids, datasets or classes.

Certificates

You can use RACF commands to display and extract keyring information, and certificates (public and private parts). These can be imported on the newer system. This means your client applications will continue to work.

ICSF

You can configure which data sets ICSF uses in the (CSFPRMXX) member in parmlib. Mine are prefixed with COLIN…

Started tasks

Many started tasks associated with OMVS, (or TCPIP) store configuration in /etc/. For example the file /etc/hosts and the directory /etc/ssh.

You may be able to change the started tasks to use files in your ZFS.

For example

//SSHD    PROC 
//SSHD EXEC PGM=BPXBATCH,REGION=0M,TIME=NOLIMIT,
// PARM='PGM /usr/sbin/sshd -f /my/etc/ssh/sshd_config '

What packages are installed?

You can issue

zopen query -i > installed 

to see what is installed

This gave me

Package   Version  File                               Releaseline             
bash 5.3.9 bash-5.3.20260204_143226.zos STABLE
curl 8.18.0 curl-8.18.0.20260205_151329.zos STABLE
git 2.53.0 git-v2.53.0.20260212_134939.zos STABLE
gpg 2.5.17 gnupg-2.5.17.20260130_021013.zos STABLE
jq 1.8.1 jq-jq-1.8.1.20250919_125054.zos STABLE
less 692 less-v692-rel.20260209_153821.zos STABLE
libpsl 1.0.0 libpsl-master.20260102_060204.zos STABLE
libssh2 1.11.1 libssh2-1.11.1.20260102_060940.zos STABLE
meta 0.8.4 meta-main.20260116_055504.zos STABLE
ncurses 6.6 ncurses-6.6.20260129_223023.zos STABLE
openssl 3.6.0 openssl-3.6.0.20260101_102819. STABLE

and

pip list

which gave

Package      Version
------------ -----------
ansible-core 2.20.3
cffi 1.14.6
cryptography 3.3.2
Jinja2 3.1.6
MarkupSafe 3.0.3
packaging 26.0
pip 26.0.1
pycparser 2.20
pysear 0.4.0
PyYAML 6.0.3
pyzfile 1.0.0.post2
resolvelib 1.2.1
six 1.16.0
tzdata 2025.3
zoautil-py 1.2.5.10

How do I allocate a Unix id on z/OS?

To use Unix services (sometimes known as USS) on z/OS, a userid needs a UserID (UID). This, as on Unix,is an integer. A user can be pre-allocated a permanent UID, or be allocated a UID when when needed. See Automatically assigning unique IDs through UNIX services.

Unique or not Unique?

It is good practice for each userid to have a unique UID. If users share the same UID,

  • The users share ownership and access to the same files.
  • If you ask for the userid associated with an id – you may get the wrong answer!

However some super users need a id of 0.

You can set this as shared with

altuser colin OMVS(UID(0)SHARED)

Instead of allocating uid(0) you can use the profile BPX.SUPERUSER resource in the FACILITY class to get the authority to do most of the tasks that require superuser authority.

  1. You can explicitly specify an id which you allocate (this means you need a list of ids and owners, so you know which ids are free).
  2. You can have z/OS do this for you. See Enabling automatic assignment of unique UNIX identities.

You can use ADDUSER COLIN OMVS(AUTOUID) which allocates an available UID.

Should I used AUTOID?

I run z/OS on a zD&T image. Every 6 months or so there is a new level of z/OS which I can download. I then need to migrate userid, datasets etc to this new system. This is different to a normal customer z/OS where you have an existing system and you migrate a new version of z/OS into it.

I have ZFS file systems for all of my user data.
On the current system my userid COLIN was automatically allocated as 0000990021. Files that I own have this id.

When I get my next system, if I allocate userid COLIN with AUTOUID, it may get a different UID say 990011. Because my userid 990011 is different to the owner of the files 990021, I may not be able to access “my” files.

I could change all of my files to have a new owner (and group), or I could ensure my userid on both systems is the same 990021. Using the same UID was much easier.

How is the range of AUTOIDs defined?

This is done with the RACF FACILITY profile BPX.NEXT.USER. On my system has has

APPLICATION DATA 990041-1000000/990020-1000000

Can I define a model profile?

You can configure OMVS to automatically give a userid a UID (if it does not have one) and define the rest of the OMVS profile using a model OMVS segment. See Steps for automatically assigning unique IDs through UNIX services.

Users need a home directory

Users need a home directory. There are several ways of doing this.

  • Give users an entry HOME(‘/u/mostusers’). Every one shares the same directory – not a good idea, because they would all share the SSH keys etc.
  • You could specify HOME(‘/u/mostusers/&racuid’) and specify the userid as part of the definition. This could be done in the model profile mentioned above. If you use this method you need to create the directory, for example as part of creating the userid.
  • Use automount. See Unix services automount is a blessing and curse. Where you define a template and the hard word is done for you. For example for each userid create a ZFS and use that.

I only use a few userids, so manually allocating the userid and the home directory was easy to do.

Note: If you use automount of a directory, such as /u/, you cannot mount other file systems in /u/; you would have to use a different directory, for example /usr/.

How do I create a load module in a PDS from Unix?

This is another of the little problems which are easy once you know the anwser.

I used the shell program to compile my program.

name=extract 

export _C89_CCMODE=1

p1="-Wc,arch(8),target(zOSV2R3),list,source,ilp32,gonum,asm,float(ieee)"
p7="-Wc,ASM,ASMLIB(//'SYS1.MACLIB') "
p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,XREF,SHOWINC -Wa,LIST(133),RENT"

# compile it
xlc $p1 $p7 $p8 -c $name.c -o $name.o

l1="-Wl,LIST,MAP,XREF,AC=1 "
# create an executable in the file system
/bin/xlc $name.o -o $name -V $l1 1>a
extattr +a $name

# create a load module in a PDS
/bin/xlc $name.o -o "//'COLIN.LOAD(EXTRACT)'" -V $l1 1>a

Create an executable in the file system

The first bind xlc step creates an object with name “extract” in the file system.

Specify the load module

The second bind step specified a load module in a PDS. The load module is stored in COLIN.LOAD. If you copy and paste the line, make sure you have the correct quotes ( double quote, //, single quote, dataset(member),single quote,double quote). Sometimes my pasting lost a quote.

Process assembler code

My program has some assembler code…

 asm( ASM_PREFIX 
" STORAGE RELEASE,...
:"r0", "r1" , "r15" );

It needs the options “-Wc,ASM,ASMLIB(//’SYS1.MACLIB’) ” to compile it, and specify the location of the assembler macros.

Binder parameters

The line parameters in -Wl,LIST,MAP,XREF,AC=1 are passed to the binder.

Message – wrong suffix on the source file

Without the export _C89_CCMODE=1 I got the message

FSUM3008 Specify a file with the correct suffix (.c, .i, .s, .o, .x, .p, .I, or .a), or a corresponding data set name, instead of -o ./extract.

How do I enter a password on the z/OS console for my program?

I wanted to run a job/started task which prompts the operator for a password. Of course being a password, you do not want it written to the job log for every one to see.

In assembler you can write a message on the console, and have z/OS post an ECB when the message is replied to.

         WTOR  'ROUTECD9 ',reply,40,ecb,ROUTCDE=(9) 
wait 1,ECB=ECB
...
ECB DC F'0'
REPLY DS CL40

The documentation for ROUTCDE says

  • 9 System Security. The message gives information about security checking, such as a request for a password.

When this ran, the output on the console was as follows The … is where I typed R 6,abcdefg

@06 ROUTECD9 
...
R 6 SUPPRESSED
IEE600I REPLY TO 06 IS;SUPPRESSED

With ROUTCDE=(1) the output was

@07 ROUTECD1                      
R 7,ABCDEFG
IEE600I REPLY TO 07 IS;ABCDEFG

With no ROUTCDE keyword specified the output was

@08 NOROUTECD                          
R 8 SUPPRESSED
IEE600I REPLY TO 08 IS;SUPPRESSED

The lesson is that you have to specify ROUTCDE=(1) if you want the reply to be displayed. If you omit the ROUTCDE keyword, or specify a value of 9 – the output is supressed.

Can I do this from a C program?

The C run time _console2() function allows you to issue console messages. If you pass and address for modstr, the _console2() function waits until there is an operator stop of modify command issued for the job. If a NULL address is passed in the modstr, then the message is displayed, and control returns immediately. The text of the modify command is visible on the console.

To get suppressed text you would need to issue the WTOR Macro using __ASM(…) in your C program.

Can I share a VSAM file (ZFS) between systems?

I had the situation where I am using ZD&T – which is a z/OS emulator running on Linux, where there 3390 disks are emulated on Linux files. I have an old image, and a new image, and I want to use a ZFS from the new image on the old image to test out a fix.

The high level answer to the original question is “it depends”.

Run in a sysplex

This is how you run in a production environment. You have a SYSPLEX, and have a (master) catalog shared by all systems. I cannot create the environment in zD&T. Setting up a sysplex is a lot of work for a simple requirement.

Copy the Linux file

Because the 3390 volumes are emulated as Linux files, you can copy the Linux file and use that file in the old zPTD image, and avoid the risk of damaging the new copy. The Linux file name is different, but the VOLID is the same. I was told you can use import catalog to get this to work. I haven’t tried it.

The cluster is in a shared user catalog.

If the VSAM cluster is defined in a user catalog, and the user catalog can be used on both systems, then the cluster can be used on both systems (but not at the same time). When the cluster is used, information about the active system is stored in the cluster. When the file system is unmounted, or OMVS is shutdown, this system information is removed. If you do not unmount, or shutdown OMVS cleanly, then when the file system is mounted on the other system, the mount will detect the file system was last used on another system, and wait for a minute or so to make sure the other system is inactive. If the mount command is issued during OMVS startup OMVS will wait for this time. If you have 10 file systems shared, OMVS will wait for each in turn – which can significantly delay OMVS start up.

When the cluster is in the master catalog

Someone suggested

You could mount the volume to your new system and import connect the master catalog of the old system to the new one and define the old alias for the ZFS in the new master pointing to the old master which is now a user catalog to the new system.  If it’s not currently different, you could rename it on the old system to a new HLQ that is different from the existing one and then do the import connect of the master as a usercat and define the new alias pointing to the old ZFS.

This feels too dangerous to me!

Pax the files in the directory

You can use Pax to unload the contents of the directory to a dataset, then load the data from the dataset on the other system.

cd /usr/lpp....
pax -W “seqparms=’space=(cyl,(10,10))'” -wzvf “//’COLIN.PAX.PYMQI2′” -x os390 .

On the other system

mkdir mydir
cd mydir
pax -rf “//’COLIN.PAX.PYMQI2A'” .

Note when using cut and paste make sure you have all of the single quotes and double quotes. I found they sometimes got lost in the pasting.

Using DFDSS

See Migrating an ADCD z/OS release: VSAM files

I can’t even spell Ansible on z/OS

The phrase “I can’t even spell….” is a British phrase which means “I know so little about this that I cannot even pronounce or write the word.”

I wanted to see if I could use Ansible to extract some information from z/OS. There is a lot of documentation available, but it felt like the documentation started at chapter 2 of the instruction book, and missing the first set of instructions.

Below are the instructions to get the most basic ping request working.

On z/OS

Ansible is a python package which you need to install.

pip install ansible-core

This may install several packages

It is better to do this in an SSH terminal session rather than from ISPF -> OMVS. For example it may display a progress bar.

On Linux

Setup

sudo apt install ansible

I made a directory to store my Ansible files in

mkdir ansible
cd ansible

There is some good documentation here.

Edit the inventory.ini

[myhosts]
10.1.1.2

[myhosts:vars]
ansible_python_interpreter=/usr/lpp/IBM/cyp/v3r12/pyz/bin/python

Where

  • [myhosts]… is the IP address of the remote system.
  • [myhosts:vars] ansible_python_interpreter=… is needed for Ansible to work. It it the location of Python on z/OS.

Check the connection

Ansible uses an SSH session to get to the back end. Check this works before you use Ansible.

ssh colin@10.1.1.2

I have set this up for password less logon.

Try the ping

ansible myhosts -u colin -m ping -i inventory.ini

Where

  • -i inventory.ini specifies the configuration file
  • myhosts which sections in the configuration file
  • -u colin logon with this userid
  • -m ping issue this command

When this worked I got

10.1.1.2 | SUCCESS => {
"changed": false,
"ping": "pong"
}

The command took about 10 seconds to run.

You may not need to specify the -u information.

What can go wrong?

I experienced

Invalid userid

ansible myhosts -u colinaa -m ping -i inventory.ini

10.1.1.2 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: colinaa@10.1.1.2: Permission denied (publickey,password).",
"unreachable": true
}

This means you got to the system, but you specified an invalid user, or the userid was unable to connect over SSH.

Python configuration missing

ansible myhosts -u colin -m ping -i inventory.ini

This originally gave me

[WARNING]: No python interpreters found for host 10.1.1.2 (tried ['python3.12', 'python3.11',
'python3.10', 'python3.9', 'python3.8', 'python3.7', 'python3.6', '/usr/bin/python3',
'/usr/libexec/platform-python', 'python2.7', '/usr/bin/python', 'python'])
10.1.1.2 | FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"module_stderr": "Shared connection to 10.1.1.2 closed.\r\n",
"module_stdout": "/usr/bin/python: FSUM7351 not found\r\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 127
}

Edit the inventoy.ini and add the ansible_python_interpreter information.

[myhosts]
10.1.1.2

[myhosts:vars]
ansible_python_interpreter=/usr/lpp/IBM/cyp/v3r12/pyz/bin/python

Logging on to Git (on z/OS)

I’ve gradually been moving away from being 100% ISPF, and moving to OMVS. I use SSH terminals to access the Command Line Interface (CLI) just like I use on Linux, and I do most of my editing with VScode on Linux accessing the files on z/OS over sshfs so they look as if they are in a local Linux directory.

I wanted to use Git on z/OS. It was easy to install and start using, but I had problems logging on to Git.

As I understand it there are several ways of logging on to Git. I’ve used two, HTTPS and SSH.

HTTPS

You can logon to Git with a userid and a Personal Access Token. A PAT is like a sophisticated password. To get a PAT, go to your Git home page, click on your photo, and click settings. On the public profile page which is displayed, at the bottom of the left hand column is<> Developer settings. Click on this link. Click on Personal access tokens.

Click on Tokens (classic) -> Generate new token (classic). You have to verify, so I clicked send code via email. Copy the PAT.

When you create a new PAT you can specify what the token can do, for example

  • full control of the private repository, or just access the public repository, or access the commit state.
  • can control the public keys
  • delete repositories

Click on generate token. A token is displayed such as ghp_7OSehXd6lP1234Gy0KRvqpmABALX8L618ycad. Copy this and save it somewhere securely. If you lose it, it is easy to delete and create another.

If you use Git using https, for example https://github.com/colinpaiceABC/ColinsRepo it will prompt for userid (colinpaiceABC) and password. Password means use a PAT.

You can store the userid and PAT for scripts etc to use to logon.

When you create the PAT you specify the validity period, for example two weeks, so you will need to have a process in place to renew the token.

SSH

You can logon to Git using SSH. Because keys are stored on your local machine, and on the Git server, you do not need to enter userid and password/PAT each time.

Git has excellent documentation on using ssh.

You need an SSH key. Check in directory ~/ssl, for files like id_….pub I have id_ed25519.pub and id_rsa.pub . If you do not have a key, follow the git documentation to create one.

Once I had my key I used the documentation on how to add it to Git.

Check you are using the ….pub file. It looks like

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA...XX/Xk colin@ColinNew

Add it using picture -> settings -> SSH and GPG keys ….

To use this you access Git via

git clone git@github.com/colinpaicemq/MQTools.git

If it doesn’t work as expected.

I got into a mess because I used

git clone https://github.com/colinpaicemq/MQTools.git

to clone the repository. When I tried to update the repository it asked me for userid and password!

You can change whether you use HTTPS or SSH to logon. For example to set SSH

git remote set-url origin git@github.com:colinpaicetest/testrepro.git

See the documentation.

Accessing SMF Real Time data.

The traditional way of processing SMF data (product statistics, and audit information), is to post-process SMF datasets. This might be done hourly, or daily, (or more frequently). This means there is a delay between the records being created, and being available for processing.

With SMF Real Time, you can connect an application program to SMF, and get records from the SMF buffers, as the records are created.

Configuring SMF

SMF needs to be in logstream mode. See Many is so last year – logstreams is the way to go.

You need to configure SMF to generate the records. See the SMFPRMxx parameter in parmlib.

I created an entry dynamically using

setsmf inmem(IFASMF.INMEM,RESSIZMAX(128M),TYPE(30,42))   

Note: 128M is the smallest buffer size.

The IBM documentation Defining in-memory resources covers various topics.

Displaying information

The command

D SMF

gave me

IFA714I 11.46.50 SMF STATUS 101                
LOGSTREAM NAME BUFFERS STATUS
A-IFASMF.DEFAULT 0 CONNECTED
A-IFASMF.COLIN 0 CONNECTED
A-IFASMF.INMEM 4826066 IN-MEMORY

The command

 D SMF,M

Gave showed my Real time, in Memory resource in use

d smf,m                                                   
IFA714I 11.48.15 SMF STATUS 109
IN MEMORY CONNECTIONS
Resource: IFASMF.INMEM
Con#: 0001 Connect Time: 2026.019 10:07:20
ASID: 004B
Con#: 0002 Connect Time: 2026.019 11:48:10
ASID: 0049

The Application Programming Interface.

The API is pretty easy to use. I based my C application on the IBM example.

I called my program from Python, so that was an extra challenge.

Query

You can issue the query API request. This returns the name of the INMEM definitions available to you, and the SMF record types in the definition.

Capture the data

You need to issue

  • connect, passing the name of the INMEM definition. It returns a 16 byte token. Once the connect has completed successfully, SMF will capture the data in a buffer.
  • get, passing the token. You can specify a flag saying blocking – so the thread waits until data is available. You do not get records from before the connect.
  • If there is too much data for your application to process – or your application is slow to process the data, SMF will wrap the data, and so lose records. The application will get return code IFAINMMissedData (Meaning: Records were skipped due to buffer re-use—that is, wrapping of the data in the in-memory resource. In this case, the output buffer might not contain a valid record.) You should reissue the get.
  • disconnect, passing the token. The disconnect can be done on a different thread. If so, it notifies any thread in a blocking get request, which gets a return code IFAINMGetForcedOut.

Problems

The problems I originally had were that my SMF was not running in log stream mode.
Once I set this up, I could get data back.

I set up INMEM record for SMF 30 records, and although I submitted some batch jobs, I did not get any SMF 30 records in my program.
If I logged off TSO, I got a record. If I issued tso omvs from ISPF I got records.

I added

SUBSYS(JES2) 

to my SMFPRMLS member, and I got SMF 30 records for batch jobs.

I later changed this to be

SUBSYS(JES2,EXITS(IEFU29,IEFU83,IEFU84,IEFUJP,IEFUSO))

to be consistent wit the SUBSYS(STC… parameter)
I got SMF 30 records when logging on using SSH, from using TSO OMVS, or spawning a thread in OMVS to run in the background, for example ls &

It is curious that I do not have SUBSYS(TSO) defined – but I get entries for TSO usage.

It is OK, but…

The code works and generates records. One problem I have is how to stop my program running.

You could use a non blocking call, loop around getting records until you get no records available, then return, do an external wait, and then reloop. This puts the control in your application, but does use CPU as it loops periodically (every second perhaps) looking for records.

You could use a blocking call where the request waits until a record is available, or another thread issues the disconnect call. This means an extra programming challenge creating a thread for the blocking request to run off, and another thread to handle the disconnect request.

The first case, non blocking case, feels easier to code – but at the cost of higher CPU.

Getting SSH to work to z/OS

I have two versions of z/OS, old and new(!). I had problems getting ssh to work because of key problems.

The problem

I tried to update my laptop key to the server

ssh-copy-id colin@10.1.1.2

This gave

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: ERROR: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ERROR: @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
ERROR: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ERROR: IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
ERROR: Someone could be eavesdropping on you right now (man-in-the-middle attack)!
ERROR: It is also possible that a host key has just been changed.
ERROR: The fingerprint for the ED25519 key sent by the remote host is
ERROR: SHA256:2mUOVfdSedJVQIzZiGsRkOe9Vkc1bkyuDNp5H+VrZ98.
ERROR: Please contact your system administrator.
ERROR: Add correct host key in /home/colin/.ssh/known_hosts to get rid of this message.
ERROR: Offending ED25519 key in /home/colin/.ssh/known_hosts:1
ERROR: remove with:
ERROR: ssh-keygen -f '/home/colin/.ssh/known_hosts' -R '10.1.1.2'
ERROR: Host key for 10.1.1.2 has changed and you have requested strict checking.
ERROR: Host key verification failed.

Searching the internet I got suggestions saying “delete the old line from the file”. I didn’t want to do this because it meant I would not be able to go back to the old system and work as before.

Solutions

I edited /home/colin/.ssh/known_hosts and commented out line 1, with a # at the front (the :1 above is the first line). I repeated the command and it report the same message for line :2. I commented that out as well.

I got further

colin@ColinNew:~$ ssh-copy-id colin@10.1.1.2
The authenticity of host '10.1.1.2 (10.1.1.2)' can't be established.
ED25519 key fingerprint is SHA256:2mUOVfdSedJVQIzZiGsRkOe9Vkc1bkyuDNp5H+VrZ98.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 2 key(s) remain to be installed -- if you are prompted now it is to install the new keys
colin@10.1.1.2: Permission denied (publickey,hostbased).

I had to start the SYSLOGD on z/OS to capture the output from SSHD.

In the /var/logSSHD (your’s may be different) it said

FOTS2307 User COLIN from 10.1.0.2 not allowed because not listed in AllowUsers 

In my SSHD config file /etc/ssh/sshd_config I had

# Allow specific user IDs 
AllowUsers IBMUSER

I added COLIN to the list and restarted SSHD. (I do not know how to refresh SSHD)

This time the error log had

trying public key file /u/tmp/zowet/colin/.ssh/authorized_keys 
Could not open authorized keys '/u/tmp/zowet/colin/.ssh/authorized_keys': ...

I fixed this, tried to logon, and this time it worked.

On Linux, I edited /home/colin/.ssh/known_hosts and un-commented the lines I had commented out before.
I tried the ssh command again, and it still worked!