Processing lines in ASCII files in ISPF edit macros

I was trying to make it easier to read the trace lines from Liberty. These lines can be hundreds of characters long – and it needs many scrolls right and left to display it.
If I extracted and displayed the line – it displayed garbage because the line was in ASCII, and if you just displayed the line, it displays the ASCII values.

This raised several challenges.

  1. How do do you convert from ASCII to EBCDIC in an ISPF Rexx macro.
  2. How do you nicely display a long line on one screen.

How do do you convert from ASCII to EBCDIC in an ISPF Rexx macro?

I could not find any easy code to copy, so I had to write code to create the ASCII to EBCDIC mapping in Rexx.

/* REXX */ 
/*
exec to display long (Liberty) logs by flowing text
*/
ADDRESS ISPEXEC
'ISREDIT MACRO'
"ISREDIT autosave off "
ascii0 = "000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F"
ebcdic0 = "................................"
ascii1= "202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F40"
ebcdic1 = " !..$.&'()*+,-./0123456789:;<=>?@"
ascii2 ="4142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F60"
ebcdic2= "ABCDEFGHIJKLMNOPQRSTUVWXYZ(\)._."
ascii3 ="6162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F"
ebcdic3= "abcdefghijklmnopqrstuvwxyz{|}~."

ascii = ascii0 || ascii1 || ascii2||ascii3
ebcdic = ebcdic0 || ebcdic1|| ebcdic2||ebcdic3

ascii = x2c(ascii)
/*get the current line number, and extract it */
"ISREDIT (l) = LINENUM .ZCSR "
"ISREDIT ( d ) = LINE " l
/* convert it to ebcdic */
out = translate(d,ebcdic,ascii)

The upper part of the code just creates the translation tables. For example ASCII 0x21i is EBCDIC “!”.

The code

"ISREDIT         (l) = LINENUM .ZCSR " 
"ISREDIT ( d ) = LINE " l
/* convert it to ebcdic from ascii */
out = translate(d,ebcdic,ascii)
say out

reads the current (.ZCSR) line of the file into variable d, converts to EBCDIC and displays it.

How do you nicely display a long line on one screen?

I used

out = translate(d,ebcdic,ascii) 
/* and display it */
out = strip(out)
do i = 1 to length(out) by 72
say substr(out,i,72 )
end

which cuts the message into 72 byte chunks.

I enhanced this to allow me to enter a number of lines on the macro

/* REXX */ 
/*
exec to display long (Liberty) logs by flowing text
*/
ADDRESS ISPEXEC
'ISREDIT MACRO (lines) '
"ISREDIT autosave off "
"ISREDIT (fcol,lcol) = DISPLAY_COLS" /* get width of the screen */
width = lcol - fcol
ascii0 = "000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F"
ebcdic0 = "................................"
ascii1= "202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F40"
ebcdic1 = " !..$.&'()*+,-./0123456789:;<=>?@"
ascii2 ="4142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F60"
ebcdic2= "ABCDEFGHIJKLMNOPQRSTUVWXYZ(\)._."
ascii3 ="6162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F"
ebcdic3= "abcdefghijklmnopqrstuvwxyz{|}~."

ascii = ascii0 || ascii1 || ascii2||ascii3
ebcdic = ebcdic0 || ebcdic1|| ebcdic2||ebcdic3

ascii = x2c(ascii)
/*get the current line number, and extract it */
"ISREDIT (last) = LINENUM .ZLAST"
"ISREDIT (l) = LINENUM .ZCSR "
if lines = "" then
lines = 1
do j = l to (l + lines - 1)
if (j > last) then leave
"ISREDIT ( d ) = LINE " j
if rc <> 0 then leave
/* convert it to ebcdic */
out = translate(d,ebcdic,ascii)
/* and display it */
out = strip(out)
/* display full width, based on screen size */

do i = 1 to length(out) by width
say substr(out,i,width)
end
say " "
end
exit

so if my macro is called zz, I can issue the command zz 3 and it displays 3 lines in the file

The couple of hours it took me to write this have made my life so much easier.

Why did my certificate mapping go wrong?

I had a working mapping for a Linux generated certificate to a z/OS userid. And then it wasn’t working. It took me 2 days before I had enlightenment. Although I had undone all of the changes I had made – well all but one.

I had defined

//IBMRACF  JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
RACDCERT DELMAP(LABEL('colinpaice'))ID(IBMUSER)
RACDCERT MAP ID(IBMUSER) -
WITHLABEL('colinpaice') -
SDNFILTER('CN=colinpaice.O=cpwebuser.C=GB')
SETROPTS RACLIST(DIGTNMAP, DIGTCRIT) REFRESH
racdcert listMAP id(IBMUSER)
/*

Which says it the certificate with Subject: C = GB, O = cpwebuser, CN = colinpaice come in, then it maps to IBMUSER. Yes, the terms are in a different order, and there are “.” instead of “.” but it worked.

I started working with JSON Web Tokens (JWT), and it stopped working. The userid was coming out as IZUSVR – which is the userid of z/OSMF. I struggled with traces, and wrote my own little program to map the certificate to a userid – but still it was IZUSVR.

The enlightenment.

With JWT they are signed by a private key, and the public key is used to check the signature (that is check the checksum of the data is valid). To do this, the keyring needs the certificate in the keyring.
I was lazy and used the same certificate to sign the JWT, as I used to do certificate logon to z/OSMF.

To put the certificate in the keyring you need to import the certificate. I copied the certificate from Linux, using cut and paste and imported it

I used

//IBMRACF2 JOB 1,MSGCLASS=H 
//S1 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
RACDCERT CHECKCERT('COLIN.COLIN.PAICE.PEM')
RACDCERT DELETE (LABEL('COLINPAICE')) ID(IZUSVR)
RACDCERT ADD('COLIN.COLIN.PAICE.PEM') -
ID(IZUSVR) WITHLABEL('COLINPAICE') TRUST


RACDCERT ID(IZUSVR) CONNECT(RING(CCPKeyring.IZUDFLT) -
USAGE(CERTAUTH) -
LABEL('COLINPAICE') -
id(IZUSVR))

SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh
/*

This imports the certificate and associates it with the specified userid, ID(IZUSVR).
Now, when the certificate arrives as part of the certificate logon to z/OSMF, it checks to see if it is in the RACF data base – yes it is – under userid IZUSVR. It does not use the RACDCERT MAP option.

I reran this job with userid ADCDB – and the JWT had ADCDB in the definition.

To make it more complex, the Liberty Web Server within z/OSMF caches some information, and this complicated the diagnosis. In the evening it worked – next morning after IPL – it didn’t!

Lesson learned

Use one certificate for certificate logon, and another certificate for JWT.

What are RACF realms and how are they used.

A year or so ago I had come across the term realms in relation to security, but could not find what they are or how they are used.

I’ve been working with JSON Web Tokens, (to identify a Linux user to the MQWEB server on z/OS) and have found out what realms are.

The short answer is a realm is a zone of definitions it could be a machine, or a company. OK – this is not very clear.

The high level view

Take the scenario, I have a z/OS and a Linux environments. On both systems I have a userid COLIN.
When I create a JWT

  • on Linux I give it parameters subject:COLIN, issuer:LINUX
  • on z/OS I give it parameters subject:COLIN, issuer:zPROD

By some magic the JWT arrive at my web server, and I have configured the server to lookup the information in RACF.

In this scenario the realm is either LINUX or ZPROD.

I define mapping on z/OS (RACMAP) which say

  • For (subject: COLIN, Realm: LINUX) set the userid=NOONE
  • For (subject: COLIN, Realm: zPROD) set the userid=IBMUSER

So depending what is in the payload I can get a different userid on z/OS to issue my MQ commands.

A more complex example

In my MQWEB server I have definitions like

openidConnectClient: when issuerIdentifier=”zOSMF” and the signing certificate matches the keyring label “COLINPAICE” in keyring …. then use realm=”zPROD”

For user COLIN, this would match a RACMAP with

For (subject: COLIN, Realm: zPROD) set the userid=IBMUSER.

Whoops

With the configuration

<openidConnectClient
mapIdentityToRegistryUser="true"
...
/>

it ignored the realms, and used the sysplex name.

I got a RACF message

ICH408I USER(START1  ) GROUP(SYS1    ) NAME(####################) 
DISTRIBUTED IDENTITY IS NOT DEFINED:
ADCDC ADCDPL

My z/OS has sysplex name of ADCDPL.

Which says there was no mapping for the userid ADCDC, and the realm name ADCDPL. This took me half a day to resolve!

With this if I had configured RACMAP to have subject ADCDC and realm ADCDPL mapping to SYS1U – if a request came in from z/OS or Linux both would get userid SYS1U – which I am sure you do not want to happen.

The short answer is a realm is a zone of definitions it could be a machine, or a company. OK should now be perfectly obvious.

Zowe cli: Tracing the data

I spent a long time trying to find what data from a Zowe CLI commands was being sent to the backend. I did traces on the back end – but they did not show me the data. I eventually found traces for the work station end.

Trace the tls setup

export NODE_DEBUG='tls,https'

The tls trace gave me

TLS 18487: client _init handle? true
TLS 18487: client initRead handle? true buffered? false
TLS 18487: client _start handle? true connecting? false requestOCSP? false
TLS 18487: client emit session
TLS 18487: client onhandshakedone
TLS 18487: client _finishInit handle? true alpn false servername false
TLS 18487: client emit secureConnect. rejectUnauthorized: false, authorizationError: SELF_SIGNED_CERT_IN_CHAIN

The https trace gave me the output below. Certificates were used, and these were in the traced data

HTTPS 18509: createConnection [Object: null prototype] {
headers: {
'Content-Type': 'application/json',
'ibm-mq-rest-csrf-token': 'true'
},
hostname: '10.1.1.2',
method: 'POST',
path: null,
port: 9443,
rejectUnauthorized: false,
timeout: 5000,
cert: <Buffer ... 4021 more bytes>,
key: <Buffer 2d ... 1654 more bytes>,
_defaultAgent: Agent {
_events: [Object: null prototype] {
free: [Function (anonymous)],
newListener: [Function: maybeEnableKeylog]
},
_eventsCount: 2,
_maxListeners: undefined,
defaultPort: 443,
protocol: 'https:',
options: [Object: null prototype] {
keepAlive: true,
scheduling: 'lifo',
timeout: 5000,
noDelay: true,
path: null
},
requests: [Object: null prototype] {},
sockets: [Object: null prototype] {
'10.1.1.2:9443:::Certificate:\n
Data:\n
Version: 3 (0x2)\n Serial Number: 683 (0x2ab)\n
Signature Algorithm: ecdsa-with-SHA256
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----\n
:::-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----\n::false:::::::::::::': []
},
freeSockets: [Object: null prototype] {},
keepAliveMsecs: 1000,
keepAlive: true,
...
},
host: '10.1.1.2',
keepAlive: true,
scheduling: 'lifo',
noDelay: true,
servername: '',
_agentKey: '10.1.1.2:9443:::Certificate:\n' +
' Data:\n' +
' Version: 3 (0x2)\n' +
' Serial Number: 683 (0x2ab)\n' +
' Signature Algorithm: ecdsa-with-SHA256\n'
...

'-----BEGIN CERTIFICATE-----\n' +
...
'-----END CERTIFICATE-----\n' +
':::-----BEGIN PRIVATE KEY-----\n' +
...
'-----END PRIVATE KEY-----\n' +
'::false:::::::::::::',
encoding: null,
keepAliveInitialDelay: 1000
}

Note: it traces setting up the handshake, it does not trace the application data flow.

Trace the application data

The section above shows the data for the TLS handshake obtained with NODE_DEBUG
To get the application data you need the Zowe CLI trace.

See Setting Cli Trace levels.

ZOWE_APP_LOG_LEVELZowe CLI logging levelLog4JS log levels (OFF, TRACE, DEBUG, INFO, WARN, ERROR, FATAL)WARN
ZOWE_IMPERATIVE_LOG_LEVELImperative CLI Framework logging levelLog4JS log levels (OFF, TRACE, DEBUG, INFO, WARN, ERROR, FATAL)WARN

The output is in the directory pointed to by ZOWE_CLI_HOME – which defaults to ~/.zowe/logs on Linux.

ZOWE_APP_LOG_LEVEL

The ZOWE_APP_LOG_LEVEL trace is boring. It is in home/colinpaice/.zowe/logs/zowe.log

[2025/08/02 15:05:37.754] [TRACE] [main.js:40] Init was successful
[2025/08/02 15:05:37.837] [DEBUG] [MQSessionUtils.js:22] Creating an MQ session from arguments
[2025/08/02 15:05:37.845] [INFO] [ConfigAutoStore.js:135] Skipping update of profile properties. Check that config file exists and autoStore is true.

ZOWE_IMPERATIVE_LOG_LEVEL

The ZOWE_IMPERATIVE_LOG_LEVEL=debug trace in logs/imperative.log has

[2025/08/02 15:12:02.162] [TRACE] [AppSettings.js:39] Attempting to load settings file: /home/colinpaice/tmp/settings/imperative.json
[2025/08/02 15:12:02.163] [TRACE] [AppSettings.js:56] Settings were loaded
[2025/08/02 15:12:02.163] [TRACE] [AppSettings.js:57] Loaded Settings:
[2025/08/02 15:12:02.163] [TRACE] [AppSettings.js:58] { overrides: { CredentialManager: '@zowe/cli' } }
[2025/08/02 15:12:02.163] [DEBUG] [ConfigManagementFacility.js:55] ConfigManagementFacility.init() - Start
[2025/08/02 15:12:02.163] [DEBUG] [UpdateImpConfig.js:38] Adding definition = 'config'
[2025/08/02 15:12:02.162] [TRACE] [AppSettings.js:39] Attempting to load settings file: /home/colinpaice/tmp/settings/imperative.json
[2025/08/02 15:12:02.163] [TRACE] [AppSettings.js:56] Settings were loaded
[2025/08/02 15:12:02.163] [TRACE] [AppSettings.js:57] Loaded Settings:
...

Lots of profile information and the schema…

The command entered

[2025/08/02 15:12:02.410] [DEBUG] [CommandYargs.js:75] Defining command: mqsc
[2025/08/02 15:12:02.411] [DEBUG] [CommandYargs.js:174] Building positional string from: mqsc
[2025/08/02 15:12:02.412] [DEBUG] [CommandYargs.js:180] Positional String: [qmgr] [cmd]
[2025/08/02 15:12:02.415] [DEBUG] [CommandYargs.js:94] Defining command builder for: mqsc
[2025/08/02 15:12:02.421] [DEBUG] [CommandYargs.js:105] Handler invoked for: mqsc
[2025/08/02 15:12:02.422] [DEBUG] [CommandYargs.js:118] Executing Handlers: mqsc
[2025/08/02 15:12:02.423] [DEBUG] [CommandYargs.js:136] Executing Handlers (1 total)
[2025/08/02 15:12:02.433] [INFO] [CommandProcessor.js:254] Invoking command "mqsc"...
[2025/08/02 15:12:02.439] [INFO] [CommandProcessor.js:255] Command issued:

zowe --mq-p mq --cert-key-file ./colinpaice.key.pem --cert-file ./colinpaice.pem --host 10.1.1.2 --port 9443 mq run mqsc CSQ9 DIS QMGR ALL


[2025/08/02 15:12:02.439] [TRACE] [CommandProcessor.js:256] Invoke parameters:
{
arguments: {
_: [ 'mq', 'run', 'mqsc' ],
'mq-p': 'mq',
'mq-profile': 'mq',
mqP: 'mq',
mqProfile: 'mq',
'cert-key-file': './colinpaice.key.pem',
certKeyFile: './colinpaice.key.pem',
'cert-file': './colinpaice.pem',
certFile: './colinpaice.pem',
host: '10.1.1.2',
H: '10.1.1.2',
port: '9443',
P: '9443',
version: undefined,
V: undefined,
'available-commands': undefined,
ac: undefined,
availableCommands: undefined,
'response-format-json': undefined,
rfj: undefined,
responseFormatJson: undefined,
help: undefined,
h: undefined,
'help-web': undefined,
hw: undefined,
helpWeb: undefined,
'help-examples': undefined,
helpExamples: undefined,
'reject-unauthorized': undefined,
ru: undefined,
rejectUnauthorized: undefined,
'show-inputs-only': undefined,
showInputsOnly: undefined,
'$0': 'zowe',
qmgr: 'CSQ9',
cmd: 'DIS QMGR ALL'
},
silent: false,
responseFormat: 'default'
}

More stuff

Send the request


[2025/08/02 15:12:02.499] [INFO] [AbstractRestClient.js:339] Setting socket connection timeout ms: 60000
[2025/08/02 15:12:02.500] [TRACE] [AbstractRestClient.js:538] Using PEM Certificate authentication
[2025/08/02 15:12:02.501] [TRACE] [AbstractRestClient.js:807] appendInputHeaders called with options on rest client {"headers":{},"hostname":"10.1.1.2","method":"POST","path":"/ibmmq/rest/v1/admin/action/qmgr/CSQ9/mqsc","port":9443,"rejectUnauthorized":false,
"timeout":60000,
"cert":{"type":"Buffer","data":[67,101,114,...,10]},
"key":{"type":"Buffer","data":[45,45,45,45,45,66,69,71,73,78,32,80,82,...5,10]}
}

MQRestClient
[2025/08/02 15:12:02.501] [TRACE] [AbstractRestClient.js:440] Rest request: POST 10.1.1.2:9443/ibmmq/rest/v1/admin/action/qmgr/CSQ9/mqsc
[2025/08/02 15:12:02.541] [DEBUG] [AbstractRestClient.js:173] will write data for request
[2025/08/02 15:12:02.542] [DEBUG] [AbstractRestClient.js:178] writing JSON for request
[2025/08/02 15:12:02.542] [TRACE] [AbstractRestClient.js:179] JSON body: {"type":"runCommand","parameters":{"command":"DIS QMGR ALL"}}
[2025/08/02 15:12:02.916] [DEBUG] [AbstractRestClient.js:575] Content length of response is: 3299
[2025/08/02 15:12:02.919] [TRACE] [AbstractRestClient.js:626] Data chunk received...
[2025/08/02 15:12:02.924] [DEBUG] [AbstractRestClient.js:668] onEnd() called for rest client MQRestClient
[2025/08/02 15:12:02.934] [INFO] [CommandProcessor.js:476] Handler for command "mqsc" succeeded.
[2025/08/02 15:12:02.937] [INFO] [CommandProcessor.js:824] Command "mqsc" completed with success flag: "true"
[2025/08/02 15:12:02.938] [TRACE] [CommandProcessor.js:825] Command "mqsc" finished.

and finally the response


{
success: true,
exitCode: 0,
message: '',
stdout: <Buffer 52 75 0a ... 1915 more bytes>,
stderr: <Buffer >,
data: {
commandResponse: [
{
completionCode: 0,
reasonCode: 0,
text: [
'CSQN205I COUNT= 3, RETURN=00000000, REASON=00000000',
'CSQM409I %CSQ9 QMNAME(CSQ9 ) DESCR(CSQ9, IBM MQ for z/OS - V9.0.1 )
...
"CSQ9022I %CSQ9 CSQMDRTS ' DIS QMGR' NORMAL COMPLETION"
]
}
],
overallReasonCode: 0,
overallCompletionCode: 0
},
error: undefined
}

MQWEB and passtickets

The RACF PassTicket is a (one-time-only/short duration) password that is generated by a requesting product or function. It is an alternative to the RACF password.
You create a passticket specifying the userid and the application, and a one off password is generated. You can specify a validity period.

By default the passticket has replay protection – in that once used, the passticket cannot be used again, and so prevent replay. You can allow a passticket to be used more than once either by specifying APPLDATA(‘NO REPLAY PROTECTION’) for basic pass tickets, or REPLAY(YES) for enhanced pass tickets.

The server can use the function __login__applid() (or similar function) to run a thread as the specified userid. You pass the userid, password (pass ticket) and the application to use.

The MQWeb server is code running on top of Liberty Web server.

For my MQWeb server, running as started task CSQ9WEB, it was configured so my mqweb/mqwebuser.xml configuration file had <safCredentials profilePrefix=”MQWEB“…./>

I created a passticket for my userid COLIN, and application MQWEB, and I was able to logon to the the MQWEB server using userid COLIN and with the pass ticket as my password.

Creating and using pass tickets on z/OS.

As part of looking into secure way of logging on to z/OS, I looked into pass tickets (because Zowe can generate a pass ticket to connect to other sub-systems). I set up the simplest (and oldest) pass ticket configuration. The best practice is to use enhanced pass tickets, and store values encrypted. With enhanced pass tickets you can specify the validity period of the pass ticket – it defaults to 10 minutes. I wanted the easiest way, so I used the older technique.

With thanks to Philippe Richard for his many comments, I’ve incorporated them in the post.

I had the usual struggles with getting the C program to work, but overall it was quite easy.

I successfully used the RACF callable function IRRSPK00 R_ticketserv (IRRSPK00).

You pass a userid and an application name and the service returns a temporary, time limited, password for that userid and application.

The application name depends on what system you are logging on to. I submitted a job from TSO on system with SYSID S0W1, and the application name was TSOS0W1. You cannot use a pass ticket for TSO on a CICS system, because the application names will not match.

When you are under TSO and enter submit the application is still TSO, so use TSOS0W1.

If, for instance, you try to submit a job through the internal reader, then it will use application MVSS0W1.

For example:

//INTRDRS1 EXEC PGM=IEBGENER 
//SYSUT1 DD DSN=PASSTIKT.ENHC.JCL(REFRESH),
// DISP=SHR
//SYSUT2 DD SYSOUT=(,INTRDR)
//*
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY

and in member REFRESH, you have a job with userid=racf admin user, password= where you substitute the passticket, like:

//SYSADMX JOB 30000000,’MVS JOB CARD ‘,MSGLEVEL=(1,1), 
// CLASS=A,MSGCLASS=Q,NOTIFY=&SYSUID,TIME=1440,REGION=0M,
// USER=SYSADM,PASSWORD=PSEG7TXM,
// JOBRC=MAXRC
//IEFPROC EXEC PGM=IKJEFT01,REGION=4M,DYNAMNBR=10
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
SETROPTS RACLIST(PTKTDATA) REFRESH

It will use MVSS0W1 as the application ID.

Andrew Mattingly has written a very well detailed blog on passtickets which is well worth a read.

It described with ample details the algorithm and the various techniques to generate pass-tickets.

Security definitions

The security definitions are in two parts

  • The profile for using a pass ticket, for example, who can use a ticket for logging on to TSO,
  • The profile for which userids can create a pass ticket.

Who can use a pass ticket with which application

You can limit who can use the application, for example

  • a profile just TSOS0W1,
  • or members of group SYS1 profile TSOS0W1.SYS1,
  • or a userid COLIN in group SYS1, profile TSOS0W1.SYS1.COLIN

Example definitions for TSOS0W1 profile

RDEFINE PTKTDATA TSOS0W1  SSIGNON(KEYMASKED(7E4304D681920260)) - 
APPLDATA('NO REPLAY PROTECTION')

SETROPTS RACLIST(FACILITY,PTKTDATA) REFRESH

The server, TSO in this case, can use the function __login__applid() to run the thread as the specified userid. You pass the userid, password (pass ticket) and the application to use (TSOS0W1).

Who can define which pass tickets?

You have to define a RACF profile for the application name, and a profile for userids than can generate a pass ticket for that application.

RDEFINE PTKTDATA   IRRPTAUTH.TSOS0W1.*  UACC(NONE)
PERMIT IRRPTAUTH.TSOS0W1.* CLASS(PTKTDATA) ID(COLIN) ACCESS(UPDATE)
PERMIT IRRPTAUTH.TSOS0W1.* CLASS(PTKTDATA) ID(IBMUSER)ACCESS(UPDATE)
SETROPTS RACLIST(PTKTDATA) REFRESH

The above statements define a profile for defining pass ticket with the TSOS0W1 application.

Userids COLIN and IBMUSER can define pass tickets for this application.

What can you use to generate a pass ticket?

My application code

See C calling a function setting the high order bit on, and passing parameters for a discussion about calling the callable service, and passing the parameters.

 //   Code to generate a pass ticket 
#pragma linkage(IRRSPK00 ,OS)
#pragma runopts(POSIX(ON))
/*Include standard libraries */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <iconv.h>

int main( int argc, char *argv??(??))
{
if (argc != 3)
{
printf("Syntax is %s userid applid\n",argv[0]);
return 12 ;
}
if (strlen(argv[1]) > 8)
{
printf("length of userid must be <= 8\n");
return 12;
}
if (strlen(argv[2]) > 8)
{
printf("length of applid must be <= 8\n");
return 12;
}

char work_area[1024];
int Option_word = 0;
int rc;
long SAF_RC,RACF_RC,RACF_RS;
SAF_RC=0 ;
long ALET = 0;
short Function_code= 3;
struct {
short length;
char value[8];
} appl;
struct {
short length;
char value[8];
} userid;
struct {
short length;
char value[20];
} ticket;
ticket.length=20;
char * u= argv[1] ;
strncpy(&userid.value[0],u,8);
userid.length =strlen(u);
char * pAppl = argv[2];
strncpy(&appl.value[0],pAppl,8);
appl.length =strlen(pAppl);


int Ticket_options = 1;
int * pTO = & Ticket_options;

rc=IRRSPK00(
&work_area,
&ALET , &SAF_RC,
&ALET , &RACF_RC,
&ALET , &RACF_RC,
&ALET , &RACF_RS,
&ALET ,&Function_code,
&Option_word,
&ticket, // length followed by area
&pTO,
&userid,
&appl
);

printf("return code SAF %d RACF %d RS %d \n",
SAF_RC,RACF_RC,RACF_RS );
if (SAF_RC == 0)
{
int l = ticket.length;
printf("Pass ticket:%*.*s\n",l,l,ticket.value);
}
return SAF_RC;

}

The compile JCL was

//IBMPASST   JOB 1,MSGCLASS=H,COND=(4,LE) 
//S1 JCLLIB ORDER=CBC.SCCNPRC
// SET LOADLIB=COLIN.LOAD
//DOCLG EXEC PROC=EDCCB,INFILE='COLIN.C.SOURCE(TICKET)',
// CPARM='OPTF(DD:COPTS)'
//COMPILE.ASMLIB DD DISP=SHR,DSN=SYS1.MACLIB
//COMPILE.COPTS DD *
LIST,SOURCE
aggregate(offsethex) xref
SEARCH(//'ADCD.C.H',//'SYS1.SIEAHDR.H')
TEST
ASM
RENT ILP32 LO
OE
NOMARGINS EXPMAC SHOWINC XREF
LANGLVL(EXTENDED) sscom dll
DEFINE(_ALL_SOURCE)
DEBUG
/*
//BIND.SYSLMOD DD DISP=SHR,DSN=&LOADLIB.
//*IND.SYSLIB DD DISP=SHR,DSN=&LIBPRFX..SCEELKED
//*IND.OBJLIB DD DISP=SHR,DSN=COLIN.OBJLIB
//BIND.CSS DD DISP=SHR,DSN=SYS1.CSSLIB
//BIND.SYSIN DD *
INCLUDE CSS(IRRSPK00)
NAME TICKET(R)
/*
//START1 EXEC PGM=TICKET,REGION=0M,PARM='ADCDB TSOS0W1'
//STEPLIB DD DISP=SHR,DSN=&LOADLIB
//SYSERR DD SYSOUT=*,DCB=(LRECL=200)
//SYSERROR DD SYSOUT=*,DCB=(LRECL=200)
//SYSOUT DD SYSOUT=*,DCB=(LRECL=200)
//SYSPRINT DD SYSOUT=*,DCB=(LRECL=200)
//CEEDUMP DD SYSOUT=*,DCB=(LRECL=200)
/&

Problems

I could not get R_GenSec (IRRSGS00 or IRRSGS64): Generic security API interface RACF callable services to work because of the 31 bit program, and the service expecting 64 bit addresses.

This blog post has code which uses R_GenSec in 64 bit C.

Zowe explorer: first baby steps to using z/OS files

When people start to use ISPF they may struggle, because they think it is not obvious. I’ve been using it for over 40 years, and I use it without thinking. My fingers just do it.
I struggled when I used Zowe explorer, but my fingers are starting to remember the keystrokes.

If you change your team configuration you can use fast refresh. You can close and restart vscode or use

  • open command palette with ctrl+shift+p
  • type or select developer: reload window

You can change Zowe Explorer settings.

Ctrl+Shift+P Preferences: Open user settings (JSON)
As you type it displays the values for what you have typed, so I had only to type pref, and the one I wanted was near the top of the list.

You can also enter just the first letter of each word, so typing ou would display a short list of valid words.

This brings up a file which you can edit. It is often easier to work with the JSON, because not all options are covered.

Working with ISPF to edit a member

When I work with ISPF, I typically

  • Use ISPF 3.4 to edit a data set, this lists the members
  • 6 tabs and press enter to sort the members, so the most recently changed members are at the top,
  • Shift return to tab down, to the first member
  • type E on the line prefix and press enter to edit it

It takes longer to read – than to execute it.

To do ISPF 3.4 in Zowe explorer

  • Activate the Zowe side bar
  • Single click on DATASETS
  • Single click on your profile
  • You can click on the magnifying lens icon (the search icon) and give a fully or partially qualified dataset name
  • Single click on your data set to list the members

Sort the members

  • Single right click on the data set
  • Single click on Sort PDS members.
    • click on Sort direction first
    • then click on the attribute
    • If you click on attribute first, you have to click on Sort PDS members a second time to be able to select the attribute

Edit the member

Single click on the member to edit it.

Big lists of members comes in pages

To speed up list processing, if there are more than 100 members in a PDS, then Zowe Explorer displays “pages”.For example

The Next page 2/2 is highlit, so there is a page available page 2 out of 2. The Previous page is grey, so there are no more pages before this one.

You can disable this, or change the number of entries per page in the settings.

Setting default sort

I added into the file the text zowe.ds.default.sort…. below

"window.menuBarVisibility": "compact",
"editor.rulers": [
],
"zowe.ds.default.sort": {
"method": "Name",
"direction": "Ascending"
}

See here.

C calling an “assembler” function, setting the high order bit on, and passing parameters.

Since days of old when knights were bold, the standard parameter list to call an assembler function was to pass the addresses of the parameters, and set on the top bit of the address for the last address.
This way the called function knows how many parameters have been passed, and you do not need to pass a parameter count.

Setting the high order bit on, for the last parameter

I had to ask for help to remind me how to do it from C, so I could call “Assembler” functions.

You can get C to do this using

#pragma linkage(IRRSPK00 ,OS)

Example

The syntax of the routine from the RACF callable services documentation is

CALL IRRSPK00 (Work_area,
ALET, SAF_return_code,
ALET, RACF_return_code,
ALET, RACF_reason_code,
ALET, Function_code,
Option_word,
Ticket_area,
Ticket_options,
Ticket_principal_userid,
Application_Id
)

Here is part of my C program.

#pragma linkage(IRRSPK00 ,OS)
...
long SAF_RC,RACF_RC,RACF_RS;
SAF_RC=0 ;
long ALET = 0;
// ticket options needs special treatment, see below
int Ticket_options = 1;
int * pTO = & Ticket_options;

rc=IRRSPK00(
&work_area,
&ALET , &SAF_RC,
&ALET , &RACF_RC,
&ALET , &RACF_RS,
&ALET ,&Function_code,
&Option_word,
&ticket, // length followed by area
&pTO,
&userid,
&appl
);

If you use #pragma linkage(IRRSPK00 ,OS) it sets on the high order bit. You pass the address of the parameters. I just used &variable, there are other ways.

Passing variables

Most of the parameters are passed by address for example &ALET inserts the address of the variable, conforming to the z/OS standards.

There is a field Ticket_principal_userid which is the name of a 10-byte area that consists of a 2-byte length field followed by the userid id for whom a PassTicket operation is to be performed followed by an 8-byte PassTicket field.

I defined a structures for each variable like

struct {
short length;
char value[8];
} ticket;

In the program I used &ticket.

Ticket option

The documentation says

Ticket_options: The name of a fullword containing the address of a binary bit string that identifies the ticket-specific processing to be performed.

It took me a while to understand what this meant. I had to use

int Ticket_options = 1; 
int * pTO = & Ticket_options;

and use it

int Ticket_options = 1; 
int * pTO = & Ticket_options;
...
&ticket, // length followed by area
&pTO,

Whoops R_GenSec (IRRSGS00 or IRRSGS64): Generic security API interface

I had great problems getting this to work. The documentation said

The address double words from 31 bit callers should have the first word filled with
zeros and the second word filled with the 31 bit address. Sub-parameter addresses will be in the format of the AMODE of the caller.

I do not know what this means. When I coded it as expected I got

CEE3250C The system or user abend S0E0 R=00000029

Which means invalid ALET supplied.

I converted the program to 64 bit and it still failed!

Getting into supervisor mode and other hard things in C, is easy.

Some functions in z/OS need a user to be in a privileged state such as key zero or supervisor state. Writing the code in assembler has been pretty easy, but writing it in a C program has been hard.

For example you could write a function in assembler, and call it. This has the cross language challenges.

I recently found an easy way – just reuse some code from Zowe. This is open source, so you need to follow the license

This program and the accompanying materials are made available under the terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at https://www.eclipse.org/legal/epl-v20.html

SPDX-License-Identifier: EPL-2.0

Copyright Contributors to the Zowe Project.

The code uses __asm() Inline assembly statements (IBM extension).

The functions are

  • ddnameExists
  • wtoMessage
  • wtoPrintf3
  • atomicIncrement compare and swap
  • testauth
  • extractPSW
  • supervisorMode or back to problem state
  • setKey
  • getExternalSecurityManager
  • getCVT
  • getATCVT
  • getIEACSTBL
  • getCVTPrefix
  • getECVT
  • getTCB
  • getSTCB
  • getOTCB
  • getASCB
  • getASXB
  • getASSB
  • getJSAB
  • getCurrentACEE
  • getFirstChildTCB
  • getParentTCB
  • getNextSiblingTCB
  • resolveSymbolBySyscall Input: A symbol starting with & and not ending with .
  • resolveSymbol Input: A symbol starting with & and not ending with .
  • lots of saf functions see here
  • loadByName
  • getDSAB Data Set Association Block
  • isCallerLocked
  • isCallerCrossMemory

Some of these need to be APF protected, so although it is easy to use the above code, you may still need to get the load library APF authorised, and the code approved.


For example

Get into supervisor state

The code here.

int supervisorMode(int enable){
// Use MODESET macro for requests
int currentPSW = extractPSW();
if (enable){
if (currentPSW & PROBLEM_STATE){
__asm(ASM_PREFIX
" MODESET MODE=SUP \n"
:
:
:"r0","r1","r15");
return TRUE;
} else{
return FALSE; /* do nothing, tell caller no restore needed */
}
} else{
if (currentPSW & PROBLEM_STATE){
return TRUE; /* do nothing, tell user was in problem state */
} else{
__asm(ASM_PREFIX
" MODESET MODE=PROB \n"
:
:
:"r0","r1","r15");
return FALSE;
}
}
}

To compile it I used

// SET LOADLIB=COLIN.LOAD 
//DOCLG EXEC PROC=EDCCB,INFILE='ADCD.C.SOURCE(C)',
// CPARM='OPTF(DD:COPTS)'
//* CPARM='LIST,SSCOMM,SOURCE,LANGLVL(EXTENDED)'
//COMPILE.ASMLIB DD DISP=SHR,DSN=SYS1.MACLIB
//COMPILE.COPTS DD *
LIST,SOURCE
aggregate(offsethex) xref
SEARCH(//'ADCD.C.H',//'SYS1.SIEAHDR.H')
TEST
ASM
RENT ILP32 LO
OE
NOMARGINS EXPMAC SHOWINC XREF
LANGLVL(EXTENDED) sscom dll
DEFINE(_ALL_SOURCE)
DEBUG
/*
...

You need to specify

  • //COMPILE.ASMLIB for the the assembler macro libraries.
  • and the compiler option ASM which enables inlined assembly code inside C/C++ programs.

I was all so easy, once I had been told about it.

Wow, how to logon securely is so complex….

I’ve been looking into login on to Zowe and z/OSMF, and have realised how complex it is to set up secure logon.

An obvious fact about real life

If someone has access to your machine, for example a hacker, then they have access to your files, and data in virtual storage.

I cover

A lot of the time it is not difficult, but you need to handle the edge cases. When there are multiple dimensions to the problem, the edge case becomes the corner case, and this is where it gets really hard.

I use the expression is not secure. Being secure is relative. If your machine is air gapped with no external access, the machine should be secure. If people can access your files, perhaps on a shared machine, or a hacker can accessed your machine, I would not consider this secure.

The simplest case of logging on manually from a client machine to the server

Of course you use a TLS session to ensure the session traffic is encrypted.

You can type the values into the userid and password fields. It works, simple. If I disconnect and reconnect, I need to re-enter the userid and password

If I am using more than one back end server, I’ll have to enter the userid and password for each system, while the session is active.

Once the userid and password have been used, the fields in memory should be overwritten (but I doubt if this is always the case), to minimise the time window when a hacker extract the values from memory.

The simple case of logging on from a script from the client to the server.

You will not be there to enter the password so it needs to be stored.

The script may prompt you for the userid and password. You enter the information once, and it keeps them in virtual storage, and does not write them to disk. At the end of the session these values need to be overwritten in case a hacker is wandering round your system.

The application may write the information to disk – and we immediately hit a major problem. If a hacker has access to your system, then they can access (and copy away from your machine) the file where the password is stored. On Linux I displayed the contents of the “secure store” with a few lines of Python. I believe it is the same for Windows and Apple machines. (If your userid needs access to the secure store for something, a hacker thread running with your userid can access the store).

We have quickly seen that the use of a password stored on the machine is not secure.

Use of a certificate

As part of the TLS handshake there is a private key kept on the local machine, and a public key is sent to the partner.

Trusting a certificate

As part of setting up the public key, I send the public key to the server. The server’s Certificate Authority does a checksum of the public key, then encrypts the checksum with the server’s private key. The public certificate, encrypted checksum and the CA’s public key are sent back to the originator.

When the public certificate is used as part of the TLS handshake, the server does the same checksum calculation as before on public certificate, and saves it momentarily. It takes the encrypted checksum from the payload, decrypts it, and compares the two checksum values. If they match, then the client’s certificate can be trusted and has not been changed. Therefore the certificate is trusted (for a given level of trust) to represent the end user.

My program has access to my private key – so authenticates as me

As part of the handshake data, the client encrypts some data using its private key, and sends this to the server. The server uses the public key it received, and decrypts this value. If the decrypted data matches what it is expecting, then this is proof that the client has the private key and is who they say they are.

The server can then use the mapping of “name in the public key” to a userid, to determine which userid the requester should run under.

Once this is set up, the client can connect to the server without specifying a userid and password.

There is a proposal to reduce the validity time of personal certificates from over a year to 47 (or less) days. This reduces the time window when a certificate can be used.

This is great … but

If your private key is stored on your disk, a hacker can steal a copy of the file and impersonate you. So once again this is not very secure.

Use a private key – external to your computer

You can get a dongle, such as a Yubikey which keep the private key secured on a detachable USB dongle. To use it, a request is passed to the dongle saying “please encrypt this data”. It encrypts the data, returns it, and it can be sent to the server. You cannot extract the private key from the dongle.

To use this for authentication you physically need the dongle. If someone has control of your machine, they can use this dongle when it is plugged in. If they just have a copy of your files they cannot use the dongle, and so this is secure.

If you remove the dongle when you are not using it, we have a secure solution, until you plug it back in! Most people may forget and leave the dongle in all day.

I’ve been to sites, where the private key is stored on their badge. When they want to authenticate they put their badge on a badge reader and authenticate. This is awkward, so they immediately take their badge off the reader after authentication.

Setting up these external key stores is not trivial.

Use of Multi Factor Authentication(MFA)

One of the main approaches to MFA is

  • something you have
  • something you know

MFA is often used in applications such as online banking.For some transactions you will have a code sent to your email address. Something you know is your application password, something you have is the code sent to you. To use the service, an application password and access to your email is required.

MFA can also use one time passwords – so if someone is monitoring your network traffic they will not be able to reuse the one-time-password.

You can get the one-time-password generated from your badge, a dongle attached to your machine, or an application, such as an authenticator application on your phone. Once these apps have been configured to the back-end, you can press a button and get a one time code.

To logon you may need your normal logon password (which you change monthly) and the one time code from the MFA device. You might just need the one-time-code – depends on how the environment has been set up.

This is great for you logon to a backed and stay connected. If you have script, or are using multiple back-end servers. This becomes impractical. You need a new one-time-code each time you logon. This can be automated if the MFA device is attached to your machine, but not if you are using a mobile phone application.

One small problem is when the user’s password has expired. They are prompted to change it, and now need another One Time Password – and they may have to wait for a period (seconds) before a new one is generated.

JSON Web tokens

See Are JSON Web Tokens secure? – Yes if used properly.

A JSON Web Token(JWT) provides a time limited key, and so avoids the problems when changing a password. Some of the concepts are similar to a certificate logon, but with a limited validity, from minutes to hours.

With RACF these are known as Identity token. (Having a different name, means RACF development can extend the support to cover additional tokens types.)

A JWT has three parts

  • Information about the JWT. This JWT has used an RSA certificate, and algorithm SHA256.
  • Identity information. The userid is…, it was issued by ( z/OSMF, or SAF), at this time…. and is valid until….
  • The above parts are check summed, and the check sum is encrypted using a private key.

The above 3 parts are base 64 encoded, and joined together with a ‘.’ between them.

Once a JWT token is created, it is valid until it expires. You need to consider if you want a JWT to be valid for a few minutes, or all day.

You create (or have the system create for you) the header and the payload. You calculate the checksum, and encrypt the checksum with the private key.

The parts of the JWT are assembled and sent to the partner.

The partner uses its copy of the public key, and checks that the checksum matches what it expects and, if it matches, can use the information in the payload part. This validation could be a call to a RACF service, a https request to a server, or Java.

Once the JWT has been validated, the server can use the information in the payload.

You do not want to use the userid in the JWT directly, because the userid COLIN on one system has no authority, but has super user authority on another system.

With RACF you can map a userid string to a RACF userid using profiles defined with the RACMAP command. See the r_usermap service.

While the JWT is valid it could be used by a hacker.

Overall

It looks like there is not one good solution to cover all cases.

Someone said “assume your machine will be hacked. Minimise what damage they can do”. This may be as simple as turning your machine off overnight, so the hackers cannot control it

The solution for scripts may be different to people logging on.

You may want to have a userid just for scripts, which has only enough authority and permissions to work, and no additional permissions.

You need to consider the options and risks, and not just sleepwalk into having an insecure system.