Parsing command line values

I wanted to pass multiple parameters to a z/OS batch program and parse the data. There are several different ways of doing it – what is the best way ?

This question is complicated by

Checking options

Processing command line options can mean a two stage process. Reading the command line, and then checking to ensure a valid combination of options have been specified.

If you have an option -debug with a value in range 0 to 3. You can either check the range as the option is processed, or have a separate section of checks once all the parameters have been passed. If there is no order requirement on the parameters you need to have separate code to check the parameters. If you can require order to the parameters, you might be able to have code “if -b is specified, then check -a has already been specified

I usually prefer a separate section of code at it makes the code clearer.

Command styles

On z/OS there are two styles of commands

def xx(abc) parm1(value) xyz

or the Unix way

-def -xx abc -parm1 -1 -a –value value1 -xyz.

Where you can have

  • short options “-a” and “-1”
  • long option with two “-“, as in “–value”,
  • “option value” as is “-xx abc”
  • “option and concatenated value” as in “-xyz”; option -x, value yz

I was interested in the “Unix way”.

  • One Unix way is to have single character option names like -a -A -B -0. This is easy to program – but it means the end user needs to lookup the option name every time as the options are not usually memorable.
  • Other platforms (but not z/OS) have parsing support for long names like – -userid value.
  • You can parse a string like ro,rw,name=value, where you have keyword=value using getsubopt.
  • I wrote a simple parser, and a table driven parser for when I had many options.

Defining the parameter string toJCL.

The traditional way of defining a parameter string in batch is EXEC PGM=MYPROG,PARM=’….’ but the parameter is limited in length.

I tend to use

// SET P1=COLIN.PKIICSF.C 
// SET P2="optional"
//S1 EXEC PGM=MYPROG,PARM='parms &P1 &P2'  

You can get round the parameter length limitation using

//ISTEST   EXEC PGM=CGEN,REGION=0M,PARMDD=MYPARMS 
//MYPARMS DD * 
/ 
 -detail 0 
 -debug 0 
 -log "COLINZZZ" 
 -cert d

Where the ‘/’ on its own delimits the C run time options from my program’s options.

The values are start in column 2 of the data. If it starts in column 1, the value is concatenated to the value in the previous line.

You can use JCL and System symbols

// EXPORT SYMLIST=(*) 
// SET LOG='LOG LOG' 
//ISTEST   EXEC PGM=CGEN,REGION=0M,PARMDD=MYPARMS 
//MYPARMS DD *,SYMBOLS=EXECSYS
/ 
 -log "COLINZZZ" 
 -log "&log"
 ...

This produced -log COLINZZZ -log “LOG LOG”

Parsing the data

C main programs have two parameters, a count of the number of parameter, and an array of null terminated strings.

You can process these

int main( int argc, char *argv??(??)) 
{ 
  int iArg; 
  for (iArg = 1;iArg< argc; iArg ++   ) 
  { 
    printf(".%s.\n",argv[iArg]); 
  } 
  return 0; 
} 

Running this job

//CPARMS   EXEC  CCPROC,PROG=PARMS 
//ISTEST   EXEC PGM=PARMS,REGION=0M,PARMDD=MYPARMS 
//MYPARMS DD * 
/ 
 -debug 0 
 -log "COLIN  ZZZ" 
 -cert 
 -ae colin@gmail.com 

gave

.-debug.                   
.0.                        
.-log.                     
.COLIN  ZZZ.               
.-cert.                    
.-ae.                      
.colin@gmail.com.          

and we can see the string “COLIN ZZZ” in double quotes was passed in as a single string.

Parsing with single character options

C has a routine getopt, for processing single character options like -a… and -1… (but not -name) for example

while ((opt = getopt(argc, argv, "ab:c:")) != -1) 
   { 
       switch (opt) { 
       case 'a': 
           printf("-a received\n"); 
           break; 
       case 'b': 
           printf("-b received \n"); 
           printf("optarg %d\n",optarg); 
           if (optarg) 
             printf("-b received value %s\n",optarg); 
           else 
             printf("-b optarg is0       \n"); 
           break; 
       case 'c': 
           printf("-c received\n"); 
           printf("optarg %d\n",optarg); 
           if (optarg) 
             printf("-c received value %s\n",optarg); 
           else 
             printf("-c optarg is0       \n"); 
           break; 
       default: /* '?' */ 
           printf("Unknown n"); 
     } 
   } 

The string “ab:c:” tells the getopt function that

  • -a is expected with no option
  • -b “:” says an option is expected
  • -c “:” says an option is expected

I could only get this running in a Unix environment or in a BPXBATCH job. In batch, I did not get the values after the option.

When I used

//BPX EXEC PGM=BPXBATCH,REGION=0M,
// PARM='PGM /u/tmp/zos/parm.so -a -b 1 -cc1 '

the output included

-b received value b1
-c received value c1

This shows that “-b v1” and “-cc1” are both acceptable forms of input.

Other platforms have a getopt_long function where you can pass in long names such as –value abc.

getsubopt to parse keyword=value

You can use getsubopt to process an argument string like “ro,rw,name=colinpaice”.

If you had an argument like “ro, rw, name=colinpaice” this is three strings and you would have to use getsubopt on each string!

You have code like

int main( int argc, char *argv??(??)) 
{ 
 enum { 
       RO_OPT = 0, 
       RW_OPT, 
       NAME_OPT 
   }; 
   char *const token[] = { 
       [RO_OPT]   = "ro", 
       [RW_OPT]   = "rw", 
       [NAME_OPT] = "name", 
       NULL 
   }; 
   char *subopts; 
   char *value; 

   subopts = argv[1]; 
 while (*subopts != '\0' && !errfnd) { 
   switch (getsubopt(&subopts, token, &value)) { 
     case RO_OPT: 
       printf("RO_OPT specified \n"); 
       break; 
     case RW_OPT: 
       printf("RW_OPT specified \n"); 
       break; 
     case NAME_OPT: 
       if (value == NULL) { 
          printf("Missing value for " 
                 "suboption '%s'\n", token[NAME_OPT]); 
           continue; 
       } 
       else 
         printf("NAME_OPT value:%s\n",value);
         break; 
    default: 
         printf("Option not found %s\n",value); 
         break; 
     }  // switch 
   } // while 
 }  

Within this is code

  • enum.. this defines constants RO_OPT = 0 RW_OP = 1 etc
  • char const * token defines a mapping from keywords “ro”,”rw” etc to the constants defined above
  • getsubopt(&subopts, token, &value) processes the string, passes the mapping, and the field to receive the value

This works, but was not trivial to program

It did not support name=”colin paice” with an imbedded blank in it.

My basic command line parser(101)

I have code

for (iArg = 1;iArg< argc; iArg ++   ) 
{ 
  // -cert is a keyword with no value it is present or not
  if (strcmp(argv[iArg],"-cert") == 0) 
  { 
    function_code = GENCERT    ; 
    continue; 
  } 
  else 
  //  debug needs an option
  if (strcmp(argv[iArg],"-debug") == 0 
      && iArg +1 < argc) // we have a value 
      { 
        iArg  ++; 
        debug = atoi(argv[iArg]); 
        continue; 
      } 
  else 
  ...
  else 
    printf("Unknown parameter or problem near parameter %s\n", 
           argv[iArg]);
  }   // for outer - parameters 

This logic processes keywords with no parameters such as -cert, and keyword which have a value such as -debug.

The code if (strcmp(argv[iArg],”-debug”) == 0 && iArg +1 < argc) checks to see if the keyword has been specified, and that there is a parameter following it (that is, we have not run off the end of the parameters).

Advanced – table – ize it

For a program with a large number of parameters I used a different approach. I created a table with option name, and pointer to the fields variable.

For example

getStr lookUpStr[] = { 
    {"-debug", &debug     }, 
    {"-type",  &type       }, 
    {(char *) -1,  0} 
   }; 

You then check each parameter against the list. To add a new option – you just update the table, with the new option, and the variable.

int main( int argc, char *argv??(??)) 
{ 
   char * debug = "Not specified"; 
   char * type   = "Not specified"; 
   typedef struct getStr 
   { 
      char * name; 
      char ** value; 
   } getStr; 
   getStr lookUpStr[] = { 
       {"-debug", &debug     }, 
       {"-type",  &type       }, 
       {(char *) -1,  0} 
      }; 
  int iArg; 
  for (iArg = 1;iArg< argc; iArg ++   ) 
  { 
   int found = 0; 
   getStr * pGetStr =&lookUpStr[0];
   // iterate over the options with string values
   for (; pGetStr -> name != (char *)  -1; pGetStr ++) 
   { 
     // look for the arguement in the table
     if (strcmp(pGetStr ->name, argv[iArg]) == 0) 
     { 
       found = 1; 
       iArg ++; 
       if (iArg < argc) // if there are enough parameters
                        // so save the pointer to the data
        *( pGetStr -> value)= argv[iArg] ; 
       else 
         printf("Missing value for %s\n", argv[iArg]);       
       break;  // skip the rest of the table
     }  // if (strcmp(pGetStr ->name, argv[iArg]) == 0) 
     if (found > 0) break; 
    } // for (; pGetStr -> name != (char *)  -1; pGetStr ++) 
   
   if (found == 0) 
   // iterate over the options with int values 
   ....
  } 
  printf("Debug %s\n",debug); 
  printf("Type  %s\n",type ); 
  return 0; 
}   

This can be extended so you have

getStr lookUpStr[] = { 
    {"-debug", &debug, "char" }, 
    {"-type",  &type ,"int"       }, 
    {(char *) -1,  0, 0} 
   }; 

and have logic like

if (strcmp(pGetStr ->name, argv[iArg]) == 0) 
     { 
       found = 1; 
       iArg ++; 
       if (iArg < argc) // if there are enough parmameters
       {
       if ((strcmp(pGetStr -> type, "char") == 0 
        *( pGetStr -> value)= argv[iArg] ; 
       else 
        if ((strcmp(pGetStr -> type, "int ") == 0 )
        *( pGetStr -> value)= atoi(argv[iArg]) ;
      ...   
     }

You can go further and have a function pointer

getStr lookUpStr[] = { 
    {"-debug", &debug,myint }, 
    {"-loop", &loop  ,myint },  
    {"-type",  &type , mystring  }, 
    {"-type",  &type , myspecial  }, 
    {(char *) -1,  0, 0} 
   };f

and you have a little function for each option. The function “myspecial(argv[iarg])” looked up values {“approved”, “rejected”…} etc and returned a number representation of the data.

This takes a bit more work to set up, but over all is cleaner and clearer.

What’s the date in ‘n’ days time?

I needed to see if a certificate is due to expire within “n” days. How do I find this date? It turns out to be pretty easy using standard C functions.

                                                                          
#include <stdio.h> 
#include <time.h> 
int main( int argc, char *argv??(??)) 
{ 
....
    char expireDate[11]; 
    time_t t1, t3; 
    struct tm *t2; 
    t1 = time(NULL); 
    t2 = localtime(&t1); 
    t2 -> tm_mday += 40 ; // 40 days from now 
    t3 = mktime(t2); 
    int s; 
    s=  strftime( expireDate, 11, "%Y/%m/%d", t2  ); 
    printf("====size  %d================\n",s); 
    printf(".%10.10s\n",expireDate); 

This successfully printed out the date 40 days, from now. The only little problem I had, was with strftime. The size of the output is 10 bytes. The “11” specifies the maximum number of characters that can be copied into the array. If this was 10… the size of the data I was expecting, The output was wrong “. 2023/06/1” ; off by one character in the buffer and a leading blank.!

With the technique of changing the value within a tm structure you can get the date-time n seconds / m minutes / d days/ from now either in the future – or in the past.

Clever stuff !

Is car marketing poor – or am I weird ?

Over the last few weeks I’ve bought a new car. I was very surprised at how bad the experience was. I wondered if my experience was common – or was it just me.

The brochure

For the sorts of cars I was interested in, you cannot just pick up a brochure at the garage. Car manufacturers have moved with the times – the brochures are now online and you have to download a PDF. I wondered if the sales and marketing teams for the motor manufacturers ever used their own products.

Trying to look at a brochure on my mobile phone was impractical. Using a tablet was a bit better, but the quality of the image was not good enough to be able to read the small print. “Is the size 38 or 36 cm?” I could not tell as the quality of the image was not good enough.

The brochure contents

Someone once said to me “for every presentation there are two aspects. One is what information you as the speaker want to present, the other aspect is what the audience wants to hear”. For example for a talk on Greece, I would go along expecting to hear about Archimedes and the dawn of science, but the speaker might want to show us photographs of the beaches (and bars) when he went to on holiday.

In a similar way, I want to get useful information about the car. Having pictures of the Scottish highlands, or driving through a city with no traffic. I’ve seen these places before (and how often do you drive through a city when there is no other cars in sight). Perhaps there is a subtle message. People who drive these cars have this life style. If so, the message was lost on me.

My first question was which brand of car to buy, the second was having decided on a brand, which model with brand do I want

I created a spread sheet of my requirements, “nice to have”s, and other factors. Then I tried to extract the information from each brochure in a format so it can be compared with other cars.

My requirements included

  • Fuel type petrol
  • Height under 2.2 meters. If the height is more than this you have to pay more to take it onto a ferry
  • Length and width
  • Size of boot. This is usually given in litres. Today when we go on holiday we can easily stack our cases in the boot. Can we do this – or is the boot a different shape?
  • Electric windows, air con, CD player or equivalent.
  • 6 speed gear box.

For my existing car, there were no online brochures, so I had to get my tape measure and collect the size information myself.

I eliminated brands of cars, then I had to decide which models met my criteria. My old car had a 1.4 l engine, and had listed that as a minimum engine size requirement. I test drove a car with a 1.0 l engine size – and that felt like it had more power.

I got down to a choice of three cars. The deciding factor was the older car where you needed to put a key in the ignition – rather than keyless starting. This was a surprise to me, as it was not one of the factors I had listed!

It would have been great if I could enter details of my current (old) car, and ones I was interested in, and get a table of cars and their interesting ( to me) facilities.

It was also interesting how my views different from the sales person. I was told that to change the interior temperature, I bring up “car settings” on the screen, and swipe with my finger to make it hotter or cooler. My comment, was “So I have to look down at the screen and tap and swipe. So that means I need to look down for about 2 seconds. I prefer a knob I can touch, with no more than a glance, and can rotate it”. The sales person said I had a valid point.

Now I have my new car how does it work?

The instruction book has 178 pages – and covers all models. It covers 3 models of audio system – two of which look identical, I do not know how to tell them apart. The book covers how to start the engine with the key and without the key (for those cars with the technology). It has instructions like push that knob and pull that lever to engage the parking brake – except I do not have the knob.

What would be good (and bring the technology up to date) to be able to go to the manufacturer’s web site, enter the Vehicle Identification Number (which uniquely identifies the car) and you are given a list of web page instructions for the features that were installed in the factory. This would allow me to quickly skip the pages telling my why an electric car is good for the environment (but my car is petrol), and how to install a child seat (which I do not want).

What is your relationship with your car?

I remember doing a survey on the car I had, and cars I might buy in the future. There were questions like

  • Q:”What is your relationship with your car?”. A:”What!? – it is a car, I do not have a relationship with it”
  • Q:”Does your can have a name?” A:”No!”
  • Q:”What sort of people drive a xxxxx (a top range sports car)” A:”Bad drivers, who drive without due care and consideration for other people”.

Perhaps I am the strange one after all.

Easy question – hard answer, how to I convert a hex string to hex byte string in C?

I have a program which takes as input a hex string, and this needed to be converted to an internal format, specifically a DER encoded format ( also known as a TLV, Tag, Length, Value);

This took me a good couple of hours to get right, and I thought the solution would be worth passing on.

The problem is: I have a C program and I pass in a parameter -serial abcd123456. I want to create a hex string 0x02llabcd123456 where ll is 5 – the length of the data.

Read the parameter

for (iArg = 1;iArg< argc; iArg ++   ) 
{ 
   if (strcmp(argv[iArg],"-serial") == 0 
      && iArg +1 < argc) // we have a value 
   { 
      iArg ++; 
      char * pData = argv[iArg]; 
      int iLen = strlen(pDataz); 
      if ( iLen > 16 ) 
      { 
         printf("Serial is too long(%d) %s.\n",iLen,pData); 
         return 8; 
      } 
    ... process it.
}

The

if (strcmp(argv[iArg],"-serial") == 0 
      && iArg +1 < argc) // we have a value 

checks to see if -serial was specified, and there is a parameter following. It handles the case of passing “-serial” and no following parameter.

Convert it from hex string to internal hex

I looked at various ways of converting the character string to hex, and decided the internal C run time sscanf function was best. This is the opposite of printf. It takes a formatting string and converts from printable to internal format.

For example

sscanf(pData,”%ix”,&i);

Would processes the characters in the data pointed to by pData and covert them, treating hen as hex data, to an integer &i. The processing continues until a non valid hex character is met, or the integer value is full.

If the parameter was –serial AC, the output value would be 0x000000AC.

I initially tried this, but then I had to go along and ignore any leading zeros.

You can use

sscanf(pData,”%6hh”,&bs[0]);

To read up to 6 characters into the byte string bs. If the parameter was –serial AC, the output value would be 0xAC….

This is almost what I want – I want a left justified byte string. But I have a variable length, and cannot pass a length as part of the string.

I managed this using a combindation of sprintf and sscanf.

The final-ish solution

 
int len = strlen(pData); // get the length of passed value
char sscan[20];   // used for sscanf string 
// we need to covert an ebcdic string to hex, so 
//  "1" needs length of 1, 
//  "12" needs length of 1 
//  "123" needs length of 2 etc 
int lHex = (len + 1) /2; 
                                                       
// convert to %4ddx 
// create a string for the sscan with the length 
// as it needs a hard coded length 
sprintf(&sscan[0],"%%%dhhx\0", len);
// if len = 4 this creates "%4hhx"
char tempOutput[16];
// Now use it 
sscanf(pData,&sscan[0],&tempOutput[0]); 

and I have &tempOutput containing my data – of length lHex.

This worked until it didn’t work

This worked fine until a couple of hours later. If the hex value was 7F… or smaller it worked. If it was 80… or larger it did not work.

This is because of the way the DER format handles signed numbers.

The value 0x02036789ab says

  • 02 this is an integer field
  • 03 of length 03
  • with value 6789ab.

The value 0x0203Abcdef says

  • 02 this is an integer field
  • 03 of length 03
  • with negative value Abcdef – negative because the top bit of the number is a 1.

Special casing for negative numbers

I had to allows for this negative number effect.

For negative numbers, the output needs to be 0x020400abcdef which says

  • 02 this is an integer field
  • 04 of length 04
  • with value 00abcdef – positive because the top bit is zero.

pBuffer points to the output byte field.

 if (tempOutput[0] < 0x80) 
 { 
   memcpy(pBuffer+1,&temp[0],lHex); // move the data
   *pBuffer = lHex; // char value of the length 
 } 
 else // we need to insert extra so we do not get -ve 
 { 
   *(pBuffer+1) = 0x00; // insert extra null 
   memcpy(pBuffer+2,&temp[0],lHex); // and the rest 
   *pBuffer = lHex +1 ; // char value of the length 
 } 

The solution is easy with hindsight.

Certificate dates with ICSF and PKISERV

Digging into certificates provided by ICSF, I got confused with the dates.

There are 3 places that start/end dates can be, and they have different meanings and uses.

  1. The validity period you see in the certificate is part of the certificate itself. That is added at certificate creation and enforced by the application (such as System SSL).
  2. Within ICSF there are fields START DATE/END DATE you see in the panels are CKA_START_DATE and CKA_END_DATE. They are defined in the PKCS#11 standards but are not enforced.
  3. Within record metadata for a KDSR format record, you will see Cryptoperiod start date/Cryptoperiod end date. This is enforced by ICSF. Usage outside this time frame is not permitted.

Note that PKCS#11 services are the only place you can see all three of these. Neither CKDS nor PKDS can hold certificates, nor do they support PKCS#11 attributes.

Official document

The standards document PKCS #11 Cryptographic Token Interface Base Specification says

CKA_START_DATE – Start date for the certificate (default empty)
CKA_END_DATE – End date for the certificate (default empty

Section 4.6.2 (Certificate objects Overview):
The CKA_START_DATE and CKA_END_DATE attributes are for reference only; Cryptoki does not attach any special meaning to them. When present, the application is responsible to set them to values that match the certificate’s encoded “not before” and “not after” fields (if any).

Section 4.7.2 (Key Objects Overview) has similar wording:
Note that the CKA_START_DATE and CKA_END_DATE attributes are for reference only; Cryptoki does not attach any special meaning to them. In particular, it does not restrict usage of a key

Thanks to Eric Rossman, for helping me understand this.

How do I format an encoded name from a digital certificate?

I had a string CN=myserver,O=test,C=us in a certificate, which has been encoded

Offset   :                                       EBCDIC          ASCII 
00000000 : 302F310B 30090603 55040613 02757331  ................ 0/1.0...U....us1  
00000010 : 0D300B06 0355040A 13047465 73743111  ................ .0...U....test1.  
00000020 : 300F0603 55040313 086D7973 65727665  ........._`..... 0...U....myserve  
00000030 : 72                                  .                r                 
    

where you can see the encoded string in ASCII on the right. How do I extract the string CN=myserver,O=test,C=us from this?

This format is used for Subjects and Issuers (and other types).

The hex string has an ASN.1 encoding. It usually starts with a 0x30. See here for a good introduction to ASN.1 encoding.

  • 30 means this is a sequence
  • 2f is the length of the data following
  • etc

On z/OS you can use system ssl to decode this back to a printable string. You need two steps.

  1. Convert to internal format,
  2. Create the printable EBCDIC string from the internal format.

Step1 create internal format of the data

#pragma runopts(POSIX(ON)) 
#include <gskcms.h> 
#include <gskssl.h>  
void x509_to_string(char * pData, // pointer to the data
                     int lData)    // length of the data
{ 
   x509_name    X509;   // internal format.
   int rc; 
   gsk_buffer cert;     // intermediate format
   if (lData == 0) 
   { 
      printf("no data"); 
      return; 
   } 
   cert.length=   lData; 
   cert.data  =   pData ; 
   // convert from 0x302F310B 30090603 to internal 
   rc = gsk_decode_name( 
       & cert, 
       & X509); 
   if ( rc != 0) 
   { 
     //use gsk_strerror(rc) to return a string.
     printf("\nInternal error:gsk_decode_name %s\n", 
                  gsk_strerror(rc)); 
     return; 
   } 

Convert from internal format to an EBCDIC string.

   char * pName ;       // output value   
   // convert from internal to CN=COLIN,C=GB 
   rc = gsk_name_to_dn( 
              &X509, 
              &pName); 
   // free the intermediate value regardless of rc
   gsk_free_name(&X509); 
   if ( rc != 0) 
     { 
       printf("\nInternal error:gsk_name_dn %s\n", 
                    gsk_strerror(rc)); 
       return; 
     } 
   printf("%s.",pName); 

   //free the string 
   gsk_free_string(pName); 
} 

Convert from EBCDIC string to intermediate or certificate format.

There are the reverse functions.

  • gsk_dn_to_name( pName, &X509);
  • rc = gsk_encode_name( &X509, & cert);

Both C=GB,CN=colin and c=GB,cn=colin produce

00000000 : 301D310E 300C0603 55040313 05636F6C …………..?% 0.1.0…U….col
00000010 : 696E310B 30090603 55040613 024742 .>…………. in1.0…U….GB

JCL

I needed binder input options.

INCLUDE GSK(GSKCMS31) 
INCLUDE GSK(GSKSSL) 

and JCL

//COMPILE  EXEC PROC=EDCCB, 
//       LIBPRFX=&LIBPRFX, 
//       CPARM='OPTFILE(DD:SYSOPTF)', 
// BPARM='SIZE=(900K,124K),RENT,LIST,RMODE=ANY,AMODE=31,AC=1' 
//COMPILE.SYSLIB DD DISP=SHR,DSN=&LIBPRFX..SCEEH.SYS.H 
//               DD DISP=SHR,DSN=&LIBPRFX..SCEEH.H 
//               DD DISP=SHR,DSN=SYS1.SIEAHDR.H 
...
//COMPILE.SYSIN  DD DSN=&SOURCE(&PROG),DISP=SHR 
//COMPILE.SYS    DD DISP=SHR,DSN=&LIBPRFX..SCEEH.SYS.H 
//COMPILE.SYSOPTF DD DISP=SHR,DSN=&SOURCE(CCOPTS) 
//* 
//* Bind the module 
//* 
//BIND.SYSLMOD DD DISP=SHR,DSN=&LOADLIB(&PROG) 
//BIND.SYSLIB  DD DISP=SHR,DSN=&LIBPRFX..SCEELKED 
//             DD DISP=SHR,DSN=SYS1.SIEALNKE 
//BIND.GSK     DD DISP=SHR,DSN=SYS1.SIEALNKE 
...
//BIND.SYSIN DD DISP=SHR,DSN=&SOURCE(&BINDOPTS) 

Where the gskcms.h and gskssl.h are in DSN=SYS1.SIEAHDR.H or /usr/include/, and the binder stubs are in DSN=SYS1.SIEALNKE.

What WLM classification does z/OSMF need?

I was asked this question; I asked around and Ive summarized the responses below.

z/OSMF has two address spaces. The Angel task and the server task

The Angel task

Once the Angel task has started it uses very little CPU. It provides cross memory services for z/OSMF.

If your default service class for this is SYSSTC, then this will be fine.

The server

This is a more interesting challenge. This address space runs all of the work. It is written in Java. On my system it takes a lot of CPU to start up, but once it is up, it only uses CPU when there is work to do. This work tends to be as a result of a web browser or REST requests, and is not usually mission critical.

Running this as with a service class of Discretionary. May be OK, but as z/OSMF is used by systems programmers, they may give it a little extra boost.

If you are using z/OSMF to run and monitor your system, you may want to put it in a service class with non Discretionary, so that it continues to run even it your system is constrained for CPU.

Personally I would use TSO and ISPF in preference to z/OSMF, but new people to z/OS are familiar with Web Server based products rather than TSO.

I asked about this problem on the IBM Mainframe mailing list,and got some responses.

Kolusu Sri said

IZUSVR1 uses the setting from IZUPRMxx parmlib member, so you can add WLM_CLASSES in there.

On my system we have

WLM_CLASSES DEFAULT(IZUGHTTP)    

            LONG_WORK(IZUGWORK)  

you also need racf permissions

SEC_GROUPS USER(IZUUSER),ADMIN(IZUADMIN),SECADMIN(IZUSECAD)

SESSION_EXPIRE(495)

The IZUGHTTP service class is for foreground z/OSMF transactions (except asynchronous beans).

The IZUGWORK transaction class is for long running asynchronous beans and for unexpected z/OSMF work, not otherwise classified. Asynchronous beans are used in some scenarios of the Software Management plugin.

See z/OSMF V2.1 Resource Requirements section WLM Classification for z/OSMF on page 11. For more information.

                                        

Mike Shorkend said

Also keep in mind that a large proportion of IZUSVR1 CPU consumption is ZIIP eligible. I prefer to send ZIIP heavy workloads to a dedicated service class (mixing workloads might cause a ZIIP eligible workload to get denied ZIIP because the service class it is associated with is meeting its goals because of other non-ZIIP workloads). On a busy development LPAR I use a service class with importance 5 and execution velocity of 30 and it performs well. Also keep in mind the HONORPRIORITY setting for the service class which can cause/prevent  spill over of ZIIP eligible work to general usage processors.

Setting up Linux to z/OS certificates

Several times I have had to set up certificates between Linux and z/OS and struggled for a day to get them working. Once you are familiar with doing it – it is easy. As the last time I needed to do this was over a year ago, I’ve forgotten some of the details. This blog post is to help me remember what I need to do, and to help other who struggle with this.

I’m ignoring self signed.

Basic TLS

A certificate contains

  • who it belongs to, such as CN=COLIN,O=SSS
  • the date range the certificate is valid
  • a public key
  • meta data about the key: What algorithm does the public key use, what parameters were used in the key generation, for example, algorithm=RSA, Keysize=2048.

There is a private key.

  • If you encrypt using the private key, you can use the public key to decrypt it.
  • If you encrypt using the public key, you can use the private key to decrypt it.
  • If you encrypt something with my public key, and then encrypt it with your private key. I know it came from you (or someone with your private key) and only I (or someone with my private key) can decrypt it.

Anyone can have the public key. You keep the private key secure.

Certificate Authority. This is used in validating the trust of certificates. You send your certificate to the CA, The CA does a checksum of your data, and encrypts this checksum with the CA private key. It returns your original data appended with the encrypted checksum, and information about the CA, and what was used to calculate the checksum. If someone else has the CA public key, they can do the opposite process. Do the checksum calculation, and decrypt the checksum value in the certificate, using the CA public key. If they match you know it was signed by the CA. This is known as signing the certificate.

To be able to validate a certificate sent to it, the client end needs the CA of the server end. The server needs the CA of the client end to be able to validate the client’s certificate.

During the handshake to establish the TLS connection there is a flow like

  • Establish the cipher spec to use
  • Server sends down its certificate, the client checks it
  • Servers sends down “Certificate request”, and these are the certificate(CAs) I know about
  • The client goes through it’s list of certificates (usually only one), to find the first certificate with a CA in the list sent from the server.
  • sends the client certificate to the server
    • The server checks the certificate. For example the server may be set up to accept a subset of valid algorithms, for example TLS 1.2, and Elliptic Curve. If a certificate is sent up using RSA, then this is not accepted
    • The server checks the signature of the certificate, finds the CA name, checks in the trust store for this CA, and validates the signature. Depending on the application it may check all the CA’s in the CA chain.

What do you need for the handshake to work

  • You need to have a Certificate Authority to sign certificates. In the CA certificate are some flags that say this is a CA.
  • You need to send the public key of each CA to the other end. You normally need to do this just once, and keep using the same certificates for all your TLS work.
  • You need to have a key store/trust store/keyring to hold certificates.
  • On z/OS
    • you may have a keyring for different projects, for example MQ, and TN3270.
    • You need to connect the client CA into each keyring where it will be used.
  • You need to check that the certificates are compatible with the remote end, such as Algorithm etc.

Openssl files

When using openssl, you can store common information in a configuration file. See here. This configuration file has some required options, and some optional options where you can specify common options you frequently use.

If you are using the openssl req command (for example), by default it will look for a section called [req]. This can in turn point to other sections. Using this file you can specify most of your fields in one place, and just override the specific ones.

Create a CA certificate on Linux

I have a bash file docca.sh file on Linux.

CA=”docca256″
casubj=” -subj /C=GB/O=DOC/OU=CA/CN=SSCA256″
days=”-days 1095″
rm $CA.pem $CA.key.pem

openssl ecparam -name prime256v1 -genkey -noout -out $CA.key.pem1

openssl req -x509 -sha384 -config caca.config -key $CA.key.pem2 -keyform pem -nodes $casubj -out $CA.pem3 -outform PEM $days

openssl x509 -in $CA.pem -text -noout|less4

This

  1. creates a private key (docca256.key.pem)
  2. self signs it. For any parameters not specified, it uses the configuration file caca.config and section “req” (signing request) within it.
  3. produces a public certificate in docca256.pem. This file will need to be sent to the backend servers. You can use cut and paste or FTP as ASCII.
  4. displays the x509 data

The caca.config file has

[ req ]
distinguished_name = ca_distinguished_name
x509_extensions = ca_extensions

prompt = no

authorityKeyIdentifier = keyid:always,issuer:always

[ca_distinguished_name ]
# C=GB
# O=DOC
# OU=Stromness
# CN=SSSCA4

####################################
[ ca_extensions ]

subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always
basicConstraints = critical,CA:TRUE, pathlen:0
keyUsage = keyCertSign, digitalSignature,cRLSign

The distinguished_name = ca_distinguished_name says go and look in the file for a section [ca_distinguished_name], and x509_extensions = ca_extensions says go and look for a section called [ca_extensions]. You can specify your own names, for example I could have used section1, and s2.

When prompt = yes, openssl takes as defaults the values in the distinguished_name section. When prompt = no, the distinguished_name is still required – but the contents of the section are ignored.

The values in the x509_extensions are defined here.

Creating an Elliptic Curve certificate on Linux

I used another bash script docecadad.sh, to document an ElliptiCal certificate for userid ADCD. It uses the CA defined above.

name="docecadcd"
key="$name.key.pem"
cert="$name.pem"
subj="-subj /C=GB/O=Doc/CN="$name
CA="docca256"
cafiles="-cert $CA.pem -keyfile $CA.key.pem "

enddate="-enddate 20240130164600Z"
passin="-passin file:password.file"
passout="-passout file:password.file"

rm $name.key.pem
rm $name.csr
rm $name.pem

#define a certificate with elliptic key with size 256

openssl ecparam -name prime256v1 -genkey -noout -out $name.key.pem 
#create a certificate request (ie hello CA please sign this)
openssl req -config openssl.config -new -key $key -out $name.csr -outform PEM -$subj $passin $passout

# sign it.

caconfig="-config ca2.config"
policy="-policy signing_policy"
extensions="-extensions clientServer"

md="-md sha384"

openssl ca $caconfig $policy $md $cafiles -out $cert -in $name.csr $enddate $extensions

# display it 
openssl x509 -in $name.pem -text -noout|less

Where the openssl.config file has

[ req ]
default_bits       = 2048

distinguished_name = server_distinguished_name
req_extensions     = server_req_extensions
string_mask        = utf8only
subjectKeyIdentifier   = hash
#extendedKeyUsage     = critical, codeSigning


[ server_req_extensions ]

subjectKeyIdentifier = hash
# subjectAltName       = DNS:localhost, IP:127.0.0.1, IP:127.0.0.6
# nsComment            = "OpenSSL"
keyUsage             = critical, nonRepudiation, digitalSignature
# extendedKeyUsage     = critical, OCSPSigning, codeSigning
subjectKeyIdentifier   = hash 

[ server_distinguished_name ]
#c=GB
#o=SSS
#cn=mqweb
  • See above for the distinguished_name value.
  • req_extensions says use the section [server_req_extensions]

The ca2.config file used to sign it has

HOME            = .
RANDFILE        = $ENV::HOME/.rnd

####################################################################
[ ca ]
default_ca    = CA_default      # The default ca section
####################################################################
[ CA_default ]
default_days     = 1000         # How long to certify for
default_crl_days = 30           # How long before next CRL
#default_md       = sha1       # Use public key default MD
default_md       = sha256       # Use public key default MD
preserve         = no           # Keep passed DN ordering

x509_extensions = ca_extensions # The extensions to add to the cert

email_in_dn     = no            # Don't concat the email in the DN
copy_extensions = copy          # Required to copy SANs from CSR to cert

#defaults
base_dir      = .
certificate   = $base_dir/cacert.pem   # The CA certifcate
private_key   = $base_dir/cakey.pem    # The CA private key
new_certs_dir = $base_dir              # Location for new certs after signing
database      = $base_dir/index.txt    # Database index file
serial        = $base_dir/serial.txt   # The current serial number

unique_subject = no  # Set to 'no' to allow creation of
                     # several certificates with same subject.


####################################################################
[ ca_extensions ]

subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer:always
basicConstraints       = critical,CA:TRUE, pathlen:0
keyUsage               = nonRepudiation

####################################################################

[ signing_policy ]
countryName            = optional
stateOrProvinceName    = optional
localityName           = optional
organizationName       = optional
organizationalUnitName = optional
commonName             = supplied

[ clientServer ]

keyUsage               = digitalSignature, keyAgreement, digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName         = DNS:localhost, IP:127.0.0.1, 
extendedKeyUsage       = serverAuth,clientAuth
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer:always
nsComment  = "clientserver"

The policy (my [ signing_policy] ) must have entries in it to create a valid Subject Distinguished name. Without it, I got a strange RACF code (0x0be8044d).

Send the CA to z/OS and import it

You need to send the CA public certificate to z/OS. This file looks like

-----BEGIN CERTIFICATE-----                                      
MIIFbTCCA1WgAwIBAgIUJw+gLLSFxqCyTdIyEUWyQ/g9JnEwDQYJKoZIhvcNAQEL.
...
-----END CERTIFICATE----- 

You can FTP the file, or create the file and use cut and paste. The file needs to be a sequential dataset with format VB. My file is VB, lrecl=256,blksize=6233. For the FTP I used

put docca256.pem ‘colin.docca256.pem’

You need to import this into RACF, and connect it to the keyrings.

//IBMRACF  JOB 1,MSGCLASS=H                                     
//S1  EXEC PGM=IKJEFT01,REGION=0M                               
//SYSPRINT DD SYSOUT=*                                          
//SYSTSPRT DD SYSOUT=*                                          
//SYSTSIN DD *                                                  
RACDCERT CHECKCERT('COLIN.DOCCA256.PEM') 
/*                       

The CHECKCERT gave me

Certificate 1:                                                          
                                                                        
  Start Date: 2022/10/09 11:45:43                                       
  End Date:   2025/10/08 11:45:43                                       
  Serial Number:                                                        
       >782A62948699FF3FB00238FB296E4A647B7DF07C<                       
  Issuer's Name:                                                        
       >CN=SSCA256.OU=CA.O=DOC.C=GB<                                    
  Subject's Name:                                                       
       >CN=SSCA256.OU=CA.O=DOC.C=GB<                                    
  Signing Algorithm: sha384ECDSA                                        
  Key Usage: HANDSHAKE, CERTSIGN                                        
  Key Type: NIST ECC                                                    
  Key Size: 256                                                         
                                                                        

Which matches what I expected, and gave me information about the certificate – ECC, 256, and signed with SHA 384 ECDSA, (from the -sha384 parameter above).

Define it to RACF and connect it to the user’s keyring

racdcert list  (label('Linux-CA256')) CERTAUTh 
RACDCERT DELETE  (LABEL('LINUXDOCCA256'))    CERTAUTH             
              
RACDCERT ADD('COLIN.DOCCA256.PEM') -                            
          CERTAUTH  WITHLABEL('LINUXDOCA256') TRUST 

RACDCERT ID(START1) CONNECT(RING(MQRING)-        
                            CERTAUTH     -       
                            LABEL('LINUXDOCCA256'))                     

If you delete a certificate, then it is removed from all keyrings. Once you have re-added it you need to reconnect it to all the keyrings. If you list the label (racdcert list (label(‘Linux-CA256’)) certauth) it will display where it is used, so you can read it.

Download the z/OS CA certificate

I downloaded the z/OS exported certificate in .pem format. it looks like

-----BEGIN CERTIFICATE-----                                       
MIIDYDCCAkigAwIBAgIBADANBgkqhkiG9w0BAQsFADAwMQ4wDAYDVQQKEwVDT0xJ  
... 
+9TRng==                                                          
-----END CERTIFICATE-----

You can use ftp or cut and paste. I created doczosca.pem.

Use it!

Before I could use it, I had to set up the server’s certificate, and download the z/OS CA certificate

I set up a bash script scl2.sh

name=docecadcd
host="-connect 10.1.1.2:4000"
CA="--CAfile doczosca.pem "

openssl s_client $host  $CA -cert $name.pem -certform PEM -key $name.key.pem -keyform PEM
 

Setting up z/OS for TLS clients

There is a lot of configuration needed when setting up TLS(SSL) between a server and a client. There are many options and it is easy to misconfigure. The diagnostic information you get when the TLS handshake fails is usually insufficient to identify any problems.

You need the following on z/OS:

  • One or more Certificate Authority certificates. You can create and use your own for testing. If you want to work with external sites you’ll need a proper (external) CA, but for validation and proof of concept you can create your own CA. You could set up a top level CA CN=CA,O=MYORG, and another one (signed by CA=CA,O=MYORG), called CN=CA,OU=TEST,O=MYORG. Either or both of the public CA certificates will need to be sent to the clients in imported into their keystore.
  • A private/public key, signed by a CA, (such as signed by CA=CA,OU=TEST,O=MYORG).
  • The private key is associated with a userid.
    • The signing operation takes the data (the public key), does a hash sum calculation on the data, encrypts this hash sum, and stores the encrypted hash value, and CA public certificate with the original data. To check the signature, the receiving application compares the CA with its local copy, if that matches, does the same checksum calculation, decodes the encrypted hash sum – and checks the decrypted and locally calculated values match.
    • A certificate is created using one from a list of algorithms. (For example, Elliptic Curve, RSA). When the certificate is sent to the client, the client needs to support the algorithm. Either end can be configured, for example, to support Elliptic Curve, but not RSA.
  • A keyring to contain your private key(s) – this can also contain CA public certificates of the partners (clients or servers).
  • A “site” keyring (public keystore, or trust ring) which holds the public CA certificates of all the other sites you work with. If you have only one keyring per user or application, you need to update each of them if you need to an a new CA to your environment. Many applications are only designed to work with one keyring. Java applications tend to have a key store(for the private key) and a trust store for the CAs.
  • Some applications can support more than one private certificate on a keyring. The certificate needs to match what the client can support.
  • For certificates which are sent to your server, you need a copy of the CA(s) used to sign the incoming certificate. If you have a copy of the CA, then you can validate any certificate that the CA signed. This means you do not have to have a copy of the public certificate of every client. You just need the CA.
    • Some application need access to just one CA in the chain, other applications need access to all certificates in the CA chain.
  • As part of the TLS handshake
    • the client sends up a list of the valid cipher specs it supports (which algorithms, and size of key)
    • the server sends down a subset of the list of cipher spec to use (from the client’s list)
    • the server can also send down its certificate, which contains information such as the distinguished name CN=zSERVER, OU=TEST, O= MYORG, and host name.
    • the client can validate these names – to make sure the host name in the certificate matches the host, and what it was expecting.
    • if requested, the client can send up its certificate for identification. The server can validate the certificate, and can optionally map it to a userid on the server.
  • A userid can be given permission to read certificate in another user’s keyring. A userid needs a higher level of authority to be able to access the private key in another id’s keyring.

Create the Certificate Authority

//IBMRACF  JOB 1,MSGCLASS=H                               
//S1  EXEC PGM=IKJEFT01,REGION=0M                         
//SYSPRINT DD SYSOUT=*                                    
//SYSTSPRT DD SYSOUT=*                                    
//SYSTSIN DD * 
RACDCERT certauth LIST(label('DOCZOSCA')) 
RACDCERT CERTAUTH DELETE(LABEL('DOCZOSCA'))               
RACDCERT GENCERT  -                                         
  CERTAUTH -                                                
  SUBJECTSDN(CN('DocZosCA')- 
             O('COLIN') -                                   
             OU('CA')) - 
  NOTAFTER(   DATE(2027-07-02  ))-                          
  KEYUSAGE(   CERTSIGN )  -      
  SIZE(2048) -                                              
  WITHLABEL('DOCZOSCA') 
/*
//                 

This certificate is created against “user” CERTAUTH. Keyusage CERTSIGN means it can be used to sign certificates. “user” CERTAUTH is often displayed internally as “irrcerta”.

Once it has been created the certificate should be connected to every ring that may use it, see below.

Export the CA certificate to a file so, clients can access it

RACDCERT CERTAUTH EXPORT(LABEL('DOCZOSCA')) -
  DSN('IBMUSER.CERT.DOC.CA.PEM') -
  FORMAT(CERTB64) -
  PASSWORD('password')

The file looks like

-----BEGIN CERTIFICATE-----                                        
MIIDYDCCAkigAwIBAgIBADANBgkqhkiG9w0BAQsFADAwMQ4wDAYDVQQKEwVDT0xJ   
TjELMAkGA1UECxMCQ0ExETAPBgNVBAMTCERvY1pvc0NBMB4XDTIyMTAwOTAwMDAw 
...  

This can be sent to the clients, so they can validate certificates sent from the server. This file could be sent using cut and paste, or FTP.

Create the keyring for user START1.

The instructions below lists the ring first – in case you need to know what it was before you deleted it”

RACDCERT LISTRING(TN3270)  ID(START1) 

RACDCERT DELRING(TN3270) ID(START1) 

RACDCERT ADDRING(TN3270) ID(START1)                                                          

RACDCERT LISTRING(TN3270)  ID(START1) 
SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh 

Connect the CA to every keyring that needs to use it

RACDCERT ID(START1) CONNECT(RING(TN3270) - 
                            CERTAUTH LABEL('DOCZOSCA'))

Create a user certificate and sign it on z/OS

This creates a certificate and gets is signed – as one operation. You can create a certificate, export it, sent it off to a remote CA, import it, and add it to a userid.

RACDCERT ID(START1) DELETE(LABEL('NISTECC521')) 
                                                                
RACDCERT ID(START1) GENCERT -                                   
  SUBJECTSDN(CN('10.1.1.2') - 
             O('NISTECC521') -                                  
             OU('SSS')) -                                       
   ALTNAME(IP(10.1.1.2))-                                       
   NISTECC - 
   KEYUSAGE(HANDSHAKE) - 
   SIZE(521) - 
   SIGNWITH (CERTAUTH LABEL('DOCZOSCA')) -                      
   WITHLABEL('NISTECC521')     -                                
                                                                
RACDCERT id(START1) ALTER(LABEL('NISTECC521'))TRUST             

RACDCERT ID(START1) CONNECT(RING(TN3270) -                      
                            ID(START1)  -                       
                            DEFAULT  - 
                            LABEL('NISTECC521') )               
SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh                    
RACDCERT LIST(LABEL('NISTECC521' )) ID(START1)                  
RACDCERT LISTRING(TN3270)  ID(START1)                           

This creates a certificate with type Elliptic Curve (NISTECC) with a key size of 521. It is signed with the CA certificate created above.

The ALTNAME, is a field the client can verify that the Source Name in the certificate matches the IP address of the connection.

It is connected to the user’s keyring as the DEFAULT. The default certificate is used if the label of a certificate is not specified when using the keyring.

Give a user access to the keyring

PERMIT START1.TN3270.LST CLASS(RDATALIB)  -    
    ID(COLIN )  ACCESS(UPDATE )                          
SETROPTS RACLIST(RDATALIB) refresh                       
  • Update access give userid COLIN access to the private key.
  • Read access only gives access to the public keys in the ring.

You would typically give a group of userids access, not just to individual userids.

Import the client’s CA’s used to sign the client certificates

This is the opposite to Export the CA certificate to a file so clients can access it above.

Copy the certificate to z/OS. This can be done using FTP or cut and paste.

Use it!

I used it in AT-TLS

TTLSConnectionAdvancedParms       TNCOonAdvParms 
{ 
 ServerCertificateLabel  NISTECC521
 ...
} 
TTLSSignatureParms                TNESigParms 
{ 
   CLientECurves Any 
} 
TTLSEnvironmentAction                 TNEA 
{ 
  HandshakeRole                       ServerWithClientAuth 
  TTLSKeyringParms 
  { 
    Keyring                   start1/TN3270 
  } 
...
} 

Changing Wi-Fi frequency on Ubuntu

The Wi-Fi on two computers was giving different performance. I wondered if this was because it was using a different frequency Wi-Fi . I found this article useful on Wi-Fi. It says the higher the frequency used, the better the performance.

Display information about what the Wi-Fi is currently using.

The iwconfig command gave

wlp4s0    IEEE 802.11  ESSID:"BTHub6-78RQ"  
          Mode:Managed  Frequency:5.24 GHz  Access Point: 94:6A:B0:85:54:AA   
          Bit Rate=585.1 Mb/s   Tx-Power=20 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on
          Link Quality=41/70  Signal level=-69 dBm  
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:32   Missed beacon:0

Which shows the frequency (5.24 GHz) and bit rate (585.1 Mega bits /second (not bytes).

On the slower computer on the same Wi-Fi network it had 2.412 GHz.

Display information about the wireless interface

The iwlist command displays information about the wireless interface and its capabilities

iwlist wlp4s0 frequency gave

wlp4s0    32 channels in total; available frequencies :
          Channel 01 : 2.412 GHz
          Channel 02 : 2.417 GHz
          Channel 03 : 2.422 GHz
...       
          Channel 48 : 5.24 GHz
...
          Channel 132 : 5.66 GHz
          Channel 136 : 5.68 GHz
          Channel 140 : 5.7 GHz
          Current Frequency:5.24 GHz (Channel 48)

On my other machine it only listed 2.412 to 2,484 GHz.

What frequency is available?

Displaying my BT hub (http://192.168.1.254/basic) it displayed 2.4 and 5 GHz and channels 1 and 48. Thus I can use frequencies 2.412 and 5.24 GHz

Change the frequency being used

sudo iwconfig wlp4s0 freq 2.412G

Turn Wi-Fi off and on – and it picks up the changed value.