Spice up a C program by using __asm__ to include inline assembler.

The C compiler on z/OS has an extension which allows you to put assembler code inline within a C program. This function can be useful, for example accessing z/OS macros. __asm__ is very badly documented in the publications, but this post gives a good overview.

Overall the use of __asm__ works, but you have to be careful. For small bits of assembler it was quicker to use __asm__ instead of creating a small assembler program and linking that with the C program.

This blog post document some of my experiences.

Using and compiling the code

You put code in __asm__(…); , _asm(..); or asm(..); . I think these are all the same.

To use macros or copy files within the code you need the ASMLIB statement in your JCL.

//S1          JCLLIB ORDER=CBC.SCCNPRC 
//STEP1    EXEC PROC=EDCC,INFILE='COLIN.C.SOURCE(ASM)', 
//         LNGPRFX='CBC',LIBPRFX='CEE', 
//         CPARM='OPTFILE(DD:SYSOPTF)' 
//COMPILE.ASMLIB DD DSN=SYS1.MACLIB,DISP=SHR 

Basic function

The asm() instruction has the following parts

  • asm(
  • “a string of source which can contain %[symbolname] “. Each line of assembler has “\n” at the end of the line.
  • the output code will be formatted to conform to normal HLASM layout standards.
  • “:” a list of output symbols and their mapping to C variables.
  • “:” a list of symbols used as input and their mapping to C variable names.
  • “:” a list of register that may have been changed (clobbered) by this code, for example “r14” and “r15”.
  • “);”

Example code

__asm__(
“*2345678901234567890xxxxxxxxxxxxxxxxxxxxxxxx\n”
” WTO ‘%[PARMS]’ \n”
:
:
[PARMS]”s”(“zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz”)
: “r0”, “r1”, “r14”, “r15”
);

The PARMS statement is a string with a value ZZZZZ… It is used in the WTO ‘%[PARMS]\n’ statement.

Long statements – wrapping and continuation

The generated code from the above statement is

*2345678901234567890xxxxxxxxxxxxxxxxxxxxxxxx                             000023  
         WTO   'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzX 000023  
               zzz'                                                      000023  

We can see

  • the *234… starts in column 1
  • the WTO instruction is in column 10
  • because the string was very long, it has been split at column 71 and wrapped onto the next line at column 16.
  • A continuation character was inserted at column 72

This means you do not need to worry too much about the formatting of the data.

The code looks a bit buggy.

Making the WTO into an operand and a comment

__asm__(
“*2345678901234567890xxxxxxxxxxxxxxxxxxxxxxxx\n”
” WTO abc ‘%[PARMS]’ \n”
:
:
[PARMS]”s”(“zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz”)
: “r0”, “r1”, “r14”, “r15”
);

Gives a warning message

*2345678901234567890xxxxxxxxxxxxxxxxxxxxxxxx                            
         WTO   abc                     'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzX
               zzzzzzzzzzzzzzzzzzzzzzzzzzz' 

ASMA432W Continuation statement may be in error - comma omitted from continued statement.                            

What are the __asm__ parameters?

The first parameter is a C string containing the assembler instructions. Each line ends with a “\n”. You specify substitution variables using %[name] where the name is defined later.

You can use multiple lines, for example

asm(
” LA R2,%[p1] \n”
” TIMEUSED LINKAGE=SYSTEM,CPU=MIC,STORADR=(2) \n”
:

The C compiler treats the “…” “…” as one long literal constant.

There are three sets of values between the delimiters “:” “:” “:”

  • Output variables
  • Input variables
  • General Registers which your code changes

A C variable can be used for

  • Output only. This is in the output section. This has a definition with “=”. For example [p1] “=m”(pASCB). asm() may generate code to load load the value before use
  • Input and output. This is in the output section. This has a definition with “+”. For example [i1] “+m”(data), asm() may generate code to load the value before use, and store it afterwards.
  • Input only. This is in the input section. It does not have character with its definition. For example [i2] “m”(data)
  • Dummy – but used as a register. If you specify you want a register (let asm() which register), it needs a variable either to load, or to store depending if it is write or read or both. For example [rr] “=r”(val). I defined the C variable “val”.

Using C variables in the assembler code

There is a variety of different “types” of data, from memory, to offset. I could not see the difference between them. Some gave the same output. I tended just to use “m” for memory fields.

Use of variables

// this code gets the ASCB address from low core

long pASCB;
asm(” LLGT 1,548 \n”
” STG 1,%[p1] \n”
: [p1] “=m”(pASCB)
:
: “r1”
);

This (64 bit) code

  • Clears and loads register 1 with the value in address decimal 548. (The ASCB value) .
  • It stores register 1 in the the variable %[p1]
  • [p1] is defined as
    • “=” means this field is write only
    • m is a memory address
    • (pASCB) is the variable to use. The compiler replaces this with (in my case) the value 2248(4) – the address of the variable in format offset(base regiser).
  • There was no input data
  • Register r1 was “clobbered” (meaning it was changed in my assembler code).

Using constants is not quite what I expected.

printf(“ttime %ld\n”,data);
asm(” LA 1,%[i1] ccp\n”
” LA 1,%[i2] \n”
” LA 1,%[i3]\n”
:
: [i1] “i”(“999”),
[i2] “i”(998)
[i3] “i”(“=c\’ABCD\'”)
: “r1″,”r2”
);

Gives code

 LA    1,999
 LA    2,998
 LA    2,=c'ABCD'            

Using [i2] “i”(“COLIN”) gave

LA 2,COLIN
ASMA044E Undefined symbol – COLIN

Using generated registers – or not

You specify that you want a register allocated to you by using the type “r”.

int val = 40;
asm(” LLGT %[rr],548 pASCB \n”
STG %[rr],%[p1] ZZZZZ \n”
: [p1] “=m”(pASCB)
: [rr] “r”(val)
: “r1″,”r2”
);

The lack of a “=”, “+” or “&” in front of the “r” means read only use of the register, so load the register with the value before my code.

Produces

   LGF   r6,val(,r4,2248)   - This is generated                  
   LLGT  6,548                   pASCB           
   STG   6,2240(4)               ZZZZZ           

This code has been given register 6 to use

  • It loaded the value of val into it – because I had specified it in the list of input variables value.
  • Used the same register where-ever I had specified %[rr] .

When I had specified the register as an input/output register by

: [p1] “=m”(pASCB), [rr] “+r”(val)
:
: “r1″,”r2”

The “+” says it is read and written the output code was

     LGF      r6,val(,r4,2248)   Generated                      
     LLGT     6,548              My Code pASCB                
     STG      6,2240(4)          My Code ZZZZZ 
     LGR      r0,r6              Generated                    
     LGFR     r0,r0              Generated                    
     ST       r0,val(,r4,2248)   Generated                    

So there is code generated to load the register from val, and save the value of the last of my instruction in the variable val.

Personally, I do not think I would use the “r”, but would select my own register(s) and use them.

If I wanted to used C variables, I can specify those, and explicitly load and save them.

Some instructions do not work.

char buffer[256];

asm(
” MVC [%p1](4),548 pASCB \n”

: [p1] “=m”(buffer)
:
: “r1″,”r2”
);

This fails with

: [p1] “=m”(buffer)
CCN4454 Operand must be an lvalue.

You need to use

You need to use [p1] “=m”(buffer[0]) instead of (buffer). (But this is just normal C)

The instruction then fails because

MVC 2240(4)(4),548

Is not a valid instruction.

You need to use

char buffer[256];
asm(
” LA 1,%[p1] \n”
” MVC 0(4,1),548 pASCB \n”

:
[p1] “=m”(buffer[0])
:
: “r1″,”r2”
);

Which successfully generates

  LA r1,2240(r4,)
  MVC 0(4,r1),548

Using literals

You can use assembler literals in your code, for example

asm(
” LA 1,=C’ABCD’ \n”
:
:
: “r1”
);

This works. There is a section in the listing

Start of ASM Literals
         =C'ABCD'
End of ASM Literals

Using assembler macros

When you use a macro, you need to review the generated code, and make a note of the registers it uses, then update the “clobbers” list.

asm(
” LA 2,%[p1] \n”
” TIMEUSED LINKAGE=SYSTEM,CPU=MIC,STORADR=(2) \n”

:…

This used r14,r0,r15

There was an error

BNZ   *+8  
*** ASMA307E No active USING for operand *+8         

I had to use the following to get it to work.

long long CPUUSED;
asm(
” PUSH USING \n”
” BASR 3,0 \n”
” USING *,3 \n”

” LA 2,%[p1] \n”
” TIMEUSED LINKAGE=SYSTEM,CPU=MIC,STORADR=(2) \n”
” POP USING \n”
:
[p1] “=m”(CPUUSED)
:
: “r0″,”r1″,”r2”,“r3”,r14″,”r15″
);
printf(“TIMEUSED %ld\n”,CPUUSED);

Using assembler services from a (64 bit) C program.

I wanted to provide a Python started task on z/OS to respond to the operator Stop, and Modify commands (for example to pass commands to the program).

I wrote a Python extension in C, and got the basics working. I then wanted to extend this a bit more. I learned a lot about the interface from C to assembler, and some of the newer linkage instructions.

The sections in this post are

Some of the problems I had were subtle, and the documentation did not cover them.

C provides a run time facility called __console2 which provide a write to the console, and a read from the console.

The output from the write to the console using __console2 looks like

BPXM023I (COLIN) from console2

With the BPXM023I prefix, which I thought looked untidy and unnecessary.

To use the Modify operator command with __console2 you have to use a command like

F PYTTASK,APPL=’mydata’

Which feels wrong, as most of the rest of z/OS uses

F PYTTASK,mydata

This can be implemented using the QEDIT assembler interface.

The journey

I’ll break my journey into logical steps.

Information on programming the C to assembler interface

There is not a lot of good information available.

Calling an assembler routine from a C program.

Setting up the linkage

A 64 bit program uses the C XPLINK interface between C programs.

To use the traditional Assembler interface you need to use

#pragma linkage(QEDIT , OS)

rc = QEDIT( pMsg, 6);

C does not set the Variable Length parameter list bit (the high bit of the last parameter to ’80…) so you cannot use parameter lists with a variable length, and expect traditional applications to work. You could always pass a count of parameters, or build the parameter list yourself.

Register 1 pointed to a block of storage, for example to parameters

00000000 203920D8 00000050 082FE3A0

which is two 64 byte addresses, the address of the pMsg data, and the address of the fullword constant 6;

The C code invokes the routine by

LG r15,=V(QEDIT)(,…,…)
BALR r14,r15

even though use of BALR is deprecated.

Allocating variable storage

The z/OS services my assembler program used, need 31 bit storage.

I allocated this in my C program using

char * __ptr32 pMsg;
pMsg = (char *) __malloc31(1024);

I then passed this to my assembler routine.

Coding the assembler routine

Assembler Linkage

The basic linkage was

BSM 14,0
BAKR 14,0

…….
PR go back

This is where it started to get complicated. The BAKR… PR is a well documented and commonly used interface.

A Branch and StacK Register instruction BAKR 14,15 says branch to the address in register 15, save the value of register 14 as the return address, and save the registers and other status in the linkage stack. The code pointed to by register 15 is executed, and at the end, there is a Program Return (PR) instruction which loads registers from the linkage stack, and goes to the “return address”.

The Branch and Stack instruction BAKR 14,0 says do not branch, but save the status, and the return address. A subsequent PR instruction will go to where register 14 points to.

Unfortunately, with the BALR code in C, and the BAKR, PR does not work entirely.

You can be executing in a program with a 64 bit address instructions (such as 00000050 089790A0), in 24 or 31, or 64 bit mode.

  • In 64 bit mode, all the contents of a register are used to address the data.
  • In 31 bit mode only the bottom(right) half of the register are used to address the data – the top half is ignored
  • In 24 bit mode, only the bottom 24 bits of the register are used to address the data.

There are various instructions which change which mode the program is executing in.

When a BAKR, or BASSM ( Branch and Save, and set Mode) is used, the return address is changed to set the mode ( 24,31,64) as part of the saved data. When this address is used as a branch back – the save mode information is used to switch back to the original mode.

When BALR (or BASR) is used to branch to a routine, the return address is saved in Register 14. The mode information is not stored. When this address is used as a branch back – the “default mode” information (24 bit) is used to set the mode. This means the return tries to execute with a 24 bit address – and it fails.

You solve this problem by using a (BRANCH AND SET MODE) BSM 14,0 instruction. The value of 0 says do not branch, so this just updates register 14 with the state information. When BAKR is issued, the correct state is saved with it.

If you use the “correct” linkage you do not need to use BSM. It is only needed because the C code is using an out dated interface. It still uses this interface for compatibility with historically compiled programs.

Note: BSM 0,14 is also a common usage. It is the standard return instruction in a program entered by means of BRANCH AND SAVE AND SET MODE (BASSM) or a BRANCH AND SAVE (BAS). It means branch to the address in register 14, and set the appropriate AMODE, but do not save in the linkage stack.

Using 64 and 31 bit registers

Having grown up with 32 bit registers, it took a little while to understand the usage 64 bit registers.

In picture terms all registers are 64 bit, but you can have a piece of paper with a hole in it which only shows the right 32 bit part of it.

When using the full 64 bit registers the instructions have a G in them.

  • LR 2,3 copies the 32 bit value from register 3 into the right 32 bits of register 2
  • LGR 2,3 copies the value of the 64 bit register 3 into the 64 bit register 2

If there is a block of storage at TEST with content 12345678, ABCDEFG

  • R13 has 22222222 33333333
  • copy R13 into Reg 7. LGR R7,R13. R7 contains 22222222 33333333
  • L R7,TEST. 32 bit load of data into R7. R7 now has 22222222 12345678. This has loaded the visible “32 bit” (4 bytes) part of the register, leaving the rest unchanged.
  • LG R8,TEST. 64 bit load into Reg 8. R8 now has 12345678 ABCDEFG . The 8 bytes have been loaded.
  • “clear high R9” LLGTR R9,R9. R9 has 00000000 ……… See below.
  • L R9,TEST . 32 bit (4 bytes) load into R9. R9 now has 00000000 12345678

Before you use any register in 32 bit code you need to clear the top.

The Load Logical Thirty One Bits (LLGTR R1,R2) instruction, takes the “right hand part” of R2 and copies it to the right hand of R1, and sets the left hand part of R1 to 0. Or to rephrase it, LLGTR R2,R2 just clears the top of the register.

Using QEDIT to catch operator commands

QEDIT is an interface which allows an application to process operator commands start, modify and stop, on the address space.For example

  • S PYTASK,,,’STARTPARM’
  • f PYTASK,’lower case data’
  • p PYTASK

Internally the QEDIT code uses CIBs (Console Information Block) which have information about the operator action

(I think QEDIT comes from “editing”= removing the Queue of CIBs once they have been processed – so Q EDIT).

The interface provides an ECB for the application to wait on.

The documentation was ok, but could be clearer. For example the sample code is wrong.

In my case I wanted a Python thread which used console.get() to wait for the operator action and return the data. You then issue the console.get() to get the next operator action.

My program had logic like

  • Use the extract macro to get the address of the CIBS.
  • If this job is a started task, and it is the ‘get’ action then there will be a CIB with type=Started
    • Return the data to the requester
    • Remove the CIB from the chain
    • Return to the requester
  • Set the backlog of CIBs supported ( the number of operator requests which can be outstanding)
  • WAIT on the ECB
  • Return the data to the requester
  • Remove the CIB from the chain
  • Return to the requester

The action can be “Start”, “Modify”, or “Stop”

Can I use __ASM__ within C code to generate my own assembler?

In theory yes. The documentation is missing a lot of information. I could not get my simplest tests to work and return what I was expecting.

Why can’t I use this C function – it is there but I cannot see it.

I’ve been writing in C for over 20 years, and it is humbling when you suddenly realise how little you know of a topic.
It reminds me of when I worked for IBM, and the company wanted a skills register. The questions were along the lines of

Rate your skills in the following areas from 0 (nothing) to 10 (expert).

  • z/OS
  • DB2
  • CICS
  • etc

Overall the skills register was found to be not useful, as the rankings were inverted. If someone put themselves down as 10 – it usually meant they knew very little, they knew enough for their day to day work. If someone put themselves down as 2 they may be an expert who realises how much they do not know, or someone who honestly realises they do not know very much.

My humbling discovery was that when I ported some existing C code to run on z/OS, the functions were not visible outside of the C program. There were two reasons for this.

  • The functions were defined as static,
  • The functions were not exported.

Static functions

static int hidden(int  i)
{
  return 0;
}
int visible(int i)
{
  int x = hidden(1);
  return 0; 
}

I think that using “static” in this case is the wrong word. It does not mean static. I think “internal” would be a clearer description, but I do not think that I’ll have any success changing the C language to use “internal”.

The function “hidden” can only be used within the compiled unit. It cannot be referenced from outside of the compiled object. The “visible” function can use the “hidden” function as the code shows.
The function”visible” is potentially visible to external programs. You can load the module and execute the function.

Exported functions

You have to tell the compiler to externalise functions within the compile unit.

For example

#pragma export(COLIN)
int COLIN(char * self, int args) {
   return 8;
}

or the compiler option EXPORTALL for example

cc… -Wc,EXPORTALL

What has been exported?

When you bind (linkedit) your program, the binder and report the exported functions. You need the binder parameters XREF and DYNAM=DLL

This gave output like

IMPORT/EXPORT     TYPE    SYMBOL              DLL                 DDNAME   SEQ  MEMBER 
-------------     ------  ----------------    ----------------    -------- ---  --------- 
   IMPORT         CODE64  __a2e_l             CELQV003            CELQS003  01  CELQS003 
   IMPORT         CODE64  malloc              CELQV003            CELQS003  01  CELQS003 
   IMPORT         CODE64  CSQB3BAK            CSQBLB16            MQ        01  CSQBMQ2X 
                                                                                                   
   EXPORT         DATA64  ascii_tab 
   EXPORT         CODE64  printHex 
   EXPORT         CODE64  COLIN 
   

This shows the imported symbols, and where they came from, and what was exported.

  • ascii_tab is a table of data in 64 bit mode
  • printHex is a function in 64 bit mode
  • COLIN is a function in 64 bit mode.

How to use it.

You can use handle= dlopen(name,mode) to get the load module into storage, and functionPointer=dlsym(handle,”COLIN”) to locate the external symbol COLIN in the load module.

Why is a static function useful?

With Python external functions (written in C), it uses static functions to hide internals. For example

static PyObject * pymqe_MQCONN(
... )
...
static struct PyMethodDef pymqe_methods[] = { 
  {"MQCONN", (PyCFunction)pymqe_MQCONN,... }, 
  {"MQCONNX", (PyCFunction)pymqe_MQCONNX,... },
  ....  
}

When the external function is imported, the initialisation routine returns the pymqe_methods data to Python.

Python now knows what functions the module provides (MQCONN, MQCONNX), and the C code to be executed when the function is executed.

This means that you cannot load the module, and accidentally try to use the function pymqe_MQCONN; which I thought was good defensive programming.

Python on z/OS using a load module (and .so from Unix Services)

As part of playing with Python on z/OS I found you can call a z/OS Unix Services load module ( a .so object) from Python. It can also use a load module in a PDSE.

What sort is it?

A load module on z/OS can be used on one of two ways.

  • Load it from steplib or PATH environment variable (which uses dllopen under the covers), call the function, and pass the parameters The parameters might be a byte string ( char * in C terms), a Unicode string, integer etc. You return one value, for example an integer return code.
  • As part of a package where you use the Python “import package” statement. This gets loaded from PYTHONPATH, the current directory, and other directories (but not steplib). The parameters passed in are Python Objects, and you have to use Python functions to extract the value. You can return a complex Python object, for example a character string, return code and reason code.

This article is on the first case.

In both cases, the values passed in are in ASCII. If you use printf to display data, the printf treats your data as ASCII.

There is a good article on Python ctypes , a foreign function library for Python.

My initial program was

int add_it(int i, int j)
{
   return i+j;
}

I compiled it with a bash script

wc64=”-Wc,SO,LIST(lst64),XREF,LP64,DLL,SSCOM,EXPORT”
cc -c -o add.o ${wc64} add.c
cc -o add -V -Wl,DYNAM=DLL,LP64 add.o 1>ax 2>bx

This created a load module object “add”. Note: You need the EXPORT to make the entry point(s) visible to callers.

My python program was

import ctypes
from ctypes.util import find_library
zzmqe = ctypes.CDLL(“add”)
print(“mql”, zzmqe.add_it(2,5))

When this ran it produced

mql 7

As expected. To be consistent with Unix platforms, the load module should be called add.so, but “add” works.

Using strings is more complex

I changed the program (add2) to have strings as input

int add_one(char * a, char *b)
{
  printf("a %s\n",a);  
  printf("b %s\n",b);
  return 2 ;
}

and a Python program, passing a byte string.

import ctypes
from ctypes.util import find_library
zzmqe = ctypes.CDLL("add2")
print("mql", zzmqe.add_one(b'abc',b'aaa'))

This ran and gave output

-@abc–@aaa-mql 2

This shows that Python has converted the printf output ASCII to EBCDIC, and so the “a” and “b” in the printf statements are converted to strange characters, and the \n (new line) is treated as hex rather than a new line.

When I compiled the program with ASCII (-Wc…ASCII), the output from the Python program was

a abc
b aaa
mql 2

Displaying the data as expected.

Using a load module in a PDSE.

The JCL

//COLINC3 JOB 1,MSGCLASS=H,COND=(4,LE)
//S1 JCLLIB ORDER=CBC.SCCNPRC
// SET LOADLIB=COLIN.C.REXX.LOAD
// SET LIBPRFX=CEE
//COMPILE EXEC PROC=EDCCB,
// LIBPRFX=&LIBPRFX,
// CPARM=’OPTFILE(DD:SYSOPTF),LSEARCH(/usr/include/),RENT’,
// BPARM=’SIZE=(900K,124K),RENT,LIST,XREF,RMODE=ANY,AMODE=64
//COMPILE.SYSOPTF DD DISP=SHR,DSN=COLIN.C.REXX(CPARMS)
// DD *
EXPORT,LP64
/*
//COMPILE.SYSIN DD *
int add_one(int i, int j)
{
return i+j;
}

//COMPILE.SYSLIB DD
// DD
// DD DISP=SHR,DSN=COLIN.C.REXX
//BIND.SYSLMOD DD DISP=SHR,DSN=&LOADLIB.
//BIND.SYSLIB DD DISP=SHR,DSN=CEE.SCEEBND2
// DD DISP=SHR,DSN=CEE.SCEELKED
// DD DISP=SHR,DSN=CEE.SCEELIB
//BIND.OBJLIB DD DISP=SHR,DSN=COLIN.C.REXX.OBJ
//BIND.SYSIN DD *
NAME ADD3(R)
/*

In Unix services

export STEPLIB=”COLIN.C.REXX.LOAD“:$STEPLIB

The python program

import ctypes
from ctypes.util import find_library
zzmqe = ctypes.CDLL(“ADD3”)
print(zzmqe)
print(dir(zzmqe))
print(“steplib”, zzmqe.add_one(2,5))

This gave

steplib 7

Byte string and character strings parameters

I set up a C program with a function COLIN

#pragma export(COLIN)
int COLIN(char * p1, int p2) { 
  printf(" p1 %4.4s\n",p1);
  printf(" p2 %i\n",p2);
  return 8;
}

This was compiled in Unix Services, and bound using JCL into a PDSE load library as member YYYYY.

I used a shell to invoke the Python script

export LIBPATH=/u/tmp/python/usr/lpp/IBM/cyp/v3r10/pyz/lib/:$LIBPATH
export STEPLIB=COLIN.C.REXX.LOAD:$STEPLIB 
python3 dll.py

where the Python dll.py script was

import ctypes
testlib = ctypes.CDLL("YYYYY")
name =b'CSQ9'
i = 7
zz = testlib.COLIN(name,i)
print("return code",zz)

This displayed

p1 CSQ9
p2 7
return code 8

The name, a binary string CSQ9, was passed as a null terminated ASCII string (0x43535139 00).

When a string name = “CSQ9” was passed in, the data was in a Unicode string, hex values

00000043 00000053 00000051 00000039 00000000

You need to be sure to pass in the correct data (binary or Unicode), and be sure to handle the data in ASCII.

Is this how ‘import’ works?

This is different process to when a module is used via a Python import statement. If you passed in b’ABCD’ to a C extension which has been “imported” this would be passed as a Python Object, rather than the null terminated string 0x’4142434400′.

Python on z/OS advanced C extension

I found that to create a standard Python package with a C extension, the package has a very specific name – depending on the level of Python, the level of z/OS, the hardware the z/OS is running on. To be able to build a package for all levels of z/OS this would be a near impossible job; because I do not have access to every combination of z/OS hardware and software.

I’ve found a way to get round it. It took a few days to get it right, but it is pretty simple.

The standard way of packaging.

With the normal way of packaging the C executable module is stored deep in the Python tree, for example

/u/tmp/python/usr/lpp/IBM/cyp/v3r10/pyz/lib/python3.10/site-packages/pymqi-1.12.0-py3.10-os390-27.00-1090.egg/pymqi/pymqe.cpython-310.so

The easy way of using it

You can copy this file, for example to my working directory. I copied it as pymqe.so.

My Python program was

import pymqe
print(dir(pymqe))
name = “CSQ9”
rv = pymqe.MQCONN(name)
print(“rv”,rv)

This locates pymqe*.so in the current directory. It located pymqe.so; if I renamed it to pymqe.cpython-310.so it also worked.

You can put the .so object in the PYTHONPATH environment variable, and have it picked up from there.

When the file is imported, an initialisation routine is invoked which defines all of the Python entry points. You can see them using the dir(pymqe) statement. This gave me

[‘MQBACK’, ‘MQCLOSE’, ‘MQCMIT’, ‘MQCONN’, ‘MQCONNX’, ‘MQCRTMH’, ‘MQDISC’, ‘MQGET’, ‘MQINQ’, ‘MQINQMP’, ‘MQOPEN’, ‘MQPUT’, ‘MQPUT1′,’MQSET’, ‘MQSETMP’, ‘MQSUB’, ‘doc’, ‘file’, ‘loader’, ‘mqbuild’, ‘mqlevels’, ‘name’, ‘package’, ‘spec_ _’, ‘__version’, ‘pymqe.error’]

When the module is loaded Python looks for the entry name PyInit_… where … is the name of the module. For pymqe.so it looks for PyInit_pymqe. If you rename the module to mq.so and import mq, you get

ImportError: dynamic module does not define module export function (PyInit_mq)

The rv=pymqe.MQCONN invokes the MQ function which returns a handle, a return code and a reason code. For me it printed

rv (549309464, 0, 0)

So .. overall an easy solution.

I could find no way of using a load module from a PDSE, so it looks like the PYTHONPATH, or current directory is best for this.

Python on z/OS – creating a C extension

I enjoy using Python on Linux, because it is very powerful. I thought it would be interesting to port the MQ Python interface pymqi to z/OS. This exposed many of the challenges of running Python on z/OS.

I’ll cover some of the lessons I learned in doing this work. Thanks to Steve Pitman who helped me package the extension.

IBM Open Enterprise Python for z/OS, V3.8, user’s guide is a useful book.

Creating files that would compile was a challenge.

See here.

Compiling files.

I copied the pymqi C code to z/OS Unix Services, and tried to compile it. This was a mistake, as it took me a long time to get the compile options right. I found that using the setup.py script was the right way to go.

My directory tree

/u/pymqi
..setup.py
..include
....sample.h
..code
....pymqi
......__init__.py        
......CMQCFC.py          
......CMQXC.py           
......CMQZC.py           
......const.py           
......CMQC.py            
......pymqe.c            

Setup.py

This script needs export _C89_CCMODE=1, otherwise you get FSUM3008 message

Specify a file with the correct suffix (.c, .i, .s, .o, .x, .p, .I, or .a), or a corresponding data set name, instead of -L

import setuptools 
from distutils.core import setup, Extension 
import os 
import sysconfig 
# 
# This script needs    export _C89_CCMODE=1 
# Otherwise you get FSUM3008  messages 
# 
import os 
os.environ['_C89_CCMODE'] = '1' 
bindings_mode = 1 
version = '1.12.0' 
setup(name = 'pymqi', 
    version = version, 
    description = 'Python...', 
    platforms='OS Independent', 
    package_dir = {'': 'code'}, 
    packages = ['pymqi'],  
    py_modules = ['pymqi.CMQC', 'pymqi.CMQCFC', 'pymqi.CMQXC', 'pymqi.CMQZC'], 
    ext_modules = [Extension('pymqi.pymqe',['code/pymqi/pymqe.c'], define_macros=[('PYQMI_BINDINGS_MODE_BUILD', 
bindings_mode)], 
    include_dirs=["//'COLIN.MQ924.SCSQC370'"], 
    extra_link_args=["//'COLIN.MQ924.SCSQDEFS.OBJ(CSQBMQ2X)'"], 
      )] 
) 
# I had extra_link_args=["-Wl,INFO,LIST,MAP",.... when setting 
# this up
# I used 
# extra_compile_args=["-Wc,LIST(c.lst),XREF"], 
# to get out a listing and cross reference.

Which says

  • The package name is packages = [‘pymqi’],
  • The Python files are py_modules = [‘pymqi.CMQC’….
  • There is an extension .. ext_modules=.. with the source program code/pymqi/pymqe.c
  • It needs “//’COLIN.MQ924.SCSQC370′” to compile and “//’COLIN.MQ924.SCSQDEFS.OBJ(CSQBMQ2X)'” at bind time. This file contains the MQ Binder input
  • When I wanted the binder output – “-Wl,INFO,LIST,MAP”. This goes to the terminal. I used a ‘>’ command to pipe the output of the python3 setup build it to a file.
  • and a C listing “-Wc,LIST(c.lst),XREF”. The listing goes to c.lst

You need

  • import setuptools so that the setup bdist_wheel packaging works. You also need the wheel package installed.

Setup

There is a buglet in the compile set up. You need to specify

export _C89_CCMODE=1

Without it you get

FSUM3008 Specify a file with the correct suffix (.c, .i, .s, .o, .x, .p, .I, or .a), or a corresponding data set name, instead of -obuild/lib.os390-27.00-1090-3.8/pymqi/pymqe.so.

You also need the binder input in a data set with the correct suffix. For example .OBJ

“//’COLIN.MQ924.SCSQDEFS.OBJ(CSQBMQ2X)'”

If you do not have the correct suffix you get

FSUM3218 xlc: File //’COLIN.MQ924.SCSQDEFS(CSQBMQ2X)’ contains an incorrect file suffix.

Doing the compile and test install

I used a shell script to do the compiles and install

touch code/pymqi/*.c
rm a b c d
export _C89_CCMODE=1
#python3 setup.py clean
python3 setup.py build 1>a 2>b
python3 setup.py install 1>c 2>d

I captured the output from the setup.py jobs using 1>a etc because I could not see how to direct the binder output to a file. It comes out on the terminal – and there was a lot of it!.

Packaging the package

Python build which worked

I had to install wheel package. See How to install software in an isolated environment, or just use python3 -m pip install wheel if your z/OS image is connected to the network.

The command I used was

python3 -m pip install –user –no-cache-dir /u/tmp/py/wheel-0.37.1-py2.py3-none-any.whl-f /u/tmp/py/wheel-0.37.1-py2.py3-none

I had to add import setuptools to my setup.py file (at the top). (This converted the install package from a dist-utils to a setuptools packaging)

python3 setup.py bdist_wheel

This created a file “/u/pymqi/dist/pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl

This file is specific to python 3.10

For a wheel package, you’ll need to build it for all major versions and cannot just use one. Note that there were some issues in 3.8/3.9 with wheels that have been resolved in 3.10, so it’s recommended you to use 3.10.

Steven Pitman

This means you need to have multiple levels of Python installed, and build for each one!

This also has the operating system level (os390_27_00 – this may be constant across machines) and the hardware 1090. For this to work on other hardware, one solution would be to manually rename the file to pretend it is for a different machine, but this both not supported nor recommended, and has no guarantee to work. So it is hard to know the best thing to do. I do not have every 390 machine from IBM to do a build on !

Failing build. This built but did not install.

python3 setup.py bdist –format=tar

It built the package and create a file

./dist/pymqi-1.12.0.os390-27.00-1090.tar

This tar file is not completely readable by the z/OS tar command.

When I used tar -tf ….tar it gave

FSUMF371 Value 1641318408.0 is not valid for keyword mtime. Keyword not set.

It uses a Python tar command, not the operating system tar command.

You can display the contents using a Python program like

import tarfile
tar = tarfile.open("dist/pymqi-1.12.0.os390-27.00-1090.tar.gz")
# tar.extractall() 
for x in tar:
    print(x)

This gave output like

<TarInfo ‘.’ at 0x5008ad3880>
<TarInfo ‘./usr’ at 0x5008ad3dc0>
<TarInfo ‘./usr/lpp’ at 0x5008ad3a00>

Installing the package

From an authorised user in OMVS,

python3 -m pip install –no-cache-dir /u/pymqi/dist/pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl /u/pymqi/dist/pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl
Processing /u/pymqi/dist/pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl
Installing collected packages: pymqi
Successfully installed pymqi-1.12.0

If you do not use –no-cache-dir, you may get

-[33]WARNING: The directory ‘/u/.cache/pip’ or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo’s -H flag.-[0]

The compile options

The following text is the compile and bind options used for my code. Some of the options are pymqi specific.

/bin/xlc -DNDEBUG -O3 -qarch=10 -qlanglvl=extc99 -q64
-Wc,DLL
-D_XOPEN_SOURCE_EXTENDED
-D_UNIX03_THREADS
-D_POSIX_THREADS
-D_OPEN_SYS_FILE_EXT
-qexportall -qascii -qstrict -qnocsect
-Wa,asa,goff -Wa,xplink
-qgonumber -qenum=int
-DPYQMI_BINDINGS_MODE_BUILD=1 -I//’COLIN.MQ924.SCSQC370′
-I/u/tmp/python/usr/lpp/IBM/cyp/v3r10/pyz/include/python3.10
-c code/pymqi/cpmqe.c
-o build/temp.os390-27.00-1090-3.10/code/pymqi/cpmqe.o

/bin/xlc build/temp.os390-27.00-1090-3.10/code/pymqi/cpmqe.o -L.
-o build/lib.os390-27.00-1090-3.10/pymqi/cpmqe.cpython-310.so -Wl,INFO,LIST,MAP,DLL //’COLIN.MQ924.SCSQDEFS.OBJ(CSQBMQ2X)’
-Wl,dll
/u/tmp/python/usr/lpp/IBM/cyp/v3r10/pyz/lib/python3.10/config-3.10/libpython3.10.x
-q64

Python on z/OS coding a C extension.

I was porting the pymqi code, which provides a Python interface to IBM MQ, to z/OS.

I’ve documented getting the code to build. I also had challenges trying to use it.

The code runs as ASCII!

When my C extension was built, it gets built with the ASCII option. This means any character constants are in ASCII not, EBCDIC.

My python program had

import sys
sys.path.append(‘./’)
import pymqi
queue_manager = ‘AB12’
qmgr = pymqi.connect(queue_manager)
qmgr.disconnect()

When my C code got to see the AB12 value, it was in ASCII. When the code tried to connect to the queue manager, it return with name error. This was because the value was in ASCII, and the C code expected EBCDIC.

You can take see if the program has been compiled with the ASCII flag using

#ifdef __CHARSET_LIB

… It is has been compiled with option ASCII
#else

If you use printf to display the queue manager name it will print AB12. If you display it in hex you will be 41423132 where A is x41, B is x42, 1 is x31 is 2 is x32.

In your C program you can convert this using the c function a2e with length option, for example

char EQMName[4];
memcpy(&EQMName[0],QMname,4);
__a2e_l(&EQMName[0],sizeof(EQMName));

// then use &EQMName

Converting from ASCII to EBCDIC in Python.

Within Python you have strings stored as Unicode, and byte data. If you have a byte array with x41423132 (the ASCII equivalent to AB12). You can get this in the EBCDIC format using

a=b’41423132′ # this is the byte array
m = a.decode(“ascii”) # this creates a character string
e = m.encode(‘cp500’) # this create the new byte array of the EBCDIC version .. or xC1C2F1F2

In Python you convert from a dictionary of fields into a “control block” using the pack function. You can use the “e” value above in this.
MQ control blocks have a 4 character eye catcher at the front eg “OD “. If you use the pack function and pass “OD ” you will pass the ASCII version. You will need to do the decode(‘ascii’) encode(‘CP500’) to create the EBCDIC version.

Similarly passing an object such as MQ queue name, will need to be converted to the EBCDIC version.

Converting from EBCDIC to ASCII

If you want to return character data from your C program to Python, you will need to the opposite.

For example m is a byte array retuned back from the load module.

#v is has the value b’C1C2F2F3′ (AB23)
m = v.decode(“cp500”)
a = m.encode(‘ascii’)

# a now has b’41423132′ which is the ascii equivilant

A short C quiz, and some gotcha’s

I’ve been looking at porting pymqi, the Python MQ interface to z/OS.

The biggest challenges where nothing to do with Pymqi.

So if you are bored after Christmas and want something stimulating… here are a few questions for you… The answers are below. I tried getting them displayed upside down, like all quality magazines; but that was too difficult.

Question 1. C question

I’ve reduced the problem I experienced,down to

int main() 
{ 
if ( 1==0 ) return 8; 
int rc; 
*=ERROR===========> CCN3275 Unexpected text 'int' encountered.
}                                                          

Hint: it works in a batch compile, using EDCCB

Question 2 binding in Unix Services

/bin/xlc a.o -L. -o b.so -Wl,INFO //’COLIN.MQ924.SCSQDEFS(CSQBRR2X)’ -Wl,dll c.x

Gave

FSUM3218 xlc: File //’COLIN.MQ924.SCSQDEFS(CSQBRR2X)’ contains an incorrect file suffix.

What do I need to do to fix it?

Question 3. Strange bind messages

Before I found the solution to problem number 2, I put the bind statements into a Unix Services file.

Using this gave me

IEW2326E 1221 THE FOLLOWING INVALID RECORD HAS BEEN SEEN:
=”lm-source” *

Copyright
IEW2326E 1221 THE FOLLOWING INVALID RECORD HAS BEEN SEEN:
IBM Corp. 2009, 2016 All Rights Reserved.

This bind statement was

cc -o mqsamp -W l,DYNAM=DLL,LP64 c.o mq.o

it worked without the mq.o

The mq.o file had

* <copyright                                                          * 
* notice="lm-source"                                                  * 
* (C) Copyright IBM Corp. 2009, 2016 All Rights Reserved.             * 
* </copyright>                                                        * 

Answer

  1. Using the cc compiler, it defaults to #pragma Langlvl(stdc89) which supports the c89 level of C. This says all variable declarations must come before any logic. This is relaxed in the c99 level, so specifying #pragma Langlvl(stdc99) cures it. You can also specify LANGLVL(EXTENDED) in the cc statement
  2. To include datasets in some of the binder options you need host file: filename with .OBJ suffix (object host file for the binder/IPA Link). When I used /bin/xlc a.o … -Wl,INFO //’COLIN.MQ924.SCSQDEFS.OBJ(CSQBRR2X)’ … it worked.
  3. The binder is not good at files in Unix Services, it likes records which are fixed block 80. The mq.o file had trailing blanks removed, and this confused it. I had to use a PDSE to get it to work.

Rexx to C to Rexx sample code

I’ve put up on github some sample code to demonstrate how you can write a function in C, and invoke it from Rexx. I’ve provided some glue code as Rexx uses R0 and R1 to pass parameters, and C programs only use R1.

I’ve create some small functions to use in your C program which hide the Rexx logic. For example

rc = CRexxDrop(pEnv,”ZDROP”);
rc = CRexxGet(pEnv,”InSymbol”,&buffer[0],&lBuffer);
rc = CRexxPut(pEnv,”CPPUTVar,”Colinsv”,0);
Iterate through all symbols

If you have any comments or suggestions, please let me know.

Where’s my invisible code.

In trying to get system exits written in C to work. I found my code was not being executed, even when the first instructions were a deliberate program check. I tried using the tried and trusted program AMASPZAP (known as Super Zap) for displaying the internals of a load module and zapping it – but my code was not there! Where was it hiding? When I took a dump of the address space my code was in the dump. Why was it invisible and not being executed?

HSM archives on tape

Like taking 20 minutes to recall a long unused dataset from HSM (mounting a physical tape to retrieve the data set), I had this vague memory of doing a presentation on the binder and the structure of load modules. After a cup of tea and a chocolate biscuit to help the recall, I remembered about classes etc in a load module.

Classes etc

When I joined IBM over 40 years ago you wrote your program, and used the link editor to create the load module, a single blob of instructions and data.

Things have moved on. Think about a C program, in read only memory. When you issue a load to use it, you get a pointer to the read only (the re-entrant instructions and data), and your own copy of the “global variables” or Writeable Static Area (WSA). When using the C compiler, at bind time it includes a bit of code with 24 bit addressing mode. This means you have code which runs in 31/64 bit mode, and some code resident in 24 bit mode! It is no longer a single blob of instructions and data.

Within the load module there are different classes of data for example

  • C_CODE – C code
  • C_WSA – for a C program compiled with RENT option. This is the global data which each instance gets its own private copy of
  • B_TEXT code from the assembler
  • Using the HL Assembler, you can define your own classes using CATTR.

A class has attributes, such as

  • Length.
  • Should it be loaded or not. You could store documentation in the load module, which can be used by programs, but not needed at execution time.
  • RMODE.
  • It is reentrant or not.
  • Should this code be merged or replaced with similar code. For example the C Globals section would be merged. A block of instructions would be replace.

Segments

The binder can take things with similar attributes and store them together within a segment. You can have mixed classes eg B_TEXT and C_CODE, with the same RMODE attributes etc and have them in one segment. The C_WSA needs to be in a different segment because it has different attributes.

So where was my invisible code?

I needed to change my SPZAP job to tell it to dump out the C_CODE section. By default it dumps the B_TEXT sections. You can specify C_* or B_*. See the AMASPZAP documentation.

//STEP EXEC PGM=AMASPZAP
//SYSPRINT DD SYSOUT=A
//SYSLIB DD DISP=SHR,DSN=COLIN.C.LOAD
//SYSIN DD *
DUMPT COLIN CPPROGH C_CODE
/*

This dumps out (decoding the data into instructions) load module COLIN, CSECT CPPROGH and the C_CODE class.

Why wasn’t my code executing? The code to set up the C environment was not invoking my program because I had compiled it with the wrong options!