Putting assembler code inside a C program

I was using a RACF service, and the documentation casually says “the application must free the returned storage”.

C does not provide facilities to use the STORAGE RELEASE z/OS service, so I had to write some assembler code to do this. It wasn’t difficult, I just fell over many little problems.

The IBM extension to the C language ASM is defined in the documentation.

I think of asm() as a special macro.

  • You pass in strings of assembler code. asm() will format it, and split the line if it would be longer than 72 characters.
  • If you want multiple lines you should end each line with \n (standard printf in C).
  • Lines starting with “*” are treated as comments, and produce no code
  • You can start your line with just one blank, before the instruction and it will be formatted as properly aligned assembler code, wrapping onto new lines if needed.
  • You do not use the C variable names in the assembler source.
    • You specified a mapping like [length] “m”(lDeletestorage), it takes the C variable lDeletestorage, determines its storage address such as it is offset 40 from register 5 (40(5)) and assigns this value to “length”
    • In the assembler code you specify ” L 2,%[length] \n” and it substitutes the value for %[length]. This instruction becomes ” L 2,40(5) “
    • At first glance it you would think it should be able to substitute the value directly, but you need to specify meta information about the field (is it used read only or read/write, is it a storage location, or a literal string). This is why there is this indirection
  • You need to specify which registers are modified so the C code can either save/restore their values, or just use different registers.

My code

if ( pDeletestorage != 0 )
{
int freerc = 0;
#define ASM_PREFIX " SYSSTATE ARCHLVL=2 \n"
asm(
ASM_PREFIX
" L 2,%[length] \n"
" L 4,%[a] \n"
" STORAGE RELEASE,LENGTH=(2),ADDR=(4),COND=YES,RTCD=%[rc] "
: [rc] "+m"(freerc) //* output
: [length] "m"(lDeletestorage,
[a] "m"(pDeletestorage)
:"r0", "r1" , "r2" , "r4" , "r15" );
printf("Storage release rc %i\n",rc);
}
  • if ( pDeletestorage != 0 ) If there is a block of storage to release
  • {
  • int freerc = 0; define and preset the return code
  • #define ASM_PREFIX ” SYSSTATE ARCHLVL=2 \n” see below. The \n makes a new line
  • asm( generate the code
  • ASM_PREFIX insert the SYSTATE code
  • ” L 2,%[length] \n” Load register 2 with the value of the variable length defined below
  • ” L 4,%[a] \n” Load register 4 with the value of the variable a(ddress) defined below
  • ” STORAGE RELEASE,LENGTH=(2),ADDR=(4),COND=YES,RTCD=%[rc] “ issue the storage macro
  • : After the first : is a list of output variables
    • [rc] “+m”(freerc) //* output [rc] matches the name in the RTCD=%[rc]
      • “+” means Indicates that the operand is both read and written by the instruction
      • “m” is use a memory object – that is, a variable. Other options are literal values or registers.
      • (freerc) is the name of the C variable
  • : After the second : is a list of input variables
    • [length] “m”(lDeletestorage),
    • [a] “m”(pDeletestorage)
  • : After the third : is a list of register that are changed (clobbered)
    • “r0”, “r1” , “r2” , “r4” , “r15”
  • );
  • printf(“Storage release rc %i”,rc);
  • }

You have to pass the data to the macro using registers. You can either select registers yourself, or let the compiler pick them for you. See Using registers with my program as a more correct way of coding the program.

Note: If you pick the registers yourself, they may be unavailable if you compile it as a 64 bit program or as XPLINK, so using Using registers with my program is better.g

With the SYSSTATE ARCHLVL=2

This statement sets the minimum architecture to  z/Architecture (which includes Jump instructions).

The generated code was

* SYSSTATE ARCHLVL=2                                              
* THE VALUE OF SYSSTATE IS NOW SET TO ASCENV=P AMODE64=NO ARCHLVX01-SYSS
* L=2 OSREL=00000000 RMODE64=NO
L 2,1368(13)
L 4,1364(13)

STORAGE RELEASE,LENGTH=(2),ADDR=(4),COND=YES,RTCD=1372(13)
LR 0,2 .STORAGE LENGTH
LR 1,4 .ADDRESS OF STORAGE
LHI 15,X'0001' .Add in parameters
L 14,16(0,0) .CVT ADDRESS
L 14,772(14,0) .ADDR SYST LINKAGE TABLE
L 14,204(14,0) .OBTAIN LX/EX FOR RELEASE
PC 0(14) .PC TO STORAGE RTN
ST 15,1372(13) .SAVE RETURN CODE

Where

  L     2,1368(13)       
L 4,1364(13)

is loading the registers with the variable data from the C code,

and

  ST     15,1372(13)                  .SAVE RETURN CODE      

saves the return code into the C variable.

Without the SYSSTATE ARCHLVL=2

The code failed to compile because of addressability issues.

         L     2,1368(13)                                              
L 4,1364(13)
STORAGE RELEASE,LENGTH=(2),ADDR=(4),COND=YES,RTCD=1372(13)
CNOP 0,4
B IHB0001B .BRANCH AROUND DATA
*** ASMA307E No active USING for operand IHB0001B
IHB0001F DC BL1'00000000'
DC AL1(0*16) .KEY
DC AL1(0) .SUBPOOL
DC BL1'00000001' .FLAGS
IHB0001B DS 0F
LR 0,2 .STORAGE LENGTH
LR 1,4 .ADDRESS OF STORAGE
L 15,IHB0001F .CONTROL INFORMATION
*** ASMA307E No active USING for operand IHB0001F
L 14,16(0,0) .CVT ADDRESS
L 14,772(14,0) .ADDR SYST LINKAGE TABLE
L 14,204(14,0) .OBTAIN LX/EX FOR RELEASE
PC 0(14) .PC TO STORAGE RTN
ST 15,1372(13) .SAVE RETURN CODE

In days of old you had base registers to locate data and instructions. Data reference was relative to (USING) a base register. Branching within a program used one or more base register. A re-entrant program would get dynamic storage, and this would be addressed by its own base register.

Modern instructions include the Jump instruction. This says jump this many half words to the instruction. These instructions do not need a base register.

With SYSSTATE ARCHLVL=2, the generated code for the C part of the program used the jump instructions, and did not need a base register. Assembler macros that generate code the old way, need a base register.

Using old instruction, to load a value into a register, it was loaded from storage – which needed a base register to locate the storage. For example

IHB0001F DC     BL1'00000000'                                          
DC AL1(0*16) .KEY
DC AL1(0) .SUBPOOL
DC BL1'00000001' .FLAGS
...
L 15,IHB0001F .CONTROL INFORMATION

Modern instructions have the “constant” value as part of the instruction

 LHI    15,X'0001'             

This effectively clears register 15 and loads the value 0x0001 into the bottom. (To be accurate it moves 0x0001 to the bottom half, then propagates the sign bit to the upper half word. The sign bit is 0.)

Using registers

The z/OS uses registers 14,15,0,1 for linkage, and these are likely to be used when using macros. Register 3 was used by the C code to locate storage, so I used registers 2 and 4.

Using variables

Reading the documentation, it seems I should be able to say

 " STORAGE RELEASE,LENGTH=%[length],ADDR=(4),COND=YES,RTCD=%[rc] "
: ...
: [length] "m"(lDeletestorage),
[a] "m"(pDeletestorage)
:...

passing in a variable. This does not work, because C does not pass in “lDeletestorage”, but the offset and register.

For example the storage for variable lDeleteme is 1368 off register 13.

It can be used in a Load instruction

  L     2,1368(13)  

But not every where. In the macro it get substituted.

STORAGE RELEASE,LENGTH=1368(13),ADDR=(4),COND=YES,RTCD=1372(13)
L 0,=A(1368(13)) .STORAGE LENGTH
*** ASMA035S Invalid delimiter - 1368(13)

Passing back the return code using RTCD=%[rc]. The macro checks to see if the value of RTCD is a register, if not then use the value in a store instruction.

Using registers

You can specify that you want to use a register, and the ASM() will pick registers for you.

 int res = 0x12345678;        
int newRes = 55;
__asm(" SR %[rx],%[rx] clear \n"
" SR %[rx],%[ry] them \n"
" LR %[rx],%[ry] COLINS\n"

: [rx]"=r"(res) // output
: [ry]"r"(newRes) // input
);

This generates code

     L        r4,newRes(,r13,1380) 
* SR 2,2 clear
* SR 2,4 them
* LR 2,4 COLINS
SR r2,r2
SR r2,r4
LR r2,r4

LR r0,r2
ST r0,res(,r13,1376)

It has selected to use registers 2 and 4.

Where

  • L r4,newRes(,r13,1380) loads the input value into a register of its choice
  • *… these are comments of the coded instructions
  • SR.. LR.. are the generated instructions
  • LR r0,r2 , ST r0,res(,r13,1376) saves the variable defined as output back into C storage.
Using registers with my program

The compiler decides which registers to user ( r2 and r4). When I compiled this code as 64 bit (using option lp64) it used registers 6 and 7.

asm( ASM_PREFIX 
" STORAGE RELEASE,LENGTH=(%[length]),ADDR=(%[a]),COND=YES,RTCD=%[rc] "
: [rc] "+m"(freerc) //* output
: [length] "r"(lDeleteme),
[a] "r"(pDeleteme)
:"r0", "r1" , "r15" );

This generates

 *          asm( 
L r2,lDeleteme(,r13,1368)
L r4,pDeleteme(,r13,1364)
SYSSTATE ARCHLVL=2

STORAGE RELEASE,LENGTH=(2),ADDR=(4),COND=YES,RTCD=1372(13)
LR 0,2 .STORAGE LENGTH
LR 1,4 .ADDRESS OF STORAGE
LHI 15,X'0001' .Add in parameters @PCA
L 14,16(0,0) .CVT ADDRESS
L 14,772(14,0) .ADDR SYST LINKAGE TABLE
L 14,204(14,0) .OBTAIN LX/EX FOR RELEASE
PC 0(14) .PC TO STORAGE RTN
ST 15,1372(13) .SAVE RETURN CODE

Using the different data types.

This is not explained very well in the documentation.

Using literal integer constants

The code

asm( "LABEL LA     4,%[i] \n" 
:
: [i] "i"(99)
: "r4" );

Generates

 * LABEL    LA    4,99                    Colins             
LA r4,99

The label I specified on the instruction is not added to the created instruction.

Using literal string constants

Instead of using this capability, you could use C constants, and the “m” definition.

The code

asm( "LABEL LA     4,%[i] Colins \n" 
:
: [i] "i"("ABCD")
: "r4" );

Gives a compile error

10 LABEL    LA    4,ABCD                  Colins    
*** ASMA044E Undefined symbol - ABCD

The code

asm("LABEL LA     4,%[i] Colins \n" 
:
: [i] "i"("=C'ABCD'")
: "r4" );

Generates

* LABEL    LA    4,=C'ABCD'              Colins          
LA r4,0(,r3)
...
Start of ASM Literals
000360 C1C2C3C4 =C'ABCD'

Using a memory operand that is offsetable (o)

The code

asm("LABEL LA     4,%[i] Colins \n" 
"LABEL2 LA 4,%[j] Colins2\n"
:
: [i] "o"(res),
[j] "m"(res)
: "r4" );

produced

* LABEL    LA    4,1376(13)              Colins      
* LABEL2 LA 4,1376(13) Colins2
LA r4,1376(r13,)
LA r4,1376(r13,)

So it looks like the “o” operand is the same as specifying “m”.

64 bit programming

When you compile in 64 bit code (option lp64). The asm() works just as well It is better to use Using registers with my program so you do not have to guess which registers you can use.

My code

asm( ASM_PREFIX 
" STORAGE RELEASE,LENGTH=(%[l]),ADDR=(%[a]),COND=YES,RTCD=%[rc] "
: [rc] "+m"(freerc) //* output
: [l] "r"(lDeleteme),
[a] "r"(pDeleteme)
:"r0", "r1" , "r15"
);

Generated

    LLGF     r6,lDeleteme(,r4,3504) 
LG r7,pDeleteme(,r4,3496)
SYSSTATE ARCHLVL=2
...

Where the LLGF loads the 32 bit length variable into register 6, and LG loads the 64 bit address variable into register 7.

Compiling the code

I used the shell script

name=irrseq
export _C89_CCMODE=1
p1="-Wc,arch(8),target(zOSV2R3),list,source,ilp32,gonum,asm,float(ieee)"
p5=" -I. "


p7="-Wc,ASM,ASMLIB(//'SYS1.MACLIB') -Wa,LIST,RENT"

p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,XREF,SHOWINC "

xlc $p1 $p5 $p7 $p8 -c $name.c -o $name.o

l1="-Wl,LIST,MAP,XREF "
/bin/xlc $name.o -o irrseq -V $l1 1>a

The important line, p7, contains

  • -Wc,ASM,ASMLIB(//’SYS1.MACLIB’) C compile options
    • ASM Enables inlined assembly code inside C/C++ programs.
    • ASMLIB Specifies assembler macro libraries to be used when assembling the assembler source code.
  • -Wa,LIST,RENT” Assembler options
    • LIST Instructs the assembler to produce a listing
    • RENT Specifies that the assembler checks for possible coding violations of program reenterability

That’s strange – the compile worked.

I was setting up a script to compile some C code in Unix Services, and it worked – when I expected the bind to fail because I had not specified where to find a stub file.

How to compile the source

I used a shell script to compile and bind the source. I was surprised to see that it worked, because it needed some Linkedit stubs from CSSLIB. I thought I needed

export _C89_LSYSLIB=”CEE.SCEELKEX:CEE.SCEELKED:CBC.SCCNOBJ:SYS1.CSSLIB”

but it worked without it.

The script

name=irrseq 

export _C89_CCMODE=1
# export _C89_LSYSLIB="CEE.SCEELKEX:CEE.SCEELKED:CBC.SCCNOBJ:SYS1.CSSLIB"
p1="-Wc,arch(8),target(zOSV2R3),list,source,ilp32,gonum,asm,float(ieee)"
p5=" -I. "
p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,XREF,SHOWINC "

xlc $p1 $p5 $p8 -c $name.c -o $name.o
# now bind it
l1="-Wl,LIST,MAP,XREF "
/bin/xlc $name.o -o irrseq -V $l1 1>binder.out

The binder output had

XL_CONFIG=/bin/../usr/lpp/cbclib/xlc/etc/xlc.cfg:xlc 
-v -Wl,LIST,MAP,XREF irrseq.o -o./irrseq
STEPLIB=NONE
_C89_ACCEPTABLE_RC=4
_C89_PVERSION=0x42040000
_C89_PSYSIX=
_C89_PSYSLIB=CEE.SCEEOBJ:CEE.SCEECPP
_C89_LSYSLIB=CEE.SCEELKEX:CEE.SCEELKED:CBC.SCCNOBJ:SYS1.CSSLIB

Where did these come from? – I was interested in SYS1.CSSLIB. It came from xlc config file below.

xlc config file

By default the compile command uses a configuration file /usr/lpp/cbclib/xlc/etc/xlc.cfg .

The key parts of this file are

* FUNCTION: z/OS V2.4 XL C/C++ Compiler Configuration file
*
* Licensed Materials - Property of IBM
* 5650-ZOS Copyright IBM Corp. 2004, 2018.
* US Government Users Restricted Rights - Use, duplication or
* disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
*

* C compiler, extended mode
xlc: use = DEFLT
...* common definitions
DEFLT: cppcomp = /usr/lpp/cbclib/xlc/exe/ccndrvr
ccomp = /usr/lpp/cbclib/xlc/exe/ccndrvr
ipacomp = /usr/lpp/cbclib/xlc/exe/ccndrvr
ipa = /bin/c89
as = /bin/c89
ld_c = /bin/c89
ld_cpp = /bin/cxx
xlC = /usr/lpp/cbclib/xlc/bin/xlc
xlCcopt = -D_XOPEN_SOURCE
sysobj = cee.sceeobj:cee.sceecpp
syslib = cee.sceelkex:cee.sceelked:cbc.sccnobj:sys1.csslib
syslib_x = cee.sceebnd2:cbc.sccnobj:sys1.csslib
exportlist_c = NONE
exportlist_cpp = cee.sceelib(c128n):cbc.sclbsid(iostream,complex)
exportlist_c_x = cee.sceelib(celhs003,celhs001)
exportlist_cpp_x = ...
exportlist_c_64 = cee.sceelib(celqs003)
exportlist_cpp_64 = ...
steplib = NONE

Where the _x entries are for xplink.

It is easy to find the answer when you know the solution.

Note:

Without the export _C89_CCMODE=1

I got

IEW2763S DE07 FILE ASSOCIATED WITH DDNAME /0000002 CANNOT BE OPENED
BECAUSE THE FILE DOES NOT EXIST OR CANNOT BE CREATED.
IEW2302E 1031 THE DATA SET SPECIFIED BY DDNAME /0000002 COULD NOT BE
FOUND, AND THUS HAS NOT BEEN INCLUDED.
FSUM3065 The LINKEDIT step ended with return code 8.

Compiling in 64 bit

It was simple to change the script to compile it in 64 bit mode, but overall this didn’t work.

p1="-Wc,arch(8),target(zOSV2R3),list,source,lp64,gonum,asm,float(ieee)" 
...
l1="-Wl,LIST,MAP,XREF -q64 "

When I compiled in 64 bit mode, and tried to bind in 31/32 bit mode (omitting the -q64 option) I got messages like

 IEW2469E 9907 THE ATTRIBUTES OF A REFERENCE TO isprint FROM SECTION irrseq#C DO
NOT MATCH THE ATTRIBUTES OF THE TARGET SYMBOL. REASON 2
...
IEW2469E 9907 THE ATTRIBUTES OF A REFERENCE TO IRRSEQ00 FROM SECTION irrseq#C
DO NOT MATCH THE ATTRIBUTES OF THE TARGET SYMBOL. REASON 3

IEW2456E 9207 SYMBOL CELQSG03 UNRESOLVED. MEMBER COULD NOT BE INCLUDED FROM
THE DESIGNATED CALL LIBRARY.
...
IEW2470E 9511 ORDERED SECTION CEESTART NOT FOUND IN MODULE.
IEW2648E 5111 ENTRY CEESTART IS NOT A CSECT OR AN EXTERNAL NAME IN THE MODULE.

IEW2469E THE ATTRIBUTES OF A REFERENCE TO … FROM SECTION … DO
NOT MATCH THE ATTRIBUTES OF THE TARGET SYMBOL. REASON x

  • Reason 2 The xplink attributes of the reference and target do not match.
  • Reason 3 Either the reference or the target is in amode 64 and the amodes do not match. The IRRSEQ00 stub is only available in 31 bit mode, my program was 64 bit amode.

IEW2456E SYMBOL CELQINPL UNRESOLVED. MEMBER COULD NOT BE INCLUDED
FROM THE DESIGNATED CALL LIBRARY.

  • The compile in 64 bit mode generates an “include…” of the 64 bit stuff needed by C. Because the binder was in 31 bit, it used the 31 bit libraries – which did not have the specified include file. When you compile in 64 bit mode you need to bind with the 64 bit libraries. The compile command sorts all this out depending on the options.
  • The libraries used when binding in 64 bit mode are syslib_x = cee.sceebnd2:cbc.sccnobj:sys1.csslib. See the xlc config file above.

IEW2470E 9511 ORDERED SECTION CEESTART NOT FOUND IN MODULE.
IEW2648E 5111 ENTRY CEESTART IS NOT A CSECT OR AN EXTERNAL NAME IN THE MODULE.

  • Compiling in 64 bit mode, generates an entry point of CELQSTRT instead of CEESTART, so the binder instructions for 31 bit programs specifying the entry point of CEESTART will fail.

Overall

Because IRRSEQ00 only comes in a 31 bit flavour, and not a 64 bit flavour, I could not call it directly from a 64 bit program, and I had to use a 32 bit compile and bind.

Python calling C functions

  1. You can have Python programs which are pure Python.
  2. You can call C programs that act like Python programs, using Python constructs within the C program
  3. You can call a C program from Python, and it processes parameters like a normal C program.

This blog post is about the third, calling a C program from Python, passing simple data types such as char, integers and strings.

I have based a lot of this on the well written pyzfile package by @daveyc.

The glue that makes it work is the ctypes package a “foreign function library” package.

Before you start

The blog post is called “Python calling C functions”. I tried using a z/OS stub code directly. This is not written in C, and I got.

CEE3595S DLL ... does not contain a CELQSTRT CSECT.

Which shows you must supply a C program.

The C program that I wrote, calls z/OS services. These must be defined like (or default to)

#pragma linkage(...,OS64_NOSTACK)     

Getting started

My C program has several functions including

int query() {  
return 0;
}

The compile instructions said exportall – so all functions are visible from outside of the load module.

You access this from Python using code like

lib_file = pathlib.Path(__file__).parent / "pySMFRealTime.so"
self.lib = ctypes.CDLL(str(lib_file))
...
result = self.lib.query()

Where

  • lib_file = pathlib.Path(__file__).parent / “pySMFRealTime.so” says get the file path of the .so module in the same directory as the current Python file.
  • self.lib = ctypes.CDLL(str(lib_file)) load the file and extract information.
  • result = self.lib.query() execute the query function, passing no parameters, and store any return code in the variable result

Passing simple parameters

A more realistic program, passing parameters in, and getting data back in the parameters is

int conn(const char* resource_name,  // input:  what we want to connect to
char * pOut, // output: where we return the handle
int * rc, // output: return code
int * rs, // output: reason code
int * debug) // input: pass in debug information
{
int lName = strlen(resource_name);
if (*debug >= 1)
{
printf("===resource_namen");
printHex(stdout,pFn,20);
}
...
return 0;
}

The Python code has

lib_file = pathlib.Path(__file__).parent / "pySMFRealTime.so"
self.lib = ctypes.CDLL(str(lib_file))
self.lib.conn.argtypes = [c_char_p, # the name of stream
c_char_p, # the returned buffer
ctypes.POINTER(ctypes.c_int), # rc
ctypes.POINTER(ctypes.c_int), # rs
ctypes.POINTER(ctypes.c_int), # debug
]
self.lib.conn.restype = c_int

The code to do the connection is

def conn(self,name: str,):
token = ctypes.create_string_buffer(16) # 16 byte handle
rc = ctypes.c_int(0)
rs = ctypes.c_int(0)
debug = ctypes.c_int(self.debug)
self.token = None
retcode = self.lib.conn(name.encode("cp500"),
token,
rc,
rs,
debug)
if retcode != 0:
print("returned rc",rc, "reason",rs)
print(">>>>>>>>>>>>>>>>> connect error ")
return None
print("returned rc",rc, "reason",rs)
self.token = token
return rc

The code does

  • def conn(self,name: str,): define the conn function and pass in the variable name which is a string
  • token = ctypes.create_string_buffer(16) # 16 byte handle create a 16 byte buffer and wrap it in ctypes stuff.
  • rc = ctypes.c_int(0), rs = ctypes.c_int(0), debug = ctypes.c_int(self.debug) create 3 integer variables.
  • self.token = None preset this
  • retcode = self.lib.conn( invoke the conn function
    • name.encode(“cp500”), convert the name from ASCII (all Python printable strings are in ASCII) to code page 500.
    • token, the 16 byte token defined above
    • rc, rs, debug) the three integer variables
  • if retcode != 0: print out error messages
  • print(“returned rc”,rc, “reason”,rs) print the return and reason code
  • self.token = token save the token for the next operation
  • return rc return to caller, with the return code.

Once I got my head round the ctypes… it was easy.

The C program

There are some things you need to be aware of.

  • Python is compiled with the -qascii compiler option, so all strings etc are in ASCII. The code name.encode(“cp500”), converts it from ASCII to EBCDIC. The called C program sees the data as a valid EBCDIC string (null terminated).
  • If a character string is returned, with printable text. Either your program coverts it to ASCII, or your Python calling code needs to convert it.
  • Your C program can be compiled with -qascii – or as EBCDIC(no -qascii)
    • Because Python is compiled in ASCII, the printf routines are configured to print ASCII. If your program is compiled as ASCII, printf(“ABCD”) will print as ABCD. If your program is compiled as EBCDIC printf(“ABCD”) will print garbage – because the hex values for EBCDIC ABCD are not printable as ASCII characters.
    • If your program is compiled as ASCII you can define EBCDIC constants.
      • #pragma convert(“IBM-1047”)
      • char * pEyeCatcher = “EYEC”; // EBCDIC eye catcher for control block
      • #pragma convert(pop)

Python calling C functions – passing structures

I’ve written how you can pass simple data from Python to a C function, see Python calling C functions.

This article explains how you can pass structures and point to buffers in the Python program. it extends Python calling C functions. It allows you to move logic from the C program to a Python program.

Using complex arguments

The examples in Python calling C functions were for using simple elements, such as Integers or strings.

I have a C structure I need to pass to a C function. The example below passes in an eye catcher, some lengths, and a buffer for the C function to use.

The C structure

typedef struct querycb {                                                         
char Eyecatcher[4]; /* Eye catcher offset 0 */
uint16_t Length; /* Length of the block 4 */
char Rsvd1[1]; /* Reserved 6 */
uint8_t Version; /* Version number 7 */
char Flags[2]; /* Flags 8 */
uint16_t Reserved8; // 10
uint32_t Count; // number returned 12
uint32_t lBuffer; // length of buffer 16
uint32_t Reservedx ; // 20
void *pBuffer; // 24
} querycb;

The Python code

# create the variables
eyec = "EYEC".encode("cp500") # char[4] eye catcher
l = 32 # uint16_t
res1 = 0 # char[1]
version = 1 # uint8_t -same as a char
flags = 0 # char[2]
res2 = 0 # uint16_t
count = 0 # uint32_t
lBuffer = 4000 # uint32_t
res3 = 0 # uint32_t
# pBuffer # void *
# allocate a buffer for the C program to use and put some data
# into it
pBuffer = ctypes.create_string_buffer(b'abcdefg',size=lBuffer)
# cast the pBuffer so it is a void *
pB = ctypes.cast(pBuffer, ctypes.c_void_p)
# use the struct.pack function. See @4shbbhhiiiP below
# @4 is 4 bytes, the eye catcher
# h half word
# bb two char fields res1, and version
# hh two half word s flags and res2
# iii three integer fields. count lBuffer and res3
# P void * pointer
# Note pB is a ctype, we need the value of it, so pB.value
p = pack("@4shbbhhiiiP", eyec,l,res1,version,flags,
res2,count,lBuffer,res3,pB.value)

#create first parm
p1 = ctypes.c_int(3) # pass in the integer 3 as an example
# create second parm
p2 = ctypes.cast(p, ctypes.c_void_p)

# invoke the function

retcode = lib.conn(p1,p2)

The C program

int conn(int * p1, char * p2) 
// int conn(int max,...)
{
typedef struct querycb {
char Eyecatcher[4]; /* Eye catcher 0 */
uint16_t Length; /* Length of the block 4 */
char Rsvd1[1]; /* Reserved 6 */
uint8_t Version; /* Version number 7 */
char Flags[2]; /* Flags 8 */
uint16_t Reserved8; // 10
uint32_t Count; // number returned 12
uint32_t lBuffer; // length of buffer 16
uint32_t Reservedx ; // 20
void *pBuffer; // 24
} querycb;

querycb * pcb = (querycb * ) p2;

printf("P1 %i\n",*p1);
printHex(stdout,p2,32);
printf("Now the structure\n")
printHex(stdout,pcb -> pBuffer,32);
return 0 ;
}

The output

P1 3
00000000 : D8D9D7C2 00200001 00000000 00000000 ..... .......... EYEC............
00000010 : 00000FA0 00000000 00000050 0901BCB0 ...........P.... ...........&....
Now the structure
00000000 : 61626364 65666700 00000000 00000000 abcdefg......... /...............
00000010 : 00000000 00000000 00000000 00000000 ................ ................

Where

  • EYEC is the passed in eye catcher
  • 00000FA0 is the length of 4000
  • 00000050 0901BCB0 is the 64 address of the structure
  • abcdefg is the data used to initialise the buffer

Observations

It took me a couple of hours to get this to work. I found it hard to get the cast, and the ctype…. functions to work successfully. There may be a better way of coding it, if so please tell me. The code works, which is the objective – but there may be better more correct ways of doing it.

Benefits

By using this technique I was able to move code from my C program to set up the structure needed by the z/OS service into C. My C program was just parse input parameters, set up the linkage for the z/OS service, and invoke the service.

If course I did not have the constants available from the C header file for the service, but that’s a different problem.

C calling an “assembler” function, setting the high order bit on, and passing parameters.

Since days of old when knights were bold, the standard parameter list to call an assembler function was to pass the addresses of the parameters, and set on the top bit of the address for the last address.
This way the called function knows how many parameters have been passed, and you do not need to pass a parameter count.

Setting the high order bit on, for the last parameter

I had to ask for help to remind me how to do it from C, so I could call “Assembler” functions.

You can get C to do this using

#pragma linkage(IRRSPK00 ,OS)

Example

The syntax of the routine from the RACF callable services documentation is

CALL IRRSPK00 (Work_area,
ALET, SAF_return_code,
ALET, RACF_return_code,
ALET, RACF_reason_code,
ALET, Function_code,
Option_word,
Ticket_area,
Ticket_options,
Ticket_principal_userid,
Application_Id
)

Here is part of my C program.

#pragma linkage(IRRSPK00 ,OS)
...
long SAF_RC,RACF_RC,RACF_RS;
SAF_RC=0 ;
long ALET = 0;
// ticket options needs special treatment, see below
int Ticket_options = 1;
int * pTO = & Ticket_options;

rc=IRRSPK00(
&work_area,
&ALET , &SAF_RC,
&ALET , &RACF_RC,
&ALET , &RACF_RS,
&ALET ,&Function_code,
&Option_word,
&ticket, // length followed by area
&pTO,
&userid,
&appl
);

If you use #pragma linkage(IRRSPK00 ,OS) it sets on the high order bit. You pass the address of the parameters. I just used &variable, there are other ways.

Passing variables

Most of the parameters are passed by address for example &ALET inserts the address of the variable, conforming to the z/OS standards.

There is a field Ticket_principal_userid which is the name of a 10-byte area that consists of a 2-byte length field followed by the userid id for whom a PassTicket operation is to be performed followed by an 8-byte PassTicket field.

I defined a structures for each variable like

struct {
short length;
char value[8];
} ticket;

In the program I used &ticket.

Ticket option

The documentation says

Ticket_options: The name of a fullword containing the address of a binary bit string that identifies the ticket-specific processing to be performed.

It took me a while to understand what this meant. I had to use

int Ticket_options = 1; 
int * pTO = & Ticket_options;

and use it

int Ticket_options = 1; 
int * pTO = & Ticket_options;
...
&ticket, // length followed by area
&pTO,

Whoops R_GenSec (IRRSGS00 or IRRSGS64): Generic security API interface

I had great problems getting this to work. The documentation said

The address double words from 31 bit callers should have the first word filled with
zeros and the second word filled with the 31 bit address. Sub-parameter addresses will be in the format of the AMODE of the caller.

I do not know what this means. When I coded it as expected I got

CEE3250C The system or user abend S0E0 R=00000029

Which means invalid ALET supplied.

I converted the program to 64 bit and it still failed!

Getting into supervisor mode and other hard things in C, is easy.

Some functions in z/OS need a user to be in a privileged state such as key zero or supervisor state. Writing the code in assembler has been pretty easy, but writing it in a C program has been hard.

For example you could write a function in assembler, and call it. This has the cross language challenges.

I recently found an easy way – just reuse some code from Zowe. This is open source, so you need to follow the license

This program and the accompanying materials are made available under the terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at https://www.eclipse.org/legal/epl-v20.html

SPDX-License-Identifier: EPL-2.0

Copyright Contributors to the Zowe Project.

The code uses __asm() Inline assembly statements (IBM extension).

The functions are

  • ddnameExists
  • wtoMessage
  • wtoPrintf3
  • atomicIncrement compare and swap
  • testauth
  • extractPSW
  • supervisorMode or back to problem state
  • setKey
  • getExternalSecurityManager
  • getCVT
  • getATCVT
  • getIEACSTBL
  • getCVTPrefix
  • getECVT
  • getTCB
  • getSTCB
  • getOTCB
  • getASCB
  • getASXB
  • getASSB
  • getJSAB
  • getCurrentACEE
  • getFirstChildTCB
  • getParentTCB
  • getNextSiblingTCB
  • resolveSymbolBySyscall Input: A symbol starting with & and not ending with .
  • resolveSymbol Input: A symbol starting with & and not ending with .
  • lots of saf functions see here
  • loadByName
  • getDSAB Data Set Association Block
  • isCallerLocked
  • isCallerCrossMemory

Some of these need to be APF protected, so although it is easy to use the above code, you may still need to get the load library APF authorised, and the code approved.


For example

Get into supervisor state

The code here.

int supervisorMode(int enable){
// Use MODESET macro for requests
int currentPSW = extractPSW();
if (enable){
if (currentPSW & PROBLEM_STATE){
__asm(ASM_PREFIX
" MODESET MODE=SUP \n"
:
:
:"r0","r1","r15");
return TRUE;
} else{
return FALSE; /* do nothing, tell caller no restore needed */
}
} else{
if (currentPSW & PROBLEM_STATE){
return TRUE; /* do nothing, tell user was in problem state */
} else{
__asm(ASM_PREFIX
" MODESET MODE=PROB \n"
:
:
:"r0","r1","r15");
return FALSE;
}
}
}

To compile it I used

// SET LOADLIB=COLIN.LOAD 
//DOCLG EXEC PROC=EDCCB,INFILE='ADCD.C.SOURCE(C)',
// CPARM='OPTF(DD:COPTS)'
//* CPARM='LIST,SSCOMM,SOURCE,LANGLVL(EXTENDED)'
//COMPILE.ASMLIB DD DISP=SHR,DSN=SYS1.MACLIB
//COMPILE.COPTS DD *
LIST,SOURCE
aggregate(offsethex) xref
SEARCH(//'ADCD.C.H',//'SYS1.SIEAHDR.H')
TEST
ASM
RENT ILP32 LO
OE
NOMARGINS EXPMAC SHOWINC XREF
LANGLVL(EXTENDED) sscom dll
DEFINE(_ALL_SOURCE)
DEBUG
/*
...

You need to specify

  • //COMPILE.ASMLIB for the the assembler macro libraries.
  • and the compiler option ASM which enables inlined assembly code inside C/C++ programs.

I was all so easy, once I had been told about it.

How do I create a header file from an assembler macro?

You can easily do this using standard JCL

//COLINZP  JOB 1,MSGCLASS=H 
// JCLLIB ORDER=CBC.SCCNPRC
//DSECT EXEC PROC=EDCDSECT,
// INFILE=COLIN.C.SOURCE(ASMMAC2),
// OUTFILE=COLIN.C.H.VB(TESTASM),
// DPARM='EQU(BIT)'
//ASSEMBLE.SYSLIB DD
// DD DISP=SHR,DSN=HLA.SASMMAC1
//ASSEMBLE.SYSIN DD *
DUMMY DSECT
ABC DS 10F
END
//DSECT.EDCDSECT DD SYSOUT=*
/*

This produced in //DSECT.EDCDSECT

#pragma pack(packed)                 

struct dummy {
int abc[10];
};

#pragma pack(reset)

You can point //DSECT.EDCDSECT to where you want your header file stored.

The syntax of the DPARM is defined here.

The comment option

By default comments are produced. One comment was

short int adplasid; /* Address space identifier @P2C */

which went past column 75.
With NOCOMMENT it produced

short int adplasid;

and all the lines were less than 72 characters long.

With COMMENT(@) it ignores anything beyond the specified string

short int adplasid; /* Address space identifier */

But some lines were still longer than 75 characters.

Using the default COMMENT

This worked first time – ish. When I tried to use the generated header file, it failed with a syntax error, because comments had wrapped over line boundary. I had to edit the output and remove the comments causing the problem.

Having fixed the obvious comment wraps, I then found comments extended into columns 72-80.

This meant, as far as C was concerned, the comment was still open, so the field on the next line was taken as part of the comment, and I was getting messages like

CCN3022 "adplserv" is not a member of "struct abdpl". 

I edited the file and fixed up these comments.

Plan b) was to specify the option NOCOMMENT and no comments are produced at all.

It could not find a macro…

The DSECT I was trying to generate referenced BIT0. My challenge was to find where this was defined!

  • If you use ISPF 3.4 on SYS1.MACLIB you can use the command SRCHFOR BIT0 and it searches all the members for the specified value. Tab to the “Prompt” heading, and press enter. The members with the value will be at top of the list.
  • You can use ISPF 2, and specify ‘SYS1.MACLIB(IEZ*)’ to display only a subset of the members – beginning with IEZ. Tab to the “prompt” heading, as above
  • You can use ISPF 3.15 Extended Search-For Utility. Specify DS name ‘SYS1.MACLIB’, and * for all members. Specify your search argument and press enter. You get one file with every occurrence it found and the member name. This was the easiest way for looking for what wanted.

The definition I wanted was not in SYS1.MACLIB, but I found it in SYS1.AMODGEN.

Calling C from Rexx – overview

Rexx is a very flexible language. You can call other Rexx programs, you can call z/OS functions, or you can call functions written in C or assembler.

Programming a C function was not straight forward because it involved twisting and bending to get it to work – one of the downsides of being flexible.

See

What options are there?

You can use

  • call function(p1,p2,p3) This gets passed an array of parameters, and has to return data in and an EVALBLOCK. See here.
  • address link program “p1 p2 p3” where “p1 p2 p3” is a literal string which is passed to your program
  • address linkmvs program “p1 p2 p3” where p1, p2, p3 are variable names, whose values are substituted before being passed to the program. If the parameter is not a variable – the parameter is passed through as is. The program gets passed a list
    • length p1 value, p1 value
    • length p2 value, p2 value
    • length p3 value, p3 value
  • address linkpgm program “p1 p2 p3” where p1, p2, p3 is a variable names, whose values are substituted before being passed to the program. If the parameter is not a variable – the parameter is passed through as is. The program gets passed a list of (null terminated values)
  • p1 value
  • p2 value
  • p3 value

If you have code

a = "COLIN A"
address linkpgm CPQUERY "A B C"

The parameters received by the program are

  • “COLIN A”
  • “B”
  • “C”

If you pass a string with hex value in it – it may have a 0x00 – which would look like an end of string. This means the linkpgm is not a good interface for passing non character data.

Passing back values

With address link you can pass back a return code. You can also use Rexx services to get and set variables in the calling rexx programs.

With address linkmvs and address linkpgm you can update the values passed in (this is non trivial). You can set a return code and use Rexx services to get and set variables in the calling rexx programs.

Which should I use?

address link

The simplest, where you have greatest control is address link. Your program gets passed a string which it can process.

address linkmvs and address linkpgm

These are similar, and you need to be careful.

A = "COLINA" 
B = ""
P = "A B C"
(1) address linkmvs CPLINKM "A B C "
(2) address linkmvs CPLINKM "P"
(3) address linkmvs CPLINKM P

With the above, my program produces.

(1)
parm 0 >COLINA<
parm 1 is Null
parm 2 >C<

(2)
parm 0 >A B C<

(3)
parm 0 >COLINA<
parm 1 is Null
parm 2 >C<

If you are using linkpgm or linkmvs, the parameter specified will be substituted (if there is a variable with the same name).

my_stem. = "used for variables"
address linkmvs CPMVS "my_stem. "

If you want to specify a stem to your program so you can set my_stem.0, my_stem.1 etc you will not be using the value you specify, it will be used for variables.0 etc.

Using

address linkmvs CPLINKM "A 'B' C "

gives return code -2.

The return code set in RC may be -2, which indicates that processing of the variables was not successful. Variable processing may have been unsuccessful because the host command environment could not: Perform variable substitution before linking to or attaching the program.

Calling C from Rexx – writing the program.

You can write a Rexx external function in C, have it process the parameters, and return data.

I found writing a program to do this was non trivial, and used bits of C I was not familiar with.

See Calling C from Rexx – accessing variables.

If you use the address… environment, the C program does not pass parameters using the normal “main” interface;

int main( int argc, char **argv ){}

because the parameters are passed in via register 1, and you need a different technique.

Header files

There are header files in SYS1.SIEAHDR.H for the Rexx facilities.

#include <irxefpl.h> // REXX External Functions Parameter List 
#include <irxargtb.h> // REXX Argument Table control block mapping
#include <irxshvb.h> // REXX Shared Variable Request Block
#include <irxenvb.h> // REXX Environment Block
#include <irxexte.h> // REXX access to shared variables etc

Program for address link …

With this, the parameters are passed in one string. The program is given the address and the length of the string.

#pragma runopts(plist(os)) 
struct plistlink {
char ** pData;
int * len;
};
int main() {
struct plistlink * pLink = (struct plistlink *)__R1;
int k = * pLink -> len;
char * pc = *pLink -> pData;
printf("plistlink length %i %.*s\n",k,k,pc);
return 0;
}

The important things to note here are

  • #pragma runopts(plist(os)) this says that the parameters are in standard z/OS parameter format based off register 1.
  • int main() this defines the entry point, and sets up the C environment.
  • __R1 is a special #define which returns the value of register one on entry to the routine.
  • struct plistlink * pLink = (struct plistlink *)__R1; defines the input parameter list.

Program for address linkmvs…

Thanks to David Crayford for his assistance in this.

With address linkmvs the specified parameters are treated as variables and their values substituted. If a variable does not exist with the name, the value is passed as-is.

On entry register 1 points to a list of pointers to data. The last element in the list has the top bit on. Process the list until the top bit is on. For example for the parameter list with values “AAA BB C” (which are not substituted)

  • addr1 -> 0X0003″AAA”
  • addr2 -> 0X0002″BB”
  • *addr3 -> 0x001″C”

where * has the top bit on.

If there are no parameters the parameter list is

  • *addr -> 0x0000…
#pragma runopts(plist(os)) 
#define EOL(a) (((unsigned int)a) & 0x80000000)

struct plist {
short len;
char parm[0];
};
int main() {
for ( i = 0; ; i++ ) {
struct plist *p = __osplist[i];
if ( p->len ==0 ) printf("parm %i is Null\n",i);
if ( p->len > 0 )
printf( "parm %i >%.*s<\n",i, p->len, p->parm );
if ( EOL(__osplist[i]) ) break;
}

Where __osplist is a special #define for the parameters based off register 1.

Return code

In both cases you can use the C return … to return an integer value.