You can write external functions for Python in C. For example I wrote one which can do a WTO and write to the system logs.
Creating this external function is not difficult, there are just several things to do.
High level view
For an external function called zconsole, there is a DLL (shared object) with name zconsole.so . Within this is an entrypoint label PyInit_zconsole.
PyInit_zconsole is a C function which returns a Python Object with information about the entry points within the DLL.
It can also define additional information such as a Python string “__doc__” which can be used to describe what the function does.
You can use the Python statement
print(dir(zconsole))
to give information about the module, the entry points, the module description, and additional fields like version number.
Conceptually the following are needed
- The entry point PyInit_xxxxxx returns a PyModuleDef object
- The PyModuleDef contains
- The name of the module
- A description of the module
- A pointer to the module functions
- The module functions contain, for each function
- The function name as used in the Python program
- A pointer to the C function
- Specification of the parameters of the function
- A description of the function
Because C needs something to be defined before it is used,the normal layout of the source is
- C functions
- Module functions definitions (which refer to the C functions)
- PyModuleDef which refers to the Module Functions
- The entry point which refers to the PyModuleDef
The initial entry point
This creates a Python object from the Python Modules Definitions, and passes it back to Python.
PyMODINIT_FUNC PyInit_zconsole(void) {
PyObject *m;
/* Create the module and add the functions */
m = PyModule_Create(&console_module);
return m;
}
Python Module Definitions
static char console_doc[] =
"z Console interface for Python";
static struct PyModuleDef console_module = {
PyModuleDef_HEAD_INIT,
"console",
console_doc,
-1,
console_methods
};
Where
- “console” is a short description. For the above if you use dir(zconsole) it gives __name__ console
- console_doc refers to the string above which is a description of the module (defined above it)
- console_methods define the C entry points – see below.
Methods
The list of functions must end in a NULL entry. The code below defines Python functions acb, taskinfo, and cancel. You can pass the description as a constant string or as a char * variable.
char * console_acb_doc[] = "...";
char * taskinfo_doc = "get the ASCB, TCB and TCBTTIME ";
....
static struct PyMethodDef console_methods[] = {
{"acb", (PyCFunction)console_acb,METH_KEYWORDS | METH_VARARGS, console_acb_doc},
{"taskinfo", (PyCFunction)console_taskinfo,METH_KEYWORDS | METH_VARARGS, taskinfo_doc},
{"cancel", (PyCFunction)console_cancel,METH_KEYWORDS | METH_VARARGS, "Cancel the subtask"},
{NULL, (PyCFunction)NULL, 0, NULL} /* sentinel */
};
A C function
This C function is passed the positional variable object, and keyword object, because of the “METH_KEYWORDS | METH_VARARGS” specified in the methods above. See below,
static PyObject *console_taskinfo(PyObject *self, PyObject *args, PyObject *keywds ) {
PyObject *rv = NULL; // returned object
...
// build the return value
rv = Py_BuildValue("{s:s,s:s,s:s,s:l}",
"jobname",jobName,
"ascb", cASCB,
"tcb", cTCB,
"tcbttime", ttimer);
if (rv == NULL)
{
PyErr_Print();
PyErr_SetString(PyExc_RuntimeError,"Py_BuildValue in taskinfo");
printf(" Py_BuildValue error in taskinfo\n");
}
return rv;
}
Passing parameters from Python to the C functions.
In Python you can pass parameters as positional variables or as a list of name=value.
Passing a list of variables.
For example in a Python program, call an external function where two parameters are required.
result = zconsole.acb(ccp,[exit_flag])
In the function definition use
static struct PyMethodDef console_methods[] = {
{"acb", (PyCFunction)console_acb, METH_VARARGS, console_cancel_doc},
{NULL, (PyCFunction)NULL, 0, NULL} /* sentinel */
};
and use
static PyObject *console_acb(PyObject *self, PyObject *args) {
PyObject * method;
PyObject * p1 = NULL;
if (!PyArg_ParseTuple(args,"OO", // two objects
&method, // function
&p1 )// parms
)
{
PyErr_SetString(PyExc_RuntimeError,"Problems parsing parameters");
return NULL;
}
...
}
Using positional and keyword parameters
For example in a Python program
rc = zconsole.console2("from console2",routecde=14)
“from console2” is a positional variable, and routecde=14 is a keyword variable.
The function definition must include the METH_KEYWORDS parameter to be able to process the keywords.
{"console2", (PyCFunction)console_console2,
METH_KEYWORDS | METH_VARARGS,
console_put_doc},
The C function needs
static PyObject *console_console2(PyObject *self,
PyObject *args,
PyObject *keywds
) {
...
}
You specify a list of keywords (which much include a keyword for positional parameters)
static PyObject *console_console2(PyObject *self,
PyObject *args,
PyObject *keywds
) {
char * p = "";
Py_ssize_t lMsg = 0;
// preset these
int desc = 0;
int route = 0;
static char *kwlist[] = {"text","routecde","descr", NULL};
// parse the passed data
if (!PyArg_ParseTupleAndKeywords(args, keywds,
"s#|$ii",
kwlist,
&p , // message text
&lMsg , // message text
&route, // i this variable is an array
&desc , // i this variable is an array
))
{
// there was a problem
return NULL;
}
In the static char *kwlist[] = {“text”,”routecde”,”descr”, NULL}; you must specify a parameter for the positional data (text).
In the format specification above
- s says this is a string
- # save the length
- | the following are optional
- $ end of positional
- i an integer parameter
- i another integer parameter.
You should initialise all variable to a suitable value because a variable is gets a value only of the relevant keyword (or positional) is specified.
How do I print an object from C?
If you have got an object (for example keywds), you can print it from a C program using
PyObject_Print(keywds,stderr,0);
Advanced configuration
You can configure additional information for example create a special exception type just for your code. You can create this and use it within your C program.
#define Py23Text_FromString PyUnicode_FromString // converts C char* to Py3 str
static PyObject *ErrorObj;
PyMODINIT_FUNC PyInit_zconsole(void) {
PyObject *m, *d;
/* Create the module and add the functions */
m = PyModule_Create(&console_module);
/* Add some symbolic constants to the module */
d = PyModule_GetDict(m);
PyDict_SetItemString(d, "__doc__", Py23Text_FromString(console_doc));
PyDict_SetItemString(d,"__version__", Py23Text_FromString(__version__));
ErrorObj = PyErr_NewException("console.error", NULL, NULL);
PyDict_SetItemString(d, "console.error", ErrorObj);
return m;
}
- The d = PyModule_GetDict(m) returns the object dict for the function (you can see what is in the dict by using print(dir(zconsole))
- PyDict_SetItemString(d, “__doc__”, Py23Text_FromString(console_doc)); Creates a unicode string from the console_doc string, and adds it to the dict with name “__doc__”
- It also adds an entry for the version.
- You could also define constants that the application might use.
- The ErrorObj creates a new exception called “console.error”. It is added to the dict as “console.error”. This can be used to report a function specific error. For example
- PyErr_Format(ErrorObj, “%s wrong size. Given: %lu, expected %lu”, name, (unsigned long)given, (unsigned long)expected); return NULL;
- PyErr_SetString(ErrorObj, “No memory for message”); return NULL;
How do I compile the C code?
I used a shell script to make it easier to compile. The setup3.py does any Python builds
touch /u/tmp/console/console.c
* generate the assembler stuff (outside of setup
as -d cpwto.o cpwto.s
as -a -d qedit.o qedit.s 1> qedit.lst
export _C99_CCMODE=1
python3 setup3.py build bdist_wheel 1>a 2>b
* copy the module into the Python Path
cp ./build/lib.os390-27.00-1090-3.8/console/zconsole.so .
* display the output. b should be empty
oedit a b
The setup3 Python program is in several logical parts
# Basic imports
import setuptools
from setuptools import setup, Extension
import sysconfig
import os
os.environ['_C89_CCMODE'] = '1'
from setuptools.command.build_ext import build_ext
from setuptools import setup
version = '1.0.0'
Override the build – so unwanted C compile options can be removed
class BuildExt(build_ext):
def build_extensions(self):
print(self.compiler.compiler_so)
if '-fno-strict-aliasing' in self.compiler.compiler_so:
self.compiler.compiler_so.remove('-fno-strict-aliasing')
if '-Wa,xplink' in self.compiler.compiler_so:
self.compiler.compiler_so.remove('-Wa,xplink')
if '-D_UNIX03_THREADS' in self.compiler.compiler_so:
self.compiler.compiler_so.remove('-D_UNIX03_THREADS')
super().build_extensions()
setup(name = 'console',
version = version,
description = 'z/OS console interface. Put, and respond to modify and stop request',
long_description= 'provide interface to z/OS console',
author='Colin Paice',
author_email='colinpaice3@gmail.com',
platforms='z/OS',
package_dir = {'': '.'},
packages = ['console'],
license='Python Software Foundation License',
keywords=('z/OS console modify stop'),
python_requires='>=3',
classifiers = [
'Development Status :: 4 - Beta',
'License :: OSI Approved :: Python Software Foundation License',
'Intended Audience :: Developers',
'Natural Language :: English',
'Operating System :: OS Independent',
'Programming Language :: C',
'Programming Language :: Python',
'Topic :: Software Development :: Libraries :: Python Modules',
],
cmdclass = {'build_ext': BuildExt},
ext_modules = [Extension('console.zconsole',['console.c'],
include_dirs=["//'COLIN.MQ930.SCSQC370'","."],
extra_compile_args=["-Wc,ASM,SHOWINC,ASMLIB(//'SYS1.MACLIB')",
"-Wc,LIST(c.lst),SOURCE,NOWARN64,XREF","-Wa,LIST,RENT"],
extra_link_args=["-Wl,LIST,MAP,DLL","/u/tmp/console/qedit.o",
"/u/tmp/console/cpwto.o",
],
)]
)
This code…
- The cmdclass = {‘build_ext’: BuildExt}, statement tells it to use the function I had defined.
- Uses the C header files from MQ, using dataset COLIN.MQ930.SCSQC370
- The C program uses the __asm__ statement to create inline assembler code. The macros libraries are for this assembler source are defined with ASMLIB(//’SYS1.MACLIB’)”,
- The C listing is put into c.lst.
- The bind options are LIST,MAP,DLL
- The generated binder statement is /bin/xlc build/temp.os390-27.00-1090-3.8/console.o -o build/lib.os390-27.00-1090-3.8/console/zconsole.so….
- The binder statements used include the assembler modules generate in the shell script are
INCLUDE C8920
ORDER CELQSTRT
ENTRY CELQSTRT
INCLUDE ‘./build/temp.os390-27.00-1090-3.8/console.o’
INCLUDE ‘/u/tmp/console/qedit.o‘
INCLUDE ‘/u/tmp/console/cpwto.o’
INCLUDE ‘/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/config-3.8/libpython
Some gotcha’s to look out for
The code is generated as 64 bit code.
Check you have the correct variable types.
You may get messages about conversion from 64 bit values to 31 bit values.
The code is generated with the ASCII option.
This means
printf(“Hello world\n”); will continue to work as expected. But a blank is 0x20, not 0x40 etc. so be careful when using hexadecimal.
Pass a character string between Python and z/OS
You will have convert from ASCII to EBCDIC, and vice versa when going back. For example copy the data from Python into a program variable, then convert and use it.
memcpy(pOurBuffer,pPythonData,lPythonData);
__a2e_l(pOurBuffer,lPythonData) ;
...
Returning character data from z/OS back to Python
If the data is in a z/OS field ( rather than a variable in your program), you will need to copy it and covert it. You can pass a null terminate string back to Python, or specify a length;
The code below uses Py_BuildValue to creates a Python dictionary {“jobname”: “COLINJOB”).
memcpy(&returnJobName,zOSJobName,8);
returnJobName[8] = 0; // null terminator
__e2a_l(&returnJobName,8);
rv = Py_BuildValue("{s:s}","jobname",returnJobName); .. null terminated
// or
rv = Py_BuildValue("{s:s#}","jobname",returnJobName,8); // specify length
ColinPaice 