Creating a C external function for Python, an easier way

I wrote about my first steps in creating a C extension in Python. Now I’ve got more experience, I’ve found an easier way of compiling the program and creating a load module. It is not the official way – but it works, and is easier to do!

The traditional way of building a package is to use the setup.py technique. I’ve found just compiling it works just as well (and is slighly faster). You still need the setup.py for building Python source.

I set up a cp4.sh file

name=zos 
pythonSide='/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/config-3.8/libpython3.8.x' 
export _C89_CCMODE=1 
p1=" -DNDEBUG -O3 -qarch=10 -qlanglvl=extc99 -q64" 
p2="-Wc,DLL -D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS" 
p2="-D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS" 
p3="-D_OPEN_SYS_FILE_EXT                     -qstrict          " 
p4="-Wa,asa,goff -qgonumber -qenum=int" 
p5="-I//'COLIN.MQ930.SCSQC370' -I. -I/u/tmp/zpymqi/env/include" 
p6="-I/usr/lpp/IBM/cyp/v3r8/pyz/include/python3.8" 
p7="-Wc,ASM,EXPMAC,SHOWINC,ASMLIB(//'SYS1.MACLIB'),NOINFO " 
p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,FLAG(W),XREF,AGG -Wa,LIST,RENT" 
/bin/xlc $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8  -c $name.c -o $name.o  -qexportall -qagg -qascii 
l1="-Wl,LIST=ALL,MAP,XREF     -q64" 
l1="-Wl,LIST=ALL,MAP,DLL,XREF     -q64" 
/bin/xlc $name.o  $pythonSide  -o $name.so  $l1 1>a 2>b 
oedit a 
oedit b 

This shell script creates a zos.so load module in the current directory.

You need to copy the output load module (zos.so) to a directory on the PythonPath environment variable.

What do the parameters mean?

Many of the parameters I blindly copied from the setup.py script.

• name=zos
  • This parametrizes the script, for example $name.c $name.o $name.so
• pythonSide=’/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/config-3.8/libpython3.8.x’
  • This is where the python side deck, for resolving links the to functions in the Python code
• export _C89_CCMODE=1
  • This is needed to prevent the message “FSUM3008 Specify a file with the correct suffix (.c, .i, .s,.o, .x, .p, .I, or .a), or a corresponding data set name, instead of -o./zos.so.”
• p1=” -DNDEBUG -O3 -qarch=10 -qlanglvl=extc99 -q64″
  • -O3 optimization level
  • -qarch=10 is the architectural level of the code to be produced.
  • –qlanglvl=extc99 says use the C extensions defined in level 99. (For example defining variables in the middle of a program, rather that only at the top)
  • -q64 says make this a 64 bit program
• p2=”-D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS”
  • The C #defines to preset
• p3=”-D_OPEN_SYS_FILE_EXT -qstrict “
  • -qstrict Used to prevent optimizations from re-ordering instructions that could introduce rounding errors.
• p4=”-Wa,asa,goff -qgonumber -qenum=int”
  • -Wa,asa,goff options for any assembler compiles (not used)
  • -qgonumber include C program line numbers in any dumps etc
  • -qenum=int use integer variables for enums
• p5=”-I//’COLIN.MQ930.SCSQC370′ -I. -I/u/tmp/zpymqi/env/include”
  • Where to find #includes:
  • the MQ libraries,
  • the current working directory
  • the header files for my component
• p6=”-I/usr/lpp/IBM/cyp/v3r8/pyz/include/python3.8″
  • Where to find #includes
• p7=”-Wc,ASM,EXPMAC,SHOWINC,ASMLIB(//’SYS1.MACLIB’),NOINFO “
  • Support the use of __ASM().. to use inline assembler code.
  • Expand macros to show what is generated
  • List the data from #includes
  • If using__ASM__(…) where to find assembler copy files and macros.
  • Do not report infomation messages
• p8=”-Wc,LIST(c.lst),SOURCE,NOWARN64,FLAG(W),XREF,AGG -Wa,LIST,RENT”
  • For C compiles, produce a listing in c.lst,
  • include the C source
  • do not warn about problems with 64 bit/31 bit
  • display the cross references (where used)
  • display information about structures
  • For Assembler programs generate a list, and make it reentrant
• /bin/xlc $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8 -c $name.c -o $name.o -qexportall
  • Compile $name.c into $name.o ( so zos.c into zos.o) export all entry points for DLL processing
• L1=”-Wl,LIST=ALL,MAP,DLL,XREF -q64″
  • bind pararameters -Wl, produce a report,
  • show the map of the module
  • show the cross reference
  • it is a 64 bit object
• /bin/xlc $name.o $pythonSide -o $name.so $L1 1>a 2>b
  • take the zos.o, the Python side deck and bind them into the zos.so
  • pass the parameters defined in L1
  • output the cross reference to a and errors to b
• oedit a
  • This will have the map, cross reference and other output from the bind
• oedit b
  • This will have any error messages – it should be empty

Notes:

• -qarch=10 is the default
• the -Wa are for when compiling assembler source eg xxxx.s
• –qlanglvl=extc99. EXTENDED may be better than extc99.
• it needs the -qascii to work with Python.