Running a Python package on z/OS is pretty easy. You just have to remember to set up the environment. This one of those little tasks you do not do very often, and forget how to do it.
When you enter OMVS your .profile shell is executed (if the file exists).
Does the file exist ?
You can use the ls command to see if the file exists
ls -altr ~/.profile
where the -a option says display all the files – including those beginning with ‘.’. By default these are omitted.
-rwx------ 1 BPXROOT TSOUSER 297 May 18 08:46 /u/colin/.profile
Note the x attribute; it means the file can be executed. You can use the following command to set this attribute.
chmod +x ~/.profile
My .profile file has
# set up Python path
# set up MQ Libraries
# set up PYTHONPATH to point to where zpymqi is
When my Python program has import pymqi, Python will look in directory /u/colin/pymqi
I struggled to decode PCF output; for example decode PCF type 1203 and its value 20. MQ provide most of what you need, there is just one little link in the chain which is missing
In the MQ provided CMQSTRC header file are routines for converting the values to strings
char *MQSYSP_STR (MQLONG v)
case 0: c = "MQSYSP_NO"; break;
case 1: c = "MQSYSP_YES"; break;
case 2: c = "MQSYSP_EXTENDED"; break;
case 10: c = "MQSYSP_TYPE_INITIAL"; break;
case 11: c = "MQSYSP_TYPE_SET"; break;
case 12: c = "MQSYSP_TYPE_LOG_COPY"; break;
case 13: c = "MQSYSP_TYPE_LOG_STATUS"; break;
case 14: c = "MQSYSP_TYPE_ARCHIVE_TAPE"; break;
case 20: c = "MQSYSP_ALLOC_BLK"; break;
so if you know this your type is a System Parameter, you can use MQSYSP_STR(20) to get back the data item MQSYSP_ALLOC_BLK.
The bit that is missing is the mapping between PCF type and the function call.
In GitHub I’ve created this mapping for all of the PCF types (MQMAP.h). This has for example
pWhat gets the name of the PCF data type (“Trigger_Type”)
pValue gets the value returned from the MQ provided function(“MQTT_DEPTH”)
pPValue gets the prettified value, with the prefix removed, and the remained made more readable(“Depth”)
This will allow you to run along PCF data and display all the data and values in a similar manner to a display command.
The data in MQMAP.h is in numerical sequence, so I can use a binary search to quickly find the mapping function. I also provide a small C function which takes the MQMAP.h file, checks it is in order and displays it, so it can be sorted.
Display your repository in a web browser, and select the “settings” icon. On the settings page, click on “pages” in the left hand window. When it displays “GitHub Pages”, go down to source and select on the “None” pull down.
Select “main” to make pages available, or none to make them unavailable.
PyDev for Eclipse Developer Resources 18.104.22.168102211157
PyDev Mylyn Integration 0.6.0
I selected the first one.
I got a pop up for a Security warning for unsigned content, I said “install anyway.”
I restarted the work bench. The restart may take a while as the plugins and features are integrated.
The Window -> Preferences should have PyDev in the list.
You can check the file associations now…
Window -> Preferences -> General -> Editors -> File Associations should have a File types: entry for *.py and the Associated Editors: section should list Python Editor as one of the options.
Window -> Preferences -> PyDev should also be populated with preferences for Editor, Interpreters, PyUnit, and ScriptingPyDev.
If you don’t set up Python via the preferences, on the first python file interaction, you may get a message that Python interpreter is not currently configured.
You can edit Python files inside the base IDz install with some of the generic file editors already installed, though they will not have Python specific language knowledge or capabilities such as content assist or syntax highlighting
To allow .py files to be edited internally, by setting the Window -> Preferences -> General -> Editors -> File Associations. Here you can choose to change the Open associated files with: option from System Editor; if none Text Editor to Text Editor as one option. Another option is in the File types: section press Add… and then in the dialog that comes up type *.py and then press OK. (If you get a dialog message that says a file type *.py already exists that is OK: this means PyDev registered as an editor which is fine. ) In the Associated editors: section press Add… and select the Internal editors option. I would recommend then choosing Generic Text Editor, Text Editor, System z LPEX Editor, or z Systems LPEX Editor but there are several other options that may work for you. Note that you can Add… more than one editor to a a single file type, at which point if you right click on the *.py files you will have an option of which editor to use. The one that is used when you double click or right click -> Open is the one you set to default, which you can adjust in this section by selection of one of the editors and then pressing the Default button. Then press Apply and Close.
Example of editing a .py file
Running the debugger
I was not able to use the debugger as it tries to run it on my work station, not on z/OS
On one of the z/OS fora I found the following good stuff from David Mingee. Three of these were new to me.
After using START command to start more than one screen, you can enter =XALL to close all screens.
In ISPF enter CMD SWAPBAR ON. This will cause a screen name to be displayed at the bottom left of your screen. There will be a screen name for each START CMD that has been done. The active screen will have an * e.g. *dslist . You can go directly to any screen listed by clicking your mouse on the desired screen name. This feature stays turned on for your next logon and until you enter SWAPBAR OFF. You can change the name of the screen using the command such as SCRNAME SDSF. By default you get names like
ISR@PRIM for the main ISPF panel
ISFPCU4 for SDSF
DSLIST for ISPF 3;4
ISRTSO for ISPF option 6 for TSO
OS390S for z/OS System Programmer Primary Option Menu
When editing a file do CC CC A to copy a range of lines AFTER line A, a new feature allows the use of CC CC AK on as many lines as wanted and then A on the last location for the copy. Lines are copied to multiple locations with one command. So do cc, cc, ak,ak,a
In ISPF use command 0 for settings then go to page 2 and change 3278 screen display to 1 for DATA vs. STD. This will automatically allow your screen to display up to 120 columns vs. only 80.
In SDSF change FIND LIMIT to 99999 vs default of 5000 as this will speed up log searches.
Use SRCHFOR command in 3.4 for pds/pdse files to find a ‘STRING’ in all members.
Use KEYS command to change your PF KEY settings, e.g. change F9 to SWAP NEXT vs. SWAP.
Use ISPF APPEND command to create a personal DSLIST of files frequently used with different DSN Qualifiers.
ISPF cut and paste. You can have multiple “clipboards”, and do ASCII or EBCDIC conversion. For example CUT .a .b BOARD2 APPEND UTF8 copies the lines labelled .a to .b to clipboard BOARD2.
I wanted to compile one of the MQ samples to run as a 64 bit program. The sample comes compiled as 31 bit.
How did I compile the sample to make it 64 bit?
There is documentation on building 64 bit applications; it provides a compile and link JCL example. However, I wanted to do a compile, bind and go.
//COLINCC JOB 1,MSGCLASS=H
//JOBLIB JCLLIB ORDER=(PP.CBC.ZOS204.SCCNPRC)
//COMPILE EXEC PROC=EDCQCBG, Needed for 64 bit compile
// LIBPRFX=CEE, This is an alias
// GPARM='M801 CP0000 1'
//COMPILE.SYSLIB DD DISP=SHR,DSN=&LIBPRFX..SCEEH.SYS.H
// DD DISP=SHR,DSN=&LIBPRFX..SCEEH.H
// DD DISP=SHR,DSN=MQM.V900.SCSQC370
//COMPILE.SYSIN DD DISP=SHR,DSN=BETACP.MP1B.JCL(CSQ4BCG1)
//COMPILE.SYS DD DISP=SHR,DSN=&LIBPRFX..SCEEH.SYS.H
//COMPILE.SYSOPTF DD *
WARN 64 RENT LP64 LO SOURCE LIST
NOMARGINS EXPMAC SHOWINC XREF
LANGLVL(EXTENDED) SSCOM DLL
//BIND.SCSQDEFS DD DISP=SHR,DSN=MQM.V900.SCSQDEFS
//BIND.SYSIN DD *
//GO.STEPLIB DD DISP=SHR,DSN=MQM.V900.SCSQANLE
// DD DISP=SHR,DSN=MQM.V900.SCSQLOAD
// DD DISP=SHR,DSN=MQM.V900.SCSQAUTH
The above JCL used
an alias (CEE) for some libraries. The CEE alias was defined as SYMBOLIC PP.ADLE370.&SYSLEVEL and RESOLVED PP.ADLE370.ZOS204
explicit High Level Qualifier PP.CBC.ZOS204 for other libraries.
If your system provides aliases for data sets, it is better to use the alias, rather than the explicit data set name (or HLQ). When the system is upgraded (to ZOS205), the symbolic will be updated from ZOS204 to ZOS205, the alias will be updated automatically, and any JCL using the alias will still work. If you used the explicit dataset name, when your system is upgraded the dataset may not be available, and you need to update your JCL.
What does an alias point to?
The easiest way I found to see what an alias points to use ISPF 3.4 to list datasets beginning with the alias, for example CEE.SCEEH, and browse it. This browsed dataset was PP.ADLE370.ZOS204.SCEEH, so the alias CEE represents PP.ADLE370.ZOS204.
You can also use a LISTCAT command. (Use ISPF 3.4 and use the I prefix command) and change the command to LISTCAT ENTRIES(CEE.* ) ALL then type EXEC on the command line to execute it.
This gives output like
LISTCAT ENTRIES(CEE.* ) ALL
ALIAS --------- CEE.SCEEBIND
IN-CAT --- ICFCAT.PLEXH.CATALOG3
DATA SET ENCRYPTION-----(NO)
You pass to mqinqmp(), a property name (or a generic like %), a buffer, the size of the buffer, the length of the data, and some other parameters (such as get first, or get next). It returns the property name, the property value ( possibly truncated) , and the true length of the property..
If your buffer is too small to contain all of the property, you get reason code MQRC_PROPERTY_VALUE_TOO_BIG, and should get the true length of the property, its name, type, and length.
This worked fine when it was a 31 bit program. When I ran it as a 64 bit program, it did not return the true length, and did not return the property name.
The documentation says
DataLength Type: MQLONG – output
This is the length in bytes of the actual property value as returned in the Value area.
If DataLength is less than the property value length, DataLength is still filled in on return from the MQINQMP call. This allows the application to determine the size of the buffer required to accommodate the property value, and then reissue the call with a buffer of the appropriate size.
The DataLength is the actual length of the property. (The value returned in the Value area is truncated at Value Length).
What should happen
After you get the MQRC_PROPERTY_VALUE_TOO_BIG reason code, you reallocate the buffer with a size of DataLength and retry the request. It should work the second time.
For me the DataLength was 0, so every time around the loop it got MQRC_PROPERTY_VALUE_TOO_BIG.
I hit this question trying to edit some Python programs on z/OS in Unix Services.
The easy answer for programs which are in EBCDIC or untagged, is just use ISPF edit, and you can use the HILITE command to highlight the constants and variables in the file. You can use this command to highlite, C, REXX, JCL, XML, assembler etc.
Example output for a C program.
Unfortunately although python supports EBCDIC source files, if you want to compile and package these files, they need to be ASCII files. The Unix Services command OEDIT will detect that the file as an ASCII file, and you can edit it, but not use the HILITE command.
To display the type of the file, use the Unix command ls -Tr *.py . This gave me
t ISO8859-1 T=on cf.py
- untagged T=off aa.py
Where cf.py is tagged as an ASCII file, an aa.py is an untagged file which defaults to EBCDIC.
Download to your work station
You can download ( FTP) these files to your work station, in binary and edit them. I use gedit on Linux. This has support built in for highlighting source.
IBM developer for z/OS.
This is an eclipse based package which allows you to edit data sets, submit jobs, debug programs etc. It has build in support for C, COBOL etc. When you right click on a file, you can select which editor to use, for example LPEX, a simple editor,a remote C/C++ editor, or an editor on your work station (gedit in my case).
For a python file (*.py), Eclipse downloads the file, and the choice is the eclipse simple text editor, or your workstation standard editor (gedit).
Pydev is an Integrate Development Environment for Python on Eclipse (IBM Developer for z/OS). I had a few problems getting it working in Eclipse. (I had to use Pydev 8.2 because of incompatible Java release problem). This provides a Python editor with highlighting, and support for “twisties” where you can hide functions etc. As an editor this worked fine for most of the things I did. There were a couple of surprises.
I could not get the debugger to work, as it tried to debug running the Python program on my laptop – not on z/OS.
Whoops who over wrote my file
With all of the non ISPF techniques there is an integrity problem. If there are two people A and B and they want to edit the file at the same time. With ISPF and ISPF
A edits the file
B is prevented from editing the file
A saves the updates
If A uses ISPF and B uses IBM developer
A edits the file
B edits the file
A saves the file
When B tries to save the file, it detects the last changed time stamp is different, and asks if you wan to overwrite it
If A uses ISPF and B uses IBM developer
A edits the file
B edits the file
B saves the file, and as the last changed time stamp has not changed, the save is successful
A saves the file, and overwrites the changes B made.
If you use FTP or similar to download the files, there is no locking, and the last person to save will overwrite any previous updates from other people.
I was using IBM Developer for z/OS (known as ID/z) to use Eclipse to access files on z/OS, and so use the Eclipse editors and other facilities. The files were displayed in a window, but did not have the handy PF7 and PF8 to scroll through the list. I also had a mixture of python source files, and other files.
I knew there was a way of filtering the displays, but it was a challenge to get this to work.
The easy bit – creating a filter
Under the Remote Systems (Window -> Show view -> Remote systems) it had my connections to z/OS systems, and a tree of my USS files.
On an subdirectory in the tree, right click -> New -> Filter. This gives
You can specify (or browse for) a subdirectory
You can either filer on file name of file type.
To filter for file type
You cannot type in the file type field. Click on Select… . You can either select from the existing list, or create your own extension. I created .py. This appears in the File types filter box as py, .
If Show files only is unticked, then the filter will displayed the directories below the path you specified as well as any matching files. If this is selected, then only the filtered file names are displayed
When you press Next it prompts you for a name.
When you get back to the Remote systems, window, you should see your filter in the list. If you open the twistie you will see the files you specified.
Changing the filter
This was initially hard because I was not familiar with the Eclipse usage.
Right click on the filter, -> Properties -> Filter Strings and you can change the information in the filter. Click Apply and Close to finish.
Extending the filter
As well as changing a filter, you can add new filter strings. Click on “New filter string”, fill in the details and select “Apply and Close”.
I could not see how to delete a filter once created.
Although you have filtered the list of files, you have all of the connections, and other displayed information. If you right click a directory, or filter and “go into” you get just that directory or filter output displayed.
Select the folder with an up arrow in it to get back.
You can have a hierarchy of filters, so I could have a set
MQ filters – with all of the filter I use for developing MQ applications
Python filters – with the filters I use for creating python program, and python unit tests
You can display or hide this through the twistie (menu button) at the top right of the Remote Systems window.
To create a new filter pool, ignore what the documentation says. Where you have “Z/OS UNIX files”, right click on it -> new -> Filter pool.
You can display more information using “Window-> Show view -> Remote Systems -> team”