Using point and shoot keys in ISPF

ISPF has had hot keys for many years – I have only just got round to using them.

At the top of ISPF screens there is usually an action bar. For example

   Menu  Utilities  Compilers  Options  Status  Help                       
 ────────────────────────────────────────────────────
                   ISPF Primary Option Menu                           
 Option ===>                                                                   

Where the actions are Menu, Utilities etc. The underlined letter is the hot key for the action.

To select Utilities, type U on the command line and press the “Actions” PF key. This will do the same as tabbing to the Utilities action bar item, and pressing enter. For me PF10 is set to Actions.

You can display your PF Key settings using the keys or pfshow command.

For people who like to use short cuts, have your command line at the top, and use the “home” key. This will get you to the command line. So you can do: home, U, PF10 and quickly get to the Utilities action bar item.

Who develops standard?

When the GUI standards came out I wondered why the standard was to put the command line at the bottom, as there was no “go to command line” key, you had to do “Home”, then shift, tab (to do back tab). This was too many keystrokes for me.

ISPF reflists provide hot lists of datasets and files.

You can set up hot lists of data sets names to make it easier to use data sets and files in Unix Services. For example if your hot list of data sets is SYS1.PROCLIB, ADCD.Z245.PROCLIB, COLIN.PROCLIB, COLIN.C.SOURCE, you can get a list of these in ISPF 3.4 format, so you can quickly move between them, rather than have to keep using ISPF 3.4 and typing in the names of the data sets.

ISPF provides a list called REFLIST with the last 30 datasets or Unix files you referenced, so when you come back from holiday and cannot remember what you were working on, then the REFLIST will show you. Recalling Those Darn Dataset Names is an interesting article.

You can have a hot list of Unix Services files.

You can use the REFLISTD command, or go into ISPF 3.4, up to the action bar, and use the pull down to select REFLIST.
By default, this gives you the a list of the last 30 data sets you used.

Example output

File View Options Help .
────────────────────────────────────────────────────────────────────────────── .
ISRPDSNL Personal Data Set List DSLIST processed .
Command ===> 
Enter a list action to perform or select a data set entry to retrieve. .
Action: S=Save A=Save As D=Delete this list E=Extended Edit L=DSLIST U=UDLIST .
.
Action   Name    Description                  Created Referenced .
L        REFLIST Last 30 referenced data sets         22/04/26 11:45 .

Select Data Set, DSLIST Level or z/OS UNIX file Volume WS 
. 'COLIN.CBT503.FILE967' 
. 'USER.Z24C.CLIST' 
. 'COLIN.SRCHML.LIST' 
. 'COLIN.ZLOGON.CLIST' 
. /u/tmp/zpymqi/code/pymqi/MQMAP.h 
. 'COLIN.CUCI.EXEC' 

The data at the bottom of the display is what is in this list. You cannot do anything with these entries. You can have datasets as well as Unix files in the list

The action only applies to the List entry itself.

If you put an L in front of the REFLIST, you get into standard ISPF 3.4 data set list – with all of the data sets from the list. You can use all of the commands you know and love from this list. You can use option U to list all of the Unix files in the list.

You can also save this list, for example use “A” to create your own list (eg COLIN).

If you want to use this list you can use the command REFACTD COLIN, or go to ISPF 3.4, action bar, REFLIST

1. Current Personal Data Set List (COLIN)
2. List of Personal Data Set Lists

Option 2 gives

ISRPLTAB EFLIST             Personal Data Set Lists                List 1 of 2
Command ===>                                                  Scroll ===> PAGE
                                                                               
Action: O=Open  A=Save As  D=Delete  E=Edit  L=DSLIST  U=UDLIST               
                                                                               
Name      Description                  Created  Referenced
_ COLIN   colin's list                 22/04/26 22/04/26 11:55
_ REFLIST Last 30 referenced data sets          22/04/26 11:57
End

This allows you to pick a list, and specify if you want a data set list, or a Unix file list.

This is really handy – but not very well documented. It feels like it has not been written for the end users but from the IBM developers view point.

Configuring ISPF for new applications.

I found an interesting ISPF extension which I wanted to try out. But I could not find any documentation on how to install it. As usual I found out even more interesting stuff trying to document it.

Logging on to TSO.

When you logon to TSO you specify a procedure such as ISPFPROC. This procedure is in the SYS1.PROCLIB concatenation.  A user can have a default procedure specified in the RACF TSO segment. A user is given access to the RACF CLASS TSOPROC. (So you use a command like RDEFINE TSOPROC NEWPROC UACC(READ) )

This procedure allocates the datasets used by TSO and ISPF, for example ISPPLIB, ISPMLIB, ISPLLIB,ISPEXEC, SYSEXEC, SYSOUT.

If you are authorised you can create your own TSO procedure, and add your application’s data sets to the JCL. Most systems do not allow this.

When this JCL is executed you can pass parameters to it, for example

//ISPFPROC EXEC PGM=IKJEFT01,REGION=0M,DYNAMNBR=200,
//             PARM=’%ISPFCL’                       

will cause the ISPFCL exec or clist to be executed.   This is usually in the SYS1.CLIST concatenation. You can specify PARM=’EXEC ”SYS1.CLIST(SPFALLOC)”’ and specify a data set.

You can use this

  1. To write a message to users
  2. Allocate additional data sets,
  3. Check the existence of a profile, and execute it.

User specific processing

With the clist specified in the PARM statement you can have code like

/* rexx */
address TSO
userid = userid()
zl =”‘”userid”.ZLOGON.CLIST'” /* so we get ‘colin.zlogon.clist’ */
if SYSDSN(zl) = “OK” then exec zlogon /* the userid.. and .clist default’ */

For my userid this would check to see if COLIN.ZLOGON.CLIST exists, and if so, it executes it.

userid.ZLOGON.CLIST

Within this data sets you can have user specific processing, for example

/* Rexx */
say "in colin.zlogon.clist"
address ispexec
"LIBDEF ISPLLIB DATASET ID('COLIN.ISPF.ISPLLIB') STACK"
"LIBDEF ISPMLIB DATASET ID('COLIN.CUCI.ISPMLIB') STACK"
"LIBDEF ISPPLIB DATASET ID('COLIN.CUCI.ISPPLIB') STACK"
"LIBDEF ISPTLIB DATASET ID('COLIN.CUCI.ISPTLIB') STACK"
address tso "ALTLIB ACTIVATE APPLICATION(CLIST)
DATASET('COLIN.CUCI.EXEC') "

This will add the datasets to the specified library, for example adds the panel library COLIN.CUCI.ISPPLIB to ISPPLIB.

You can use “LIBDEF ISPPLIB” to “pop” the definitions back to what they were before the call.

Running your own application

If you have your own panel or application you want to use, you could use the following to set up the libraries and display Mypanel

address ispexec
"LIBDEF ISPLLIB DATASET ID('COLIN.ISPF.ISPLLIB') STACK"
"LIBDEF ISPMLIB DATASET ID('COLIN.CUCI.ISPMLIB') STACK"
"LIBDEF ISPPLIB DATASET ID('COLIN.CUCI.ISPPLIB') STACK"
"LIBDEF ISPTLIB DATASET ID('COLIN.CUCI.ISPTLIB') STACK"
address tso "ALTLIB ACTIVATE APPLICATION(CLIST)
DATASET('COLIN.CUCI.EXEC') "
"SELECT PANEL(Mypanel ) NEWAPPL(MYA) PASSLIB"
/* afterwards reset things back */
address tso "ALTLIB DEACTIVATE APPLICATION(CLIST)"
"LIBDEF ISPLLIB"
"LIBDEF ISPPLIB"
"LIBDEF ISPMLIB"
"LIBDEF ISPTLIB"

What has been libdef-ed?

You can use the TSO command ISPLIBD to display the data sets that have been libdef-ed for the logical screen. Note each logical screen can have its own libdef statements – depending on the set up. See here for a description.

FTP to z/OS when your fingers forget

They say that large dinosaurs had a brain in the head, and a brain near the back legs, because it took too long for information from the feet to reach the brain. I’ve always thought that programmers have a brain in each hands so they can type when the brain is thinking of other things. The problem with this is when you do not use commands, the hand brains forget the commands, and you have to use the head brain. I experienced this trying to FTP a file to z/OS and my fingers would not type the commands I used to know and love!

The FTP daemon needs to be running, for example S FTPD.

FTP to z/OS

  • ftp 10.1.1.2
  • userid ….
  • password

Set up the environment

  • ASCII
  • BIN

Setup the dataset parameters using the site command

site cyl pri=10 sec=10 recfm=fb lrecl=80 blksize=3200 vol=(volid1,volid2)

You can specify size units

  • cyl
  • trk
  • blk

I could not see how to display the default “site” parameters, so if in doubt specify all the parameters.

Change directory ( HLQ) at the remote end

cd ‘COLIN’

Change directory at the local end

lcd /u/colin/

List directories ( HLQ) at the remote end

cd ‘COLIN.CBT’
dir

or

ls ‘COLIN.CBT.*’

Gave

125 List started OK
Volume Unit Referred  Ext Used Recfm Lrecl BlkSz Dsorg Dsname
C4USR1 3390 2022/04/22  1   45    FB    80  5600 PO    V502.FILE967.PDS
250 List completed successfully.

Send files

You can use PUT or GET; MPUT or MGET.

MPUT and MGET will prompt for each file. You can toggle the prompting using the prompt subcommand

prompt
gave
Interactive mode off.

prompt
gave
Interactive mode on.

Additional information

  • ? – displays a summary of the commands
  • help command – gives a short description of the command
  • quit – exit FTP

How do I use SFTP to ftp to a z/OS data set?

You cannot do it directly, you have to do a two step process.

Sending files to z/OS using SFTP

SFTP can send files to Unix Services Subsystem. It cannot send to data sets.

sftp colin@10.99.88.77

You can use commands like

  • cd to change directory on the remote system
  • lcd change directory on the local system
  • put
  • get
  • chmod
  • chown
  • exit

There is no command “bin” nor “quote…”.

Getting from a Unix Services file to a dataset.

You can use the cp command.

To copy a binary file to a dataset

For example as if you were using FTP with BIN; quote site cyl pri=1 sec=1 recfm=fb blksize=3200 lrecl=80; put mp1b.load.xmit ‘COLIN.MP1B.LOAD.XMIT’)

cp -W “seqparms=’RECFM=FB,SPACE=(500,100),LRECL=80,BLKSIZE=3200′” mp1b.load.xmit “//’COLIN.MP1B.LOAD.XMIT'”

Where the seqparms are in upper case. If they are in mixed case you get

FSUM6258 cannot open file “… “: EDC5121I Invalid argument.

I could then do TSO RECEIVE INDSN(‘COLIN.MP1B.LOAD.XMIT’).

To copy a text file to a data set

If you use SFTP to copy a text file to Unix Services, it gets sent in bin, and, on z/OS, looks like

ñà ÈÇÁ ËøÁÄÑÃÑÁÀ….

You can tag a file so Unix Services knows it is an ASCII file, using

chtag -tc ISO8859-1 aaa

This makes the file editable from Unix Services, but you cannot just use cp to copy and create a dataset, as above.

You can convert it from ASCII to EBCDIC using

iconv -f ISO8859-1 -t IBM-037 ascii_file  > ebcdic-file

Then use

cp -W “seqparms=’RECFM=VB,SPACE=(CYL,(1,1)),LRECL=800,BLKSIZE=8000′” ebcdic-file “//’COLIN.EBCDFILE'”

Using CISCO openconnect to tunnel to another system from Linux

I needed to use openconnect from CISCO to be able to logon from my Ubuntu system to someone else’s z/OS system.

This was pretty easy, but understanding some of the under the cover’s bits took a bit of time.

Basic install

  • Use sudo apt install openconnect
  • Download the VPNC script from http://www.infradead.org/ 
  • Create the configuration script
    • I saved the script as vpnc-script.sh
    • Using ls /etc/vpnc showed the directory did not exist. Create it and move the file
    • sudo mkdir /etc/vpnc/
    • sudo mv vpnc-script.sh /etc/vpnc/
    • sudo chmod +x /etc/vpnc/vpnc-script.sh
  • You need information from the owners of the vpn server.
    • vpn userid
    • vpn password
    • name of their system
    • IP address of their internal system
    • tso userid
    • tso password
  • I created a script (openc.sh), where XXXXX is the short userid, and password is your long userid:
    • printf ‘%s’ “password” | sudo openconnect –user=XXXXXX –script=/etc/vpnc/vpnc-script.sh vpn.customer.com
  • When you run openc.sh it prompts for your su password on the machine. The print… means you can store the password in the shell script. If you do not specify it, openconnect will prompt you for it.
  • Once the connection is made you can use ping 10.66.77.88, or x3270 -model 5 10.66.77.88 to access the system, where 10.66.77.88 is the IP address the owner of the vpn server gave you.

x3270

The owner of the vpn server gave me the address of the z/OS machine, my userid and password.

I then used

x3270 -model 5 10.66.77.88 to logon to the system.

Hot key

I like to hot key to my z/OS sessions. I used Ubuntu “Settings”-> Keyboard shortcuts, and added a shortcut

  • name: mvsCust
  • Command: wmctrl -a 10.66.77.88
  • Hot key: Ctrl + H

The wmctl -a says make the window active which has 10.66.77.88 in the window page title.

When I press Ctrl +H it makes the customers x3270 session the active window.

Change the x3270 colours

I wanted to change the screen colours, to distinguish it from other 3270 sessions. See Making x3270 green screens blue or red, or yellow with green bits.

FTP

I had to use SFTP colin@10.66.77.88 to ftp to the remote z/OS system (where colin is my TSO userid).

What happens with openconnect, under the covers.

The handshake has several stages

  • Establish a TLS session using the certificate from the server. Once this has completed, any traffic is encrypted. In my case I used the vpn userid and password. The vpn server can be configured to accept certificates instead of userid and password.
  • The server sends down configuration information from the vpn server’s configuration. For example
    • The IP addresses it supports , such as 10.66.0.0 and netmask 255.255.0.0
    • Any changes to the DNS configuration, so it knows to route 10.66.77.78 via the VPN session.
    • The “banner” such as “Welcome to mycom.com. Users of this system do so at their own risk”.
    • A default domain.
    • Which tunnelling device to use – such as tun0.
    • How many configuration statements.
    • Each set of configuration statements.
    • You can see this information by using the -v option on the openconnect command.
  • Using the information sent from the the vpn server, the openconnect client creates environment variables.
  • The script defined (or defaulted, for example /etc/vpnc/vpnc-script.sh) on the openconnect command is invoked, and it uses these environment variables to manage the ip and dns configuration, changing files like /etc/resolv.conf (the local DNS file).

Oh p*x, I’ve lost my changes

I have been using pax to backup the files in my Unix Services directory and needed to restore a file so I could compare it with the last version (and work out why my updates didn’t work). Unfortunately I managed to overwrite my latest version instead of creating a copy.
I backed up my directory using

pax -W “seqparms=’space=(cyl,(10,10))'” -wzvf “//’COLIN.PAX.PYMQI2′” -x os390 /u/tmp/pymqi2/

This created a data set COLIN.PAX.PYMQI2 with the give space parameters, and os390 format.
(If you do not want all the files listed, use –wzf)

To list the contents of this file use

pax -f “//’COLIN.PAX.PYMQI2′”

To display a subset of the files use

pax -f “//’COLIN.PAX.PYMQI2′” /u/tmp/pymqi2/code

which gave

/u/tmp/pymqi2/code/
/u/tmp/pymqi2/code/pymqi/
/u/tmp/pymqi2/code/pymqi/__init__.py
/u/tmp/pymqi2/code/pymqi/old__init__.old
/u/tmp/pymqi2/code/pymqi/aa

And provide more information using the -v option

drwxrwxrwx 1 COLIN    1000      0 Jan 22 17:04 /u/tmp/pymqi2/code/
drwxr-xr-x 1 COLIN    1000      0 Feb 11 13:10 /u/tmp/pymqi2/code/pymqi/
-rw-r--r-- 1 OMVSKERN 1000 133011 Feb 22 13:15 /u/tmp/pymqi2/code/pymqi/init.py
-rw-r----- 1 COLIN    1000 119592 Feb  3 12:59 /u/tmp/pymqi2/code/pymqi/old__init__.old
-rwx------ 1 OMVSKERN 1000 119565 Jan 22 16:43 /u/tmp/pymqi2/code/pymqi/aa

The whoops

To restore an individual file and overwrite the original I used the -r option.

pax -rf “//’COLIN.PAX.PYMQI2′” /u/tmp/pymqi2/pymqi/__init__.py

I was expecting the file to be restored relative to the directory I was in; No – because I had backed up the files using an absolute path it restored the file to the same place, and so it overwrote my changes to the file. I had changed to a temporary directory, but I had not realised how the command worked.

There are several ways of doing it properly.

Restore with rename

pax -rf “//’COLIN.PAX.PYMQI2′” -i /u/tmp/pymqi2/pymqi/__init__.py

The -i option means rename.

I ran the command and it prompted me to rename it

Rename “/u/tmp/pymqi2/pymqi/__init__.py” as…

/tmp/oldinit.py

Set “do not overwrite”

I could also have used the -k option which prevents the overwriting of existing files.

Rename on restore

I could also have used the rename

pax -rf “//’COLIN.PAX.PYMQI2′” -s#/u/tmp/pymqi2/pymqi#/tmp/# /u/tmp/pymqi2/pymqi/__init__.py

Where the -s#/u/tmp/pymqi2/pymqi#/tmp/# / says use the regular expression to change /u/tmp/pymqi2/pymqi to /tmp and so restore it to a different place. Note: The more obvious -s/abc/xyz/, where / is used as the delimiter, would not work, as there is a ‘/’ in the file path.

All of the above

I could have use all of the options -i -k -s…. .

A better way to backup.

I had specified an absolute directory /u/tmp/pymi2/. If I was in this directory when I did the backup I could have used

pax … -x os390 .

Where the . at the end means from this directory, and so backup a relative directory.

If I list the files I get

pax -f “//’COLIN.PAX.PYMQI2A'” ./aa
./aa

And now if I restore the file…

pax -rf “//’COLIN.PAX.PYMQI2A'” ./aa

It restored the file into my working directory /tmp/aa .

So out of all the good ways of backing up and restoring – I chose the worst one. It only took me about 2 hours to remake all the changes I had lost.

What do you mean, I can’t set the maximum queue depth?

Ive been involved with MQ for the whole of its life (I worked on the very first release), and I’ve just discovered something which is really basic!

I was testing out some code, and wanted to set the max depth of the queue to a low value for the duration of a test.

I thought this was easy; just use MQSET, specify parameter MQIA_MAX_Q_DEPTH, and the value I wanted (9).

When it reported

Reason 2067: FAILED: MQRC_SELECTOR_ERROR

I added in extra code to printout the values and yes, this was the value returned. If this was a bug I was sure it would have been spotted many years ago. I checked the documentation (remember, if all else fails, read the documentation). Look, I can set get inhibit, put inhibit etc ahh – max depth was not listed.

So when you get Reason 2067: FAILED: MQRC_SELECTOR_ERROR please check the documentation.

Debugging PIP install on z/OS

I had various problems installing Python packaged on z/OS.

Firstly the Python file system was read only – that’s ok. I can try to install in a virtual environment. This also had it’s problems!

The main problem was, I had the HOME environment variable pointing to a directory which my userid did not have write access to – /u/. When I changed it to my working directory /u/tmp/pymi2, it worked with no problems.

I’ll document in the blog the debugging I did.

Preparation

I FTP’d the wheels package in binary from my Linux machine, into /tmp on z/OS.

I switched to my virtual environment using

. env/bin/activate

This gave a command line like

(env) COLIN:/u/tmp/pymqi2:

I used the command

python3 -m pip install /tmp/wheel-0.37.1-py2.py3-none-any.whl /tmp/wheel-0.37.1-py2.py3-none-any.whl –no-cache-dir

But this gave

Installing collected packages: wheel
ERROR: Could not install packages due to an EnvironmentError: [Errno 111] EDC5111I Permission denied.: ‘/u/.local’

Using the –verbose argument

python3 -m pip –verbose install /tmp/wheel-0.37.1-py2.py3-none-any.whl /tmp/wheel-0.37.1-py2.py3-none-any.whl &x45;&x45;no-cache-dir

gave more information, but not enough to resolve the problem.
Using

export DISTUTILS_DEBUG=1
python3 -m pip &x45;&x45;verbose install..

gave much more information including

config vars:
{‘abiflags’: ”,
‘base’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘dist_fullname’: ‘UNKNOWN-0.0.0’,
‘dist_name’: ‘UNKNOWN’,
‘dist_version’: ‘0.0.0’,
‘exec_prefix’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘platbase’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘prefix’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘py_version’: ‘3.8.5’,
‘py_version_nodot’: ’38’,
‘py_version_short’: ‘3.8’,
‘sys_exec_prefix’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘sys_prefix’: ‘/usr/lpp/IBM/cyp/v3r8/pyz’,
‘userbase’: ‘/u/.local’,
‘usersite’: ‘./lib/python3.8/site-packages’}

where I could see where the /u/.local came from.

You can change the userbase (see here) using

export PYTHONUSERBASE=.

and the product installed. When I fixed my HOME environment variable to point to my working directory this also worked!

What is installed?

Using the command:

python3 -m pip --verbose list

This gave

Package      Version Location                                                   Installer          
------------ ------- ---------------------------------------------------------- ---------          
cffi         1.14.0  /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages                    
cryptography 2.8     /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages                    
ebcdic       1.1.1   /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages pip                
numpy        1.18.2  /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages pip                
pip          20.2.1  /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages pip                
pycparser    2.20    /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages pip                
setuptools   47.1.0  /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages pip                
six          1.15.0  /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages                    
wheel        0.37.1  /u/tmp/pymqi2/lib/python3.8/site-packages                  pip                
zos-util     1.0.0   /Z24C/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/site-packages                    

so we can see the wheel package was installed in my user directory.

Without the --verbose, just package and version were displayed, no location or installer.

Uninstall it

I used

python3 -m pip –verbose uninstall wheel

to uninstall it

Python virtual environments on z/OS

Virtual environments are very useful as they allow you to test install components without affecting the shared libraries.

You create an environment called env with

python3 -m venv env

but this gave

Error: Command ‘Ý’/u/py/env/bin/python3’, ‘-Im’, ‘ensurepip’, ‘–upgrade’, ‘–default-pip’¨’ returned non-zero exit status 1.

I think this was because my z/OS had no direct network connection. However

python3 -m venv –without-pip env –system-site-packages

worked. I needed the –system-site-packages, so I could build the extension.

Activate the environment

The instruction said use the command env/bin/activate but this failed

env/bin/activate: FSUM9209 cannot execute: reason code = ef076015: EDC5111I Permission denied.

but

. env/bin/activate

worked. (Just . env/b*/a* worked for me.)

If you get
 FSUM7332 syntax error: got Word, expecting )    

You need to issue the command ( or put it in your .profile)

export _BPXK_AUTOCVT=ON

See here. (_BPXK_AUTOCVT Used when enabling automatic conversion of tagged files).

To get out of the virtual environment use

deactivate

To delete the environment

remove the directory

rm -r env

Use the environment

You should now be able to use the environment.

You should check that the HOME environment variable is a directory with read/write access.

You may want to set up PYTHONHOME for special python packages.