Getting sshfs to work to z/OS

You can “mount” a remote file system as a local directory over sshfs. (ssh file system).

Getting this working was a challenge. I do not know if it is an FTP problem, or a z/OS problem

The command, from Linux, is

sshfs colin@10.1.1.2: ~/mountpoint

where mountpoint is a local directory, and my z/OS system is on 10.1.1.2

This flows into the SSH daemon (SSHD) on z/OS which handles the handshake and encryption.

For the IBM provided SSHD, the /etc/ssh/sshd_config config file has

Subsystem sftp /usr/lib/ssh/sftp-server

Where /usr/lib/ssh/sftp-server is the executable to do the work. The IBM supplied object is a load module. You could replace this with a script or other module.

Once the session has been established you can access the files, as if they were on the local system.

What is running on z/OS?

If you use the ps -ef command it displays

     UID        PID       PPID  CMD                                               
OMVSKERN   50397264   67174474  /usr/sbin/sshd -f /etc/ssh/sshd_config -R                                          
   COLIN   67174482   50397264  /usr/sbin/sshd -f /etc/ssh/sshd_config -R         
   COLIN   50397267   67174482  sh -c /usr/lib/ssh/sftp-server
   COLIN   83951719   50397267  /usr/lib/ssh/sftp-server

This shows the calling chain – the first (SSHD) is at the top, and the last, /usr/lib/ssh/sftp-server, is doing the work to process the files

The shell used depends on the OMVS(PROGRAM()) defined for the userid.

When did sshfs work?

If I had OMVS(PROGRAM(‘/bin/sh’)) then the sshfs worked ok, I could used the files as expected.

If the program was for bash or for zhs, then the data as seen from Linux was in EBCDIC and so was not usable.

So how do I use zsh or bash?

I got round this problem…

I specified the userid as having OMVS(PROGRAM(‘/bin/sh’)), and changed to use the bash shell in the logon script

If I logon with ssh colin@10.1.1.2 then there are environment variables in /etc/profile and ~/.profile.

SSH_CLIENT="10.1.0.2 44898 22"
SSH_CONNECTION="10.1.0.2 44898 10.1.1.2 22"
SSH_TTY="/dev/ttyp0000"

In my ~/.profile I’ve put (thanks to Kirk Wolf for suggesting the better if interactive shell sta

tement)

# for all interactive sessions use the following if
#if [[ $- = *i* ]];
# for sessions only with ssh use the following ig statement
if [[ ! -z "$ SSH_CONNECTION"  ]] 
    # ssh: switch to bash....
     #set -x 
     #     bash="/usr/lpp/Rocket/rsusr/ported/bin/bash" 
      bash="/u/zopen/usr/local/bin/bash" 
      echo "shell was  $SHELL bash $bash"
      if [[ $SHELL != $bash ]] 
        then 
          echo "using the bash shell" $bashs
          export SHELL="$bash" 
          exec "$bash" # replace the current script with bash
         # any code after the exec is not executed  
      fi 
 fi

which says. If the $SSH_CONNECTION variable is not the empty string, (the session came in over an ssh connection) then invoke $bash, and it replaces the current environment with the /u/zopen/usr/local/bin/bash.

With this I could use both sshfs for remote file access, and ssh for terminal access.

If there are better ways of doing this, please let me know

OMVS is the way ahead!

If you have any suggestions in this area – please let me know!

I recently found this article which covers the same ground with more/better explanations.

With lots of development of open source tools for OMVS on z/OS, I thought I would try it out. I’ve been amazed at how good it is. This blog post is “one liners” to help people get started and over the initial hump to moving towards this. I’ll add more blog posts as I go further down the path.

My original task was to develop some Python scripts to extract profiles from RACF. I use Ubuntu Linux on my laptop.

I used to use OMVS from ISPF, because I thought the interface through SSH was poor in comparison. I now think that the OMVS interace is limited compare to the SSH interface, because of all of the enhancements and packages available to it.

One example is I use the “less” command on Linux very frequently. This does not work with ISPF OMVS, but it is available through SSH.

See Setting Up the z/OS UNIX Shell (Correctly & Completely) for an excellent article on moving to OMVS though SSH.

Editing is easy

Create a mountpoint on your laptop.
use sshfs colin@10.1.1.2: ~/mountpoint
use vscode to edit the files. This is a very popular editor/IDE.
you have to be careful of tagging the file. Create a file using touch, then use “chtag -t -c ISO8859-1 filename “, and then edit it. It is editable in vscode and ISPF (but not at the same time of course). Yesterday the files needed the tag ISO8859-1, today they only work without the tag! ( chtag -r newtry.py) – I do not know what has changed!

I used other tools such as diff, from my laptop to files in my z/OS Home directory.

You can install packages like zowe on z/OS and use vscode to edit files and datasets from lists, to issue z/OS commands, and look at spool. This is a heavy weight package, but is very popular. The editing via sshfs is very easy.

Install zopen

zopen is a set of packages ported from open source. It was easy to install.

I used zopen install … to install packages. I used

zopen install which, this tells you the full path of a command
zopen install less, less is a fast display of a file in a terminal, with search capability. It is often faster than an editor/browser. Less is a more advanced version of the more command. The more command allows you to page through a file.

Use the bash or zsh shell

In my OMVS userid profile I set PROGRAM(/u/zopen/usr/local/bin/bash) This version of bash has proper key support. For example delete really does delete characters. For the Rocket port of bash the delete key is a dead key.

If your delete key does not work on the command line

Use the zopen bash shell, PROGRAM(/u/zopen/usr/local/bin/bash) or the zsh shell PROGRAM(/bin/zsh).

Logon

Use a command like ssh colin@10.1.1.2 to get to z/OS. You can configure SSH and transfer a key file to z/OS, so you can logon without a password.

Using the right shell

The default borne shell is so back level. You should use bash or zsh.

bash

You should use bash from zopen. Use PROGRAM(/u/zopen/usr/local/bin/bash) in your RACF profile. Use bash from zopen because this has more function than Rocket’s bash – and the delete key works as expected.

zsh

Many people recommend zsh. Use program(/bin/zsh) in your RACF profile to use it. See Oh My Zsh or: How I Learned to Stop Worrying and Love the Shell to a good introduction to zsh extensions. For example there are sudo, and a git interface.

Issuing TSO commands

You can use the tsocmd to issue a command and get a response back

tsocmd "LU COLIN" |less

you can then page through the output.

Command complete

Bash and zsh have command completion.
if you type zopen [tab] [tab] it will display the options available for the zopen command

You can use ls [tab] [tab] to display all the files in the current directory

RACF

I’ve been using the Python interface (pysear) to RACF to display information, and manage profiles. It’s great and very flexible.

SDSF

There is a python interface to sdsf, available in z/OS 3.1, but it is not available in the 3.1 images I have.

My ~/.profile

export _BPXK_AUTOCVT=ON
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt
export CXX="/bin/xlclang++"
export CC="/bin/xlC"
export CC="/bin/xlclang++"
export PATH=${PATH}:/bin
export PATH=${PATH}:/u/colin/.local/bin
export PATH=${PATH}:/u/tmp/zowep/bin/
export PATH=${PATH}:/usr/lpp/IBM/cyp/v3r12/pyz/bin
export LIBPATH=${LIBPATH}:/usr/lpp/IBM/cyp/v3r12/pyz/lib
. /u/zopen//etc/zopen-config --override-zos-tools
# if Ive come in from SSH....
if [[ -z "$SSH_CLIENT"  ]]
   then
     # dummy
     xxx="$SSH_CLIENT"
   else
     set -x
     zopenkw="alt audit build clean compare-versions compute-builddeps  \
      config-helper create-cicd-job create-repo diagnostics  \
      generate help2man info init install md2man migrate-buildenv \
      migrate-groovy promote publish query remove split-patch     \
      Cupdate-cacert usage version whichproject "
     echo $kw
     complete -W "$zopenkw "  zopen
 fi

That’s as far as I’ve got.

Keeping people out of the master catalog.

I had written Here’s another nice mess I’ve gotten into! My master catalog is full of junk which describes what I did once I found my master catalog was full of stuff which should not be there.

I’ve now got round to finding out how to stop people from putting rubbish there in the first place!

See One minute mvs: catalogs and datasets for an introduction to master and user catalogs.

The master catalog should have some system datasets, aliases, and not much else.

An alias says for this high level qualifier (COLIN) go to the usercatalog(‘USER.COLIN.CATALOG).

A catalog is a dataset, and you can use a RACF profile to protect it, so only authorised people can update it. Typically, when you define a userid or a high level qualifier, you should also define an alias for that userid (or HLQ), pointing to a user catalog.

To keep user data out of the master catalog you need

one or more user catalogs – for example do you give each user their own catalog, have one per deparment, or one for all users. These catalogs are typically defined by storage administrators (or automation set up by storage administrators).
an alias for each userid and the name of the catalog that userid should use. These aliases are set up by people (or automation) which defines userids.
an alias for each dataset High Level Qualifier (HLQ) and the name of the catalog that the HLQ should use. These aliases are set up by people (or automation) which defines the high level qualifiers. An example HLQ is CEE, or DB2.

If you migrate to a system with a new master catalog (for example with zPDT or zD&T), you will need to import the usercatalogs into the master catalog, and redefine the aliases.

Import a user catalog

When I tried to import a user catalog into the master catalog, I got

 ICH408I USER(COLIN   ) GROUP(TEST    ) NAME(CCPAICE             ) 
   CATALOG.Z31B.MASTER CL(DATASET ) VOL(B3SYS1)                    
   INSUFFICIENT ACCESS AUTHORITY                                   
   FROM CATALOG.Z31B.* (G)                                         
   ACCESS INTENT(UPDATE )  ACCESS ALLOWED(READ   )

so any userid importing or exporting a catalog needs update access to the catalog.

Defining and deleting an alias

Having set up RACF profiles, and given my userid COLIN only READ access to the master catalog, I found my userid could still define and delete aliases. It took a couple of days to find out why.

If a userid has ALTER access to CLASS(FACILITY) STGADMIN.IGG.DEFDEL.UALIAS the userid can define and delete ALIAS profiles. This overrides dataset access checks.
If a userid does not have ALTER access to the profile, then normal dataset checks are made.

What I learned…

My userid had “special”. As the documentation says The RACF SPECIAL attribute allows you to update any profile in the RACF database. This meant I could display and update any profile.
There is a profile class(facility) STGADMIN.IGG.DEFDEL.UALIAS which allows you to define and delete user aliases in the (master) catalog
If my userid had SPECIAL, or the userid was in group SYS1 I could issue the command

rlist facility STGADMIN.IGG.DEFDEL.UALIAS

and it gave

CLASS      NAME
-----      ----
FACILITY   STGADMIN.IGG.* (G)
LEVEL  OWNER      UNIVERSAL ACCESS  YOUR ACCESS  WARNING
-----  --------   ----------------  -----------  -------
 00    IBMUSER         NONE              ALTER    NO
 
USER      ACCESS
----      ------
SYS1      ALTER
IBMUSER   ALTER

If my userid did not have special and was not in SYS1, I got

ICH13002I NOT AUTHORIZED TO LIST STGADMIN.IGG.*

When my userid was connected to the group SYS1, it got the ALTER access to the profile, and overrode the RACF profiles for the catalog data set.

Which is my master catalog?

At IPL, it reports

IEA370I MASTER CATALOG SELECTED IS CATALOG.Z31B.MASTER

You can use the operator command D IPLINFO

SYSTEM IPLED AT 07.26.58 ON 01/02/2026                                              
RELEASE z/OS 03.01.00    LICENSE = z/OS                                             
USED LOADCP IN SYS1.IPLPARM ON 00ADF

My load parm member, SYS1.IPLPARM(LOADCP) has

IODF     99 SYS1 
INITSQA  0000M 0008M 
SYSCAT   B3SYS1113CCATALOG.Z31B.MASTER 
SYSPARM  CP 
IEASYM   (00,CP)

The catalog is called CATALOG.Z31B.MASTER and is on volume B3SYS1

Does a RACF profile exist?

See What RACF profile is used for a data set?

tso listdsd dataset(‘CATALOG.Z31B.MASTER’)
tso listdsd dataset(‘CATALOG.Z31B.MASTER’) generic

Showed there was no profile defined.

Create the profile

* DELDSD  'CATALOG.Z31B.*'                                   
ADDSD  'CATALOG.Z31B.*' UACC(READ)                         
PERMIT 'CATALOG.Z31B.*' ID(IBMUSER ) ACCESS(CONTROL)       
PERMIT 'CATALOG.Z31B.*' ID(COLIN   ) ACCESS(READ   )

When I tried to use the master catalog from a general userid the request failed.

DELETE TEST  ALIAS                                                                                                 
IDC3018I SECURITY VERIFICATION FAILED+                                                                               
IDC3009I ** VSAM CATALOG RETURN CODE IS 56 - REASON CODE IS IGG0CLFT-6                                               
IDC0551I ** ENTRY COLIN.TEST NOT DELETED                                                                             
IDC0014I LASTCC=8

Hmm that’s strange

With userid COLIN, I could still issue commands, such as DELETE TEST ALIAS, even though I had given it only read access.
If I displayed the profile from userid COLIN it had

 INFORMATION FOR DATASET CATALOG.Z31B.* (G)
  
 LEVEL  OWNER    UNIVERSAL ACCESS   WARNING   ERASE
 -----  -------- ----------------   -------   -----
  00    COLIN           READ          NO      NO
 YOUR ACCESS  CREATION GROUP  DATASET TYPE
 -----------  --------------  ------------
     READ        SYS1           NON-VSAM

This had me confused for several hours. That’s when I found out about the presence of the STGADMIN.IGG.DEFDEL.UALIAS profile.

Summary

You want users (non system) datasets to be in a user catalog, rather than the master catalog. This makes migrating to a new master catalog much easier, You just have to import the catalogs, and redefine the aliases.

You need to set up

one (or more) user catalogs
aliases to connect the userid (and High Level Qualifiers) to a catalog
give authorised used alter access to class(facility) STGADMIN.IGG.DEFDEL.UALIAS to allow them to maintain aliases.
define a RACF profile for the master catalog and make the UACC(READ).
for those people who need to need to define, import or export catalogs, they need update access to the master catalog dataset.

What is RACF GLOBAL….

With RACF you can define a profile and give userids access to it. You can also define a global profile for high used datasets, so the profile is cached, and no I/O is needed to the RACF dataset.

Defined a normal profile

 ADDSD  'COLIN.Z31B.*' UACC(READ)                         
 PERMIT 'COLIN.Z31B.*' ID(IBMUSER,COLIN) ACCESS(CONTROL)

You can list it

LISTDSD DATASET('COLIN.Z31B.*') ALL

and delete it

DELDSD DATASET('COLIN.Z31B.*')

For some resources used very frequently, you can cache definitions in memory. These are called GLOBAL definitions. When a check is made for a userid to access a resource, if the definition is a global definition, then there should be no RACF database I/O, and should be fast.

Define a global resource

You need to set up the global resource before you can use it. See the IBM documentation.

Example 1 contains

SETROPTS GLOBAL(DATASET)
RDEFINE  GLOBAL DATASET
SETROPTS GLOBAL(DATASET) REFRESH

and

RALTER   GLOBAL DATASET ADDMEM('SYS1.HELP'/READ)
ADDSD    'SYS1.HELP' UACC(READ)
SETROPTS GLOBAL(DATASET) REFRESH

to define a resource. It gives a default of read access to the data set SYS1.HELP.

You can display the contents of the global data set class

rlist global dataset

which gives

CLASS      NAME
-----      ----
GLOBAL     DATASET
...
RESOURCES IN GROUP
--------- -- -----
SYS1.HELP/READ
...

You can delete a global profile

RALTER   GLOBAL DATASET DELMEM('SYS1.HELP'/READ)
SETROPTS GLOBAL(DATASET)

You can remove the global dataset class if there are no elements in the glas

RDElete  GLOBAL DATASET
SETROPTS NOGLOBAL(DATASET)
SETROPTS GLOBAL(DATASET) REFRESH

If you now list the global profile

rlist global dataset

gives

 ICH13003I DATASET NOT FOUND

I’m guessing that if you want READ access to the SYS1.HELP data set, the entry in the GLOBAL DATASET will be found. If you want UPDATE access to the SYS1.HELP data set, because there is no entry in the GLOBAL DATASET, checking will fall through to the normal profiles defines like ADDSD.

You do not need to configure the GLOBAL DATASET, but it can give performance benefits, if you are on a heavily used system. It is not enabled on my one person zD&Y system.

Beware

In the documentation it also defines a “normal” profile like “ADDSD
‘SYS1.HELP’ UACC(READ)”. I’m guessing that this is a fall back if someone deactivates the global dataset profiles.

So you should read the documentation and follow its instructions.