Getting sshfs to work to z/OS

You can “mount” a remote file system as a local directory over sshfs. (ssh file system).

Getting this working was a challenge. I do not know if it is an FTP problem, or a z/OS problem

The command, from Linux, is

sshfs colin@10.1.1.2: ~/mountpoint

where mountpoint is a local directory, and my z/OS system is on 10.1.1.2

This flows into the SSH daemon (SSHD) on z/OS which handles the handshake and encryption.

For the IBM provided SSHD, the /etc/ssh/sshd_config config file has

Subsystem sftp /usr/lib/ssh/sftp-server 

Where /usr/lib/ssh/sftp-server is the executable to do the work. The IBM supplied object is a load module. You could replace this with a script or other module.

Once the session has been established you can access the files, as if they were on the local system.

What is running on z/OS?

If you use the ps -ef command it displays

     UID        PID       PPID  CMD                                               
OMVSKERN 50397264 67174474 /usr/sbin/sshd -f /etc/ssh/sshd_config -R
COLIN 67174482 50397264 /usr/sbin/sshd -f /etc/ssh/sshd_config -R
COLIN 50397267 67174482 sh -c /usr/lib/ssh/sftp-server
COLIN 83951719 50397267 /usr/lib/ssh/sftp-server

This shows the calling chain – the first (SSHD) is at the top, and the last, /usr/lib/ssh/sftp-server, is doing the work to process the files

The shell used depends on the OMVS(PROGRAM()) defined for the userid.

When did sshfs work?

If I had OMVS(PROGRAM(‘/bin/sh’)) then the sshfs worked ok, I could used the files as expected.

If the program was for bash or for zhs, then the data as seen from Linux was in EBCDIC and so was not usable.

So how do I use zsh or bash?

I got round this problem…

I specified the userid as having OMVS(PROGRAM(‘/bin/sh’)), and changed to use the bash shell in the logon script

If I logon with ssh colin@10.1.1.2 then there are environment variables in /etc/profile and ~/.profile.

SSH_CLIENT="10.1.0.2 44898 22"
SSH_CONNECTION="10.1.0.2 44898 10.1.1.2 22"
SSH_TTY="/dev/ttyp0000"

In my ~/.profile I’ve put

if [[ ! -z "$SSH_CLIENT"  ]] 
then
set -x
# SSH_CLIENT has a value ... so an SSH terminal
# bash="/usr/lpp/Rocket/rsusr/ported/bin/bash"
bash="/u/zopen/usr/local/bin/bash"
echo "shell $SHELL bash $bash"
if [[ $SHELL != $bash ]]
then
echo "using the bash shell"
export SHELL="$bash"
exec "$bash" # replace the current script with bash
# any code after the exec is not executed
fi
fi

which says. If the $SSH_CLIENT variable is not the empty string, (the session came in over an ssh connection) then invoke $bash, and it replaces the current environment with the /u/zopen/usr/local/bin/bash.

With this I could use both sshfs for remote file access, and ssh for terminal access.

If there are better ways of doing this, please let me know

What’s hammering my Linux Ethernet and how do I stop it?

I was downloading some stuff on one machine, and noticed that my Ethernet connection had a very high throughput – but it was doing nothing useful. This blog post gives some of the things I did to identify and resolve the problem.

Mount the file system

I used the command

sshfs colin@10.1.0.3:/home/zPDT/ ~/mountpoint

to mount the file system from 10.1.03 on my local machine.

Identify the problem

I used the Linux command nload to show the network activity.

For my wireless link (downloading a big file) the output was

I cannot currently reproduce the sustained Ethernet usage problem.

Wireshark showed my a lot of activity for SSH from port 55401 to port 22.

If you do not have access to Wireshark, the following command show all the socket activity which may help.

ss -t -a -i -O |grep delivery|awk '{print $4,$5, " ", $30,$31 }'

To find the owner of port 55401 I used the show socket command

ss -p |grep 55104
tcp ESTAB 0 0 10.1.0.2:55104 10.1.0.3:ssh users:(("ssh",pid=7258,fd=3))

This gave me the process id of the owner of the port. The ps command gives more information

ps -ef |grep 7258
colinpa+ 7258 ... ssh -x -a -oClearAllForwardings=yes -2 colin@10.1.0.3 -s sftp

Showing the sftp to 10.1.0.3.

How to stop the sftp?

The documentation for sshfs says use the fusermount3 command.

$fusermount3 -u ~/mountpoint 
fusermount3: failed to unmount /home/colinpaice/mountpoint: Device or resource busy

I needed to use the lazy unmount option -z

 fusermount3 -z  -u ~/mountpoint

and this successfully unmounted the remote file system

Chaff

I found out that information can be obtained from the profile of key strokes, and so chaff has been added to the SSH flow.

I fixed it by using setting ObscureKeystrokeTiming no in /etc/ssh/ssh_config. The documentation says

Specifies whether ssh(1) should try to obscure inter-keystroke timings from passive observers of network traffic. If enabled, then for interactive sessions, ssh(1) will send keystrokes at fixed intervals of a few tens of milliseconds and will send fake keystroke packets for some time after typing ceases. The argument to this keyword must be yes, no or an interval specifier of the form interval:milliseconds (e.g. interval:80 for 80 milliseconds). The default is to obscure keystrokes using a 20ms packet interval. Note that smaller intervals will result in higher fake keystroke packet rates.