It’s easy – use the chmount command.
chmount -w /usr
or
chmount -r /usr
Category: USS
How do I allocate a Unix id on z/OS?
To use Unix services (sometimes known as USS) on z/OS, a userid needs a UserID (UID). This, as on Unix,is an integer. A user can be pre-allocated a permanent UID, or be allocated a UID when when needed. See Automatically assigning unique IDs through UNIX services.
Unique or not Unique?
It is good practice for each userid to have a unique UID. If users share the same UID,
- The users share ownership and access to the same files.
- If you ask for the userid associated with an id – you may get the wrong answer!
However some super users need a id of 0.
You can set this as shared with
altuser colin OMVS(UID(0)SHARED)
Instead of allocating uid(0) you can use the profile BPX.SUPERUSER resource in the FACILITY class to get the authority to do most of the tasks that require superuser authority.
- You can explicitly specify an id which you allocate (this means you need a list of ids and owners, so you know which ids are free).
- You can have z/OS do this for you. See Enabling automatic assignment of unique UNIX identities.
You can use ADDUSER COLIN OMVS(AUTOUID) which allocates an available UID.
Should I used AUTOID?
I run z/OS on a zD&T image. Every 6 months or so there is a new level of z/OS which I can download. I then need to migrate userid, datasets etc to this new system. This is different to a normal customer z/OS where you have an existing system and you migrate a new version of z/OS into it.
I have ZFS file systems for all of my user data.
On the current system my userid COLIN was automatically allocated as 0000990021. Files that I own have this id.
When I get my next system, if I allocate userid COLIN with AUTOUID, it may get a different UID say 990011. Because my userid 990011 is different to the owner of the files 990021, I may not be able to access “my” files.
I could change all of my files to have a new owner (and group), or I could ensure my userid on both systems is the same 990021. Using the same UID was much easier.
How is the range of AUTOIDs defined?
This is done with the RACF FACILITY profile BPX.NEXT.USER. On my system has has
APPLICATION DATA 990041-1000000/990020-1000000
Can I define a model profile?
You can configure OMVS to automatically give a userid a UID (if it does not have one) and define the rest of the OMVS profile using a model OMVS segment. See Steps for automatically assigning unique IDs through UNIX services.
Users need a home directory
Users need a home directory. There are several ways of doing this.
- Give users an entry HOME(‘/u/mostusers’). Every one shares the same directory – not a good idea, because they would all share the SSH keys etc.
- You could specify HOME(‘/u/mostusers/&racuid’) and specify the userid as part of the definition. This could be done in the model profile mentioned above. If you use this method you need to create the directory, for example as part of creating the userid.
- Use automount. See Unix services automount is a blessing and curse. Where you define a template and the hard word is done for you. For example for each userid create a ZFS and use that.
I only use a few userids, so manually allocating the userid and the home directory was easy to do.
Note: If you use automount of a directory, such as /u/, you cannot mount other file systems in /u/; you would have to use a different directory, for example /usr/.
How do I create a load module in a PDS from Unix?
This is another of the little problems which are easy once you know the anwser.
I used the shell program to compile my program.
name=extract
export _C89_CCMODE=1
p1="-Wc,arch(8),target(zOSV2R3),list,source,ilp32,gonum,asm,float(ieee)"
p7="-Wc,ASM,ASMLIB(//'SYS1.MACLIB') "
p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,XREF,SHOWINC -Wa,LIST(133),RENT"
# compile it
xlc $p1 $p7 $p8 -c $name.c -o $name.o
l1="-Wl,LIST,MAP,XREF,AC=1 "
# create an executable in the file system
/bin/xlc $name.o -o $name -V $l1 1>a
extattr +a $name
# create a load module in a PDS
/bin/xlc $name.o -o "//'COLIN.LOAD(EXTRACT)'" -V $l1 1>a
Create an executable in the file system
The first bind xlc step creates an object with name “extract” in the file system.
Specify the load module
The second bind step specified a load module in a PDS. The load module is stored in COLIN.LOAD. If you copy and paste the line, make sure you have the correct quotes ( double quote, //, single quote, dataset(member),single quote,double quote). Sometimes my pasting lost a quote.
Process assembler code
My program has some assembler code…
asm( ASM_PREFIX
" STORAGE RELEASE,...
:"r0", "r1" , "r15" );
It needs the options “-Wc,ASM,ASMLIB(//’SYS1.MACLIB’) ” to compile it, and specify the location of the assembler macros.
Binder parameters
The line parameters in -Wl,LIST,MAP,XREF,AC=1 are passed to the binder.
Message – wrong suffix on the source file
Without the export _C89_CCMODE=1 I got the message
FSUM3008 Specify a file with the correct suffix (.c, .i, .s, .o, .x, .p, .I, or .a), or a corresponding data set name, instead of -o ./extract.
What is my Unix process doing?
I was familiar with the USS command ps -ef which displays output like
UID PID PPID C STIME TTY TIME CMD
WEBSRV 16842766 1 - 07:23:05 ? 0:00 -sh -c /web/httpd1/bin/apachectl -k start -f /web/httpd1/conf/httpd.conf -DNO_f
For Zowe threads I was getting
/u/tmp/zowep33//bin/utils/configmgr -script /u/tmp/zowep33//bin/commands/inter
which was annoyingly truncated.
The command ps -e -o args > aa gives the whole command line (up to 1024 bytes) such as
/u/tmp/zowep33//bin/utils/configmgr -script /u/tmp/zowep33//bin/commands/internal/start/component/cli.js
Another useful command when you know it.
Oh p*x, it didn’t copy across some files.
I had managed to mess up the files for a product, so I wanted to copy them across from an older system.
This worked for some of the files – but when I came to start the subsystem – it was missing some files! For example /u/my/zosmf/liberty/lib/native/zos/s390x/bbgzsrv
I copied the files across again – and they were still not there!
Once you know the answer it is obvious…
There is a directory /usr/lpp/zosmf/liberty – and it was this directory that was missing.
Once I looked into it more carefully – this was not a directory, but a symbolic link to another directory liberty -> ../liberty_zos/current
To fix this I used
# go to my version of zosmf
cd /u/my/zosmf
# remove the symbolic link
rm liberty
#make the new link
ln -s /usr/lpp/liberty_zos/current liberty
and now I could use ls /u/tmp/zosmfp/liberty/lib/native/zos/s390x/bbgzsrv and it found the file.
If I had checked this before I started, I would have save myself a half day of IPLing older systems!
Programming shared memory – more head banging.
I was trying to use shared memory (to look at Java Shared Classes), and it took me a day to get it working – better documentation would have helped.
I had two basic problems
- Using smctl to display information about the shared memory, gave the size as 0 bytes, even though the ipcs command showed me there were megabytes of data in the shared memory area.
- Trying to attach the shared memory gave me “invalid parameters” return code – even though the documented reasons for this error code did not apply to my program.
I tried many things, from using different userids, to running with a different storage key, running APF authorised….
I eventually got it to work by compiling my C program in 64 bit mode rather than 31 bit mode. There is no discussion about 31 bit/64 bit in the documentation. If the shared memory in 64 bit mode, you will need 64 bit addressability, so you need a 64 bit program. But there is no way of determining that the shared memory is 64 bit!
My basic program
{
//struct shmid_ds buf;
struct shmid_ds64 buf;
memset(&buf ,0,sizeof(buf));
int shmid = 8197;
int rc = 0;
long l;
int cmd = IPC_STAT;
char * fn = "COLIN";
int shmflg =0;
shmflg = IPC_STAT;
// rc =shmctl(shmid, cmd, &buf);
rc =shmctl64(shmid, cmd, &buf);
perror("shmctl " );
printf("shctl rc %i\n",rc);
l = buf.shm_segsz;
printf("size %ld\n",l);
printHex(stdout,&buf,sizeof(buf));
///////////////////////////////////////////////
// shmat
///////////////////////////////////////////////
char * pData = NULL;
pData = shmat(shmid, NULL , 0 );
printf("Address %ld\n",pData);
printHex(stdout,pData+4096*1024,1024*1024);
int e = errno;
perror("shmat ");
printf("Errno: %s\n",strerror(e));
return 0;
}
Originally I was using EDCCB to compile and bind this.
The EINVAL error return code was (from the documentation) for cases where the pointer in shmat was non NULL. I was passing NULL – so none of this made sense.
The reason code 0717014A was
JRInvalidAmode: An incorrect access mode was specified on the access service
Action: The access mode specified on the access service has unsupported bits turned on. Reissue the request and specify a valid access mode.
It turned out that my program was 31 bit. When I used 64 bit – it magically started working.
I compiled it with EDCQCB, and had to change a few things to be 64 bit mode.
- shmid_ds buf -> shmid_ds64
- shmctl -> shmctl64
When I ran it in 31 bit mode, the length of the storage returned was 0. In 64 bit mode, it gave the correct length. This looks like a way of telling what mode the shared memory is!
How do I diff on Z/OS with Unix files and directories
I wanted to compare the contents of two directories in Unix System Services on z/OS before I merged them. This took me some time to do because the documented is lacking.
With ISPF you can use SUPERC (3.13), give it two PDSs and it shows you the differences.
On Unix there is the diff command. This can compare individual files, or directories. It can display just the changes, or the changes in context.
File a
AOnly
line1
Aline2
line3
File b
line1
Bline2
line3
BOnly
BOnly2
- diff -c1 a b – show the files and the changes in context
- diff a b – show the changes
- diff -s dir1 dir2 compare the directory contents
- diff -s -r dir1 dir2 compare the directory contents, recursively
- using ispf edit compare to show differences.
Using ispf edit compare to show differences
You can use diff to show the differences in two files, but it is not easy to understand. ISPF EDIT has the compare facility. If you know two files are different you can use
- oedit /etc/zexpl/rseapi.env
- use the primary command compare
- enter the fully qualified name /u/ibmuser/zexpl/rseapi.env,, in the “Name . . . . .” field. if you specify +/filename the + means the same directory.
- press pf3 and it will show the differences
Because I tend to remove comments to make it easier to see the content, I tend to use
- oview /etc/ssh/sshd_config you get into ISPF edit, but no changes are saved.
- Comments start with a #. x all;f ‘#’ all 1 1; f ‘ ‘ 1 1 all;del all nx removes comment lines and blank lines
- reset to show the hidden lines
- compare I then specify my version of the file, and see the changes.
- Lines like ====== TrustedUserCAKeys /etc/ssh/user_ca_key.pub are from my copy.
- Lines in green with a line number are in both files 000006 Subsystem sftp /usr/lib/ssh/sftp-server
- Lines like .OAAAA UseDNS yes are from the base file
Update your version, make a copy of the original, then replace the original with your version.
diff -c1 a b – show the files and the changes in context
The line prefix for input file going to output file
- – to be removed
- ! to be changed
- + to be added
The command diff -c1 a b gives
*** a Tue Aug 27 02:48:56 2024
--- b Tue Aug 27 02:50:41 2024
***************
*** 1,4 ****
- AOnly
line1
! Aline2
line3
--- 1,5 ----
line1
! Bline2
line3
+ BOnly
+ BOnly2
- *** a Tue Aug 27 02:48:56 2024 the first file name, and last changed date
- — b Tue Aug 27 02:50:41 2024 the second file name, and last changed date
- *** 1,4 **** the*** show it is file 1, lines 1 to 4
- – AOnly this line is in file a is not in file b, so would need to be removed(-) from file a
- ! Aline2 this line is in file b – but different
- — 1,5 —- this is file b, lines 1 to 5
- ! Bline2 this line is also in file a – but different
- + BOnly this line is in file b and and was additional(+) to file a
When one file exists but is empty you get output like
*** /etc/resolv.conf Wed Mar 6 11:54:50 2024
--- /u/ibmuser/temp/resolv.conf Thu Dec 7 05:40:24 2023
***************
*** 0 ****
--- 1,2 ----
+ nameserver 127.0.0.1
+ TCPIPJOBNAME TCPIP
which follows the rules I explained above. *** 0 **** shows the content after line 0 is empty, because the next line is — 1,2 —- from the other file.
diff a b – show just the changes
gives
1d0
< AOnly
3c2
< Aline2
---
> Bline2
4a4,5
> BOnly
> BOnly2
The output can be split into sections. The first line of each section is like
- 1d0 the first line of a needs to be deleted from b, line 0
- 3c2 line 3 of a is changed from line 2 of b
- 4a4,5 lines 4,5 of b need to be added to a
The < and > tell you which file the data came from
When data is changed it gives the lines
- < content of file a
- — output divider
- > content of file b
When the data is in file a and not file b
- < contents of file a
When the data is in file b and not file a
- > contents of file b
diff -s dir1 dir2 compare the directory contents
If you specify -s, or just specify two directories, it compares the directory content.
You can use
diff -c1 dir1 dir2
the -c1 to display the contents (how I like it).
With the directory entries you get records like
Only in /u/ibmuser/temp: test.tar
Common subdirectories: /etc/wbem and /u/ibmuser/temp/wbem
Only in /etc: yylex.c
diff -c1 /etc/hosts /u/ibmuser/temp/hosts
*** /etc/hosts Wed Mar 6 11:06:55 2024
--- /u/ibmuser/temp/hosts Tue Feb 28 12:43:07 2023
You can find which files are missing from /etc , by using grep ‘Only in /u/ibmuser/temp’ on the output.
It shows the command used for the individual files, and the output
diff -c1 /etc/hosts /u/ibmuser/temp/hosts
*** /etc/hosts Wed Mar 6 11:06:55 2024
--- /u/ibmuser/temp/hosts Tue Feb 28 12:43:07 2023
...
diff -s -r dir1 dir2 compare the directory contents
The -r option displays the data recursively.
Restoring pax files onto z/OS
Some products and packages that run in Unix System Services provide .pax files you download and install. Often the instructions are not very clear.
The steps are usually:
- Download the .pax file to your work station
- Upload the file to z/OS ( usually Unix Services, but a sequential data set can be used.
- Optionally create a ZFS file system, or find space on an existing file system
- Unload the file
Download the .pax file to your work station
This is usually pretty simple – just make sure you are downloading the correct file. I spent a short while trying to get a .tar.gz file for Linux installed.
Upload the file to z/OS ( usually Unix Services, but a sequential data set can be used.
- Upload the file in binary
- I usually upload it into a zfs. This means you do not need to allocate space, and DCB information.
Optionally create a ZFS file system, (or find space on an existing file system)
This is where it starts to get harder – as there is less useful information. The first question is how much space do you need? Sometimes you get information in (mega) bytes, sometimes in number of 512 byte records, sometimes in number of 8KB blocks.
If you need to create a ZFS, you need to create a directory on the file system where you want the file system mounted, for example
mkdir /usr/lpp/java/J11.0.19.0_64
My pax file was in /tmp/ibm-semeru-certified-jdk_s390x_zos_11.0.19.0.pax.Z.
You can list the contents of the file using
pax -ppx -Evzf /tmp/ibm-semeru-certified-jdk_s390x_zos_11.0.19.0.pax.Z 1>aout 2>b
where aout has content like
drwxr-xr-x 1 JENKINS USERGRP 0 May 16 20:31 J11.0_64/lib/ -rwxr-xr-x apsl 1 JENKINS USERGRP 98304 May 16 20:13 J11.0_64/lib/default/libcuda4j29.so
Each line has extended attributes and the file size.
- If the first character is “d” then this is a directory,
- if it is a “-“, it is a file.
- The sixth column is the size of the file.
You can use the following command to give the size of all of the files.
cat aout | awk ' substr($0,1,1) == "-" {print $0}' | awk '{sum+=$6;}END{print sum;}'
This command
- passes the file aout into the awk command
- awk… if the first character is a “-” =, it is a file, so print the record
- awk…add up the 6th column and display the final result.
This gave me 5.24403e+08 (bytes) for the size of all of the files. You need to add perhaps 5-10% for overhead (for example directory entries). If you specify a secondary extend to the data set, it will try to expand if the ZFS fills up.
I created the ZFS
//IBMUZFS JOB ,' ',COND=(4,LE) RESTART=MOUNT
//DEFINE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE JVB800.V11.ZFS CLUSTER
SET MAXCC=0
DEFINE -
CLUSTER -
(NAME(JVB800.V11.ZFS)-
VOLUMES(USER02) -
LINEAR -
MEGABYTES(500 254) -
SHAREOPTIONS(3 3))
/*
//FORMATFS EXEC PGM=IOEAGFMT,REGION=0M,COND=(0,NE,DEFINE),
// PARM=('-aggregate JVB800.V11.ZFS -compat')
//SYSPRINT DD SYSOUT=*
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//*
and mounted it over the directory you created above
//MOUNT EXEC PGM=IKJEFT1A,COND=((0,NE,DEFINE),(0,NE,FORMATFS))
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
MOUNT FILESYSTEM('JVB800.V11.ZFS') TYPE(ZFS) +
MOUNTPOINT('/usr/lpp/java/J11.0.19.0_64') +
MODE(RDWR) PARM('AGGRGROW') AUTOMOVE
/*
You can use the MOUNT command in the BPXPRMxx member
MOUNT FILESYSTEM('JVB800.V11.ZFS') TYPE(ZFS)
MOUNTPOINT('/usr/lpp/java/J11.0.19.0_64')
MODE(RDWR) PARM('AGGRGROW') AUTOMOVE
Note: without the “+” signs.
Size after unpacking is similar to the calculated value above.
After the file was unpacked the command
df -P /usr/lpp/java/J11.0_64
reported
Filesystem 512-blocks Used Available Capacity Mounted on JVB800.V11.ZFS 1178880 1045634 133246 89% /usr....
“Used blocks” 1045634 * 512 bytes gives 535364608 = 5.4e+08
The size calculation above is close to the final space used.
Unload the file
When I first unpacked one of the files I got messages about not being authorised to set the shared library attribute(l) in the directory – even though my userid was a super user. I had to define a security profile, and give my userid access to it.
RDEFINE FACILITY BPX.FILEATTR.SHARELIB UACC(NONE) PERMIT BPX.FILEATTR.SHARELIB CLASS(FACILITY) ID(IBMUSER) ACCESS(READ) SETROPTS RACLIST(FACILITY) REFRESH
Finally! In Unix System Services I issued the commands
cd /usr/lpp/java/J11.0.19.0_64 pax -ppx -rvzf /tmp/ibm-semeru-certified-jdk_s390x_zos_11.0.19.0.pax.Z
This unloaded the files into /usr/lpp/java/J11.0_64/J11.0_64
Discussion about where to put the file system.
If I had an existing directory /usr/lpp/java/J11.0_64 I could have mounted the new ZFS at this address, and issued
cd /usr/lpp/java pax -ppx -rvzf /tmp/ibm-semeru-certified-jdk_s390x_zos_11.0.19.0.pax.Zf
and this would have unload the files into /usr/lpp/java/J11.0_64, overwriting what was already there. See Follow the instructions to install Java on z/OS and screw up production. Think carefully where you want to put your files.