Month: March 2021

What certificates do I need for AMS with a client going to z/OS

The MQ documentation usually shows a local application doing a put and get, to show “how simple” it is.
Most people have a more complex environment where they have a mixture of z/OS and midrange, and clients into both.

This blog posts shows how I got clients to work with AMS on z/OS. My starting point was a working z/OS queue manager with working Java client able to put and get messages from a queue. I then enabled AMS on the z/OS queue manager.

Getting the z/OS end working

The userids on z/OS used a CA signed certificate (which is more typical in production than a self signed certificate).

Set up the xxxxAMSM keystore

My AMSM adress space has a started task userid of START1.

The xxxxAMSM address space needs a keystore, and the CA for the certificates on z/OS

#RACDCERT ID(START1) DELRING(drq.ams.keyring)
RACDCERT ID(START1) ADDRING(drq.ams.keyring)
RACDCERT ID(START1) CONNECT(CERTAUTH LABEL(‘TEMP-CA’) –
RING(drq.ams.keyring))
RACDCERT LISTRING(drq.ams.keyring) ID(START1)

Set up the keyring and certificate for the tso userid

Userid COLIN needs a keyring and an RSA certificate to identify COLIN. It is signed by “TEMP-CA”.

#RACDCERT ID(COLIN) DELRING(drq.ams.keyring)

RACDCERT ID(COLIN) ADDRING(drq.ams.keyring)

#RACDCERT ID(COLIN) DELETE(LABEL(‘AMS’))

#It defaults to certificate type RSA
RACDCERT ID(COLIN) GENCERT –
SUBJECTSDN(CN(‘COLIN’) O(‘SSS’)) –
SIZE(2048) –
SIGNWITH (CERTAUTH LABEL(‘TEMP-CA’)) –
WITHLABEL(‘AMS’)

RACDCERT id(COLIN) ALTER(LABEL(‘AMS’))TRUST

# add the certificate to user’s keyring
RACDCERT ID(COLIN) CONNECT(RING(drq.ams.keyring ) –
ID(COLIN) –
DEFAULT –
LABEL(‘AMS’))

# Connect the user’s public certificate to the xxxxAMSM keyring
#Usage(SITE) only exposes the public certificate, the private certificate
# is not visible in the keyring
RACDCERT ID(START1) CONNECT(ID(COLIN) LABEL(‘AMS’) –
RING(drq.ams.keyring) USAGE(SITE))

RACDCERT LISTRING(drq.ams.keyring) ID(COLIN)

RACDCERT ID(COLIN) LIST

# export it so we can send public key to Linux machine.

RACDCERT ID(COLIN) EXPORT(LABEL(‘AMS’))-
DSN(‘COLIN.AMS.PEM’) –
FORMAT(CERTB64) –
PASSWORD(‘password’)

# give the userid access to the keyring

permit IRR.DIGTCERT.LISTRING class(facility) access(read) id(COLIN)

setropts raclist(facility) refresh

FTP the COLIN.AMS.PEM certificate to Linux as ASCII.

Make the queue AMS protected

You need to use setmqspl to define the level of protection 

// EXPORT SYMLIST=* 
// SET QM=CSQ9
//CSQ40CFG EXEC PGM=CSQ0UTIL, 
//         PARM='ENVAR("_CEE_ENVFILE_S=DD:ENVARS") /' 
//STEPLIB  DD DSN=COLIN.MQ921.SCSQANLE,DISP=SHR 
//         DD DSN=COLIN.MQ921.SCSQAUTH,DISP=SHR 
//ENVARS   DD DSN=COLIN.MQ921.SCSQPROC(CSQ40ENV),DISP=SHR 
//EXPORT   DD SYSOUT=* 
//SYSPRINT DD SYSOUT=* 
//SYSIN    DD *,,SYMBOLS=(JCLONLY)  
 setmqspl -m &QM.
   -p AMSQ 
   -remove 
 setmqspl -m &QM.
   -p AMSQ 
   -s SHA256 
   -e AES256 
   -a CN=COLIN,O=SSS 
   -a "CN=testuser, O=aaaa, C=GB" 
   -r "CN=testuser, O=aaaa, C=GB" 
   -r CN=COLIN,O=SSS 
dspmqspl -m &QM. 
dspmqspl -m &QM. -export

Tell the xxxxAMSM address space to pick up the changes

You can use refresh all | refresh policy to get the the xxxxAMSM address space to use the changed policy information.

F xxxxAMSM,REFRESH ALL

or

F xxxxAMSM,REFRESH POLICY

Set up Linux

Create the keystore.conf

By default the configuration file used by AMS is in $HOME/.mqs/keystore.conf . I used different files, so I could test different scenarios with just one userid. I could set up a files called Alice.conf and Bob.conf so my ID can have different roles.

The location of the keystore.conf is available using

  • export MQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf for C program and
  • -DMQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf for java programs

Create a file for the configuration (/home/colinpaice/mqamsclient/ks.conf)

JKS.keystore = /home/colinpaice/ssl/ssl2/trust.jks
JKS.certificate =testuser

JKS.encrypted = no
JKS.keystore_pass = zpassword
JKS.key_pass = zpassword

# in V9 JKS.provider is ignored
# JKS.provider = IBMJCE

The options are documented here.

The key_pass is to access the private key within the keystore, see -keypass here. Because you are storing the clear text password in the file, it is recommended to use runamscred to encrypt the passwords.

Create the Linux keystore and certificate

I have scripts which generate my certificates, so I modified an existing one. I created a certificate with CN=testuser,O=aaaa,C=GB.

name=”testuser”
rm $name.pem $name.key.pem $key.csr $key.p12

# create an RSA private key
openssl genpkey -out $name.key.pem -algorithm RSA -pkeyopt rsa_keygen_bits:2048

# Create a certificate requests – so we can get it signed
openssl req -config eccert.config -passin password -sha384 -new -key $name.key.pem -out $name.csr -outform PEM -subj “/C=GB/O=aaaa/CN=”$name -passin file:password.file -passout file:password.file

#use this CA
ca=”carsa1024″

# sign it
openssl ca -config openssl-ca-user.cnf -policy signing_policy $ext -md sha256 -cert $ca.pem -keyfile $ca.key.pem -out $name.pem -in $name.csr $enddate -extensions clientServer

#Create the .p12 file so we can import it into the .jks
openssl pkcs12 -export -inkey $name.key.pem -in $name.pem -out $name.p12 -CAfile $ca.pem -chain -name $name -passout file:password.file -passin file:password.file

# remove the old one and import it
/opt/mqm/bin/runmqckm -cert -delete -db trust.jks -type jks -label $name -pw zpassword

/opt/mqm/bin/runmqckm -cert -import -target trust.jks -target_type jks -file $name.p12 -label $name -pw password -target_pw zpassword

#Add the z/OS user’s public key, so we can encrypt with the certificate and public key
/opt/mqm/bin/runmqckm -cert -add -db trust.jks -type jks -file zCOLIN.PEM -label zcolin -pw zpassword

# extract the Linux certificate and public key, along with its CA.
runmqckm -cert -extract -db trust.jks -pw zpassword -label $name -target $name.PEM -format ascii -type jks

exit

FTP the Linux public key to z/OS and add it to the keyring

The public key for testuser was extracted to testuser.PEM ( $name.PEM above). FTP this to z/OS (in ASCII). This became file COLIN.TESTUSER.PEM on z/OS.

To simplify the administration, you could setup an ID to “own” the certificates for each enterprise you work with. For example certificates from YOURORG are attached to ID(YOURORG).

I’ve used id ADCDA.

#Try to delete it if needed
#RACDCERT ID(ADCDA) DELETE (LABEL(‘LINUXTESTUSER’))


RACDCERT ADD(‘COLIN.TESTUSER.PEM’) –
ID(ADCDA) WITHLABEL(‘LINUXTESTUSER’) TRUST

# Connect it to the xxxxAMSM’s keyring
RACDCERT ID(START1) CONNECT(RING(drq.ams.keyring ) –
ID(ADCDA) LABEL(‘LINUXTESTUSER’))

Tell xxxxAMAM to pick up the updated keystore

You can use refresh all | refresh keystore to refresh the xxxxAMSM keyring information

f CSQ9AMSM,REFRESH ALL

or

f CSQ9AMSM,REFRESH keyring

Run the java client

. /opt/mqm/java/bin/setjmsenv64

#Java does not used export MQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf

ksc=”-DMQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf”
lib=”-Djava.library.path=/opt/mqm/java/lib64″

#java $lib $ksc JmsBrowser -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

#java $lib $ksc JmsConsumer -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

java $lib $ksc JmsProducer -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

Any error messages are written to file mqjms.log.0 .

Other AMS blog posts

Installing AMS on z/OS

Before you start to configure AMS on z/OS, you need to understand if you are licensed for AMS, see here. You need to understand this so you know what value to use in your AMSPROD. If you use the wrong value, your billing may be wrong.

I used the instructions here. There are a lot of steps, but they are clear, and worked. The AMS address space is started by the queue manager, once all the set up has been done. If you try to start it by hand, it will fail.

If you have to put in change requests for changes to security profiles you may want to build one big list for the security team to do (or perhaps better,create the JCL with all of the commands, and ask the security team to run the commands for you). The security team may have views about the granularity of access to keyrings (using FACILITY (a user can access any keyring) or RDATALIB (you give access to a specific ring <ringowner>.<ringName>.LST)).

In Enable Advanced Message Security, I created a new zparm module, for example CSQZAMSP with the SPLCAP=YES and the AMSPROD=xxxx value.

I started the queue manager with

%csq9 start qmgr parm(csqzamsp)

If you need to start the queue manager without AMS, just use your previous version.

Once MQ starts, and the AMS address space starts.

If you want to change the AMS policy you need to use CSQ0UTIL.

If you change the keyring, or the AMS policy you need to refresh the AMS address space.

F qmgrAMSM,REFRESH KEYRING
F qmgrAMSM,REFESH POLICY
F qmgrAMSM,REFRSH ALL

If you cancel the xxxxAMSM address space the queue manager will end.

Generation of certificates.

AMS only supports RSA certificates, so you cannot use Elliptic Curves, you must use GENCERT.. RSA ( or let it default to RSA).

Other AMS blog posts

Installing AMS on midrange for end to end message protection

Installing AMS on the server.

I tried to follow the instructions here but they use .rpm files. I had downloaded the MQ Developer Edition from here.

The download was mqadv_dev921_ubuntu_x86-64.tar.gz.

I extracted the ./ibmmq-ams_9.2.1.0_amd64.deb file, and used

sudo apt install ./ibmmq-ams_9.2.1.0_amd64.deb

I don’t know what you need to install just for the client.

Testing the setup

A scenario is documented Quick Start Guide for AMS on AIX. There is not a quick start guide for Linux; and the AIX guide wasn’t very quick. You have to create additional userids (which my “Enterprise security team” will not allow) do additional setup for these ids, and did not tell you how to clean up afterwards.

Below is my “quicker start to configuring AMS on Ubuntu”

Even quicker start up guide for AMS on Linux (and other Unix platforms), and this would be a good start for Windows

The examples in the IBM documentation implement the classic picture of Alice sending an encrypted message to Bob. This requires a lot of set up, new queue manager, userids etc – most of which are not needed when you use AMS.

In the example below, it takes a working system and adds AMS functions to an existing queue.

Note: although Structure of the keystore.config file refers to the different key store parameters, for example CMS and JKS, the C client needs cms keystore.

A userid using AMS needs a key store and a certificate; and a file to hold configuration information.

In summary the steps are:

  • Initial set up – to define shell variables
  • Define a queue to be protected, and an alias
  • Check that your userid can put and get messages to/from this queue
  • Define the keystore and certificate
  • Change the queue to have AMS protection
  • Check the id can put to the queue
  • Checking the message on the queue is protected
  • Getting the message

I used a shell script, and commented/uncommented statements as I progressed. I did this as it took several iterations before it all worked. It was easy to mistype a parameter, and hard to tell when you had made a mistake.

Initial set up – to define shell variables

It is less error prone to use environment variables to hold constants to avoid spelling mistakes, or using the wrong case.

export dn=”CN=CCP,C=GB,O=MINE”
export mydir=”$HOME/.mqs”
export keystore=$d/”colins”
export label=”COLINS

# create the default directory
mkdir $mydir -p

# optionally clear out this directory
# rm $mydir/*

#create the AMS configuration file
echo “cms.keystore = $key” > $mydir/keystore.conf
echo “cms.certificate = $label” >> $mydir/keystore.conf

Define a queue to be protected and an alias

If you access the protected queue you will get the AMS protection. If you define the queue as the target of an alias queue, you can bypass protection and see the message on the queue, so you can check it is protected as you expect.

You can use runmqsc to

define ql (AMSQ)
define qalias(AAMSQ) targq(AMSQ)

Or use a command in a shell script

qm=”QMA”
q=”AMSQ”

echo “define ql ($q)” | runmqsc $qm
echo “define qalias(A$q) targq($q)” |runmqsc $qm

Check that your userid can put and get messages to/from this queue

Use the amqsput sample to put data, until a null line is entered, then use amqsget to get the messages, and leave the queue empty. Note the amqsput does not prompt for input, just enter the data and press enter; no data and enter ends it.

/opt/mqm/samp/bin/amqsput AMQS QMA
/opt/mqm/samp/bin/amqsget AMQS QMA

Resolve any problems. Do this before your protect the queue, so you know you have a working environment.

Define the keystore and certificate

C programs need a cms keystore. Using the environment variables defined above

runmqakm -keydb -create -db $key.kdb -pw passw0rd -stash
runmqakm -cert -create -db $key.kdb -pw passw0rd -label $label -dn $dn -default_cert yes

Change the queue to have AMS protection

setmqspl -m $qm -p $q -s SHA1
dspmqspl -m $qm -p $q

#remove it if required
#setmqspl -m $qm -p $q -remove

Check the userid can still put to the queue

Now the setmqspl command has been issued for the queue, any put will create a protected message. It uses the default environment variable MQS_KEYSTORE_CONF pointing to “$HOME/.mqs” to point to the configuration file $HOME/.mqs/keystore.conf (created above).

/opt/mqm/samp/bin/amqsput $q $qm

Enter some data (AAAAA) and press return, then press return again to give null data.

If this was not successful, resolve the problems. See below for some help on this.

Check the message on the queue is protected.

A couple of times I though the AMS set up had worked, as the put and get both worked without error. I later found that the messages were not protected. I like to check the messages on the queue are protected.

You can browse the message from the alias queue, it should contain additional data before the message, including distinquished name of the signer.

/opt/mqm/samp/bin/amqsbcg $qm $q

This displays the data in hex. The data should start with the header PDMQ showing there is protected data on the queue. Within the PDMQ header, should be the encoded signing information.

If you get the same data as you put (AAAAA) then the protection has failed; check the set up.

Example data showing the content of the queue after the data has been signed.

00000000:  5044 4D51 0200 0200 7000 0000 7000 0000           'PDMQ....p...p...'
00000010:  0400 0000 B804 0000 0800 0000 0000 0000           '................'
00000020:  4D51 5354 5220 2020 0000 0000 0000 0000           'MQSTR   ........'
00000030:  0000 0000 0000 0000 414D 5351 2020 2020           '........AMSQ    '
00000040:  2020 2020 2020 2020 2020 2020 2020 2020           '                '
00000050:  2020 2020 2020 2020 2020 2020 2020 2020           '                '
00000060:  2020 2020 2020 2020 0000 0000 0000 0000           '        ........'
00000070:  3082 0527 0609 2A86 4886 F70D 0107 02A0           '0..'..*.H.......'
00000080:  8205 1830 8205 1402 0101 310F 300D 0609           '...0......1.0...'
00000090:  6086 4801 6503 0402 0305 0030 1706 092A           '`.H.e......0...*'
000000A0:  8648 86F7 0D01 0701 A00A 0408 4141 4141           '.H..........AAAA'
000000B0:  4141 4141 A082 0300 3082 02FC 3082 01E4           'AAAA....0...0...'
000000C0:  A003 0201 0202 0859 7651 E018 1827 E030           '.......YvQ...'.0'
000000D0:  0D06 092A 8648 86F7 0D01 0105 0500 301C           '...*.H........0.'
000000E0:  310B 3009 0603 5504 0613 0247 4231 0D30           '1.0...U....GB1.0'
000000F0:  0B06 0355 0403 1304 4343 5032 301E 170D           '...U....CCP20...'
00000100:  3231 3033 3239 3134 3433 3335 5A17 0D32           '210329144335Z..2'
00000110:  3230 3333 3031 3434 3333 355A 301C 310B           '20330144335Z0.1.'
00000120:  3009 0603 5504 0613 0247 4231 0D30 0B06           '0...U....GB1.0..'
00000130:  0355 0403 1304 4343 5032 3082 0122 300D           '.U....CCP20.."0.'
00000140:  0609 2A86 4886 F70D 0101 0105 0003 8201           '..*.H...........'

Where

  • PDMQ is the AMS header
  • AMSQ is the original queue name
  • AAAAAAAA is the original message data – note this is visible because encryption is not being used
  • GB comes from C=GB in the DN. This is part of the encoded DN
  • CCP2 comes from CN=CCP2 in the DN. This is part of the encoded DN

Destructively get the message

/opt/mqm/samp/bin/amqsget AMQS QMA

Should display the original message.

If this was not successful, resolve the problems. See below for some help on this.

Encrypt the messages

If you specify “use encryption with AES256”, and “this id” as a recipient, you will get encrypted messages on the queue

setmqspl -m $qm -p $q -e AES256 -s SHA512 -r $dn

and repeat the put, and browse from the alias queue, you will see the the PDMQ in the header, and parts of the signing DN in the header, as before. The message payload will be encrypted.

Giving another user access or using a different DN and certificate

By default, the configuration file ~/.mqs/keystore.conf contains information as to which key store, and which key within the key store to use.

You can override the location of the config file using the environment variable MQS_KEYSTORE_CONF .

This means one userid can use a different keystore.conf file file for each application, or you can have a group of userids using the same keystore.conf and so the same certificate keystore file, and certificate.

If it did not work…

Check the error logs, for example the /var/mqm/qmgrs/QMA/errors/*01* .

Most of the errors I introduced had a clear message description of the problem.

GSKIT return codes are described here.

While setting up, it is worth clearing the queue if you get errors, for example there may be a message left on the queue which you are not expecting.

Here is the full script I use ( with extra stuff in it)


d="$HOME/.mqs"
key=$d/"colin"
mkdir $d -p
rm $d/*
dn="CN=CCP2,C=GB"
qm="QMA"
q="AMSQ"
rm "COLIN.pem"
echo "define ql ("$q")" | runmqsc $qm
echo "define qalias(A$q) targq($q)" |runmqsc $qm
runmqakm -keydb -create -db $key.kdb -pw passw0rd -stash

# chmod +r $key
runmqakm -cert -create  -db $key.kdb -pw passw0rd  -label COLIN -dn $dn -default_cert yes
runmqakm -cert -extract -db $key.kdb -pw passw0rd  -label COLIN -target COLIN.pem  -format ascii 
/opt/mqm/bin/runmqckm -cert -add  -db $key.jks -type jks  -file ~/ssl/ssl2/zADCDB.pem -label zADCDB -pw passw0rd
chmod 600 $key.kdb
# chmod 777 $key.kdb

echo "cms.keystore = $key" > $d/keystore.conf
echo "cms.certificate = COLIN" >> $d/keystore.conf 
chmod 777 $d/keystore.conf 
cat $d/keystore.conf 
setmqspl -m $qm -p $q   -remove

# setmqspl -m $qm -p $q   -s SHA1 -a $dn 
setmqspl -m $qm -p $q  -s SHA512 
# setmqspl -m $qm -p $q  -e AES256  -s SHA512 -r $dn
dspmqspl -m $qm -export 
#sudo rm /var/mqm/trace/*
# strmqtrc -m $qm

export MQS_KEYSTORE_CONF=/home/colinpaice/.mqs/keystore.conf
printenv |grep MQ
# print echo in bold 
BOLD='\033[1m'
NC='\033[0m' # No Color

echo "${BOLD}type some data and press enter, then press enter again${NC}"
/opt/mqm/samp/bin/amqsput $q  $qm
#display it in hex from qalias
opt/mqm/samp/bin/amqsbcg A$q $qm 
# /opt/mqm/samp/bin/amqsgbr $qm $q  
/opt/mqm/samp/bin/amqsget $q $qm

# less /var/mqm/qmgrs/$qm/errors/*01*

Other AMS blog posts

Pushing the security envelope – protect the ccdt

Every year the security boundaries are being pushed to protect more resources. For example

  • Encrypt DNS lookup requests. Without this, bad guys can see what sites you are looking at. Or, depending on your view point, good guys can see what sites a bad guy is looking at.
  • Encryption of cookies in http traffic. In https, cookies are encrypted, but some sites just use https for the logon, and then use http for normal traffic.
  • Padding of strings, so the length of data cannot be seen. If you send up an encrypted password and the length is 4 characters, you can guess that the password is short.

MQ CCDT

The CCDT in MQ defines parameters for a client connection, it contains, for example,

  • The connection host address and port
  • The cipher specification
  • The certificate label to use
  • The certificate peer name (what is in the certificate sent from the server)

You may want to protect this information, so bad guys cannot intercept and change it.

The CCDT can be stored in the local file system, or it can be accessed from the network using http, or ftp.

This file will need to be updated for example, when there are changes to the configuration of the back-end servers, or the cipher spec is being upgraded.

Local file system.

You need to be careful how you update the local file system. Most enterprises have technology which securely distribute updates to client machines, for example with operating fixes, application upgrades, and configuration information.

Emailing a file, or simple FTP may not be secure enough for some organisations. Secure FTP (SFTP) or FTPS may be good enough.

Getting the CCDT from the network

Using HTTP and having the CCDT on a central site has many advantages. You update it, and the next time the client starts, it will phone home, and get the updated CCDT. You may want to use a secure site so only production machines can access the production CCDT files, and only test machines can access the test CCDT files.

The MQ documentation has

Unauthenticated connections

export MQCHLLIB=ftp://myhost.sample.com/var/mqm/qmgrs/QMGR/@ipcc
export MQCHLLIB=http://myhost.sample.com/var/mqm/qmgrs/QMGR/@ipcc
export MQCHLLIB=file:///var/mqm/qmgrs/QMGR/@ipcc

Authenticated connections

export MQCHLLIB=ftp://myuser:password@myhost.sample.com/var/mqm/qmgrs/QMGR/@ipcc
export MQCHLLIB=http://myuser:password@myhost.sample.com/var/mqm/qmgrs/QMGR/@ipcc

And says : If you want to use authenticated connections you must, as with JMS, provide the user name and password encoded in the URL.

To me these are not very secure, as they could be intercepted, and you could be sending userid and password on an potentially unencrypted connection.

In my Java program I managed to use https://127.0.0.1:9444/myccdt.json, to get the CCDT from my little Python https server. The url can be http, https, ftp, or file, so the only two “secure” ones are https: and file:, but the MQ documentation does not mention https.

I could not get a C program to work when I used export MQCCDTURL=”https://127.0.0.1:9443/myccdt.json”. It hung during the TLS handshake.

To make your CCDT secure, you are stuck with getting the file securely to the machine and using export MQCCDTURL=”file:///…”, which means it will updated whenever your machine gets the automatic updates – perhaps once a week.

Remember you can never be too paranoid when it comes to security. If someone steals your wallet or purse you know. If someone copies your identity file you may never know.

Understanding setmqspl to set up AMS definitions.

At first glance, how difficult can it be to understand a command with only five operands? As I do not usually write about easy topics, you may be able to guess that there is some subtlety to it.

Background to AMS protection

There are three protection models in AMS. See the IBM Knowledge centre.

  1. Integrity.
  2. Encryption.
  3. Privacy = Encryption and Integrity.

Integrity

A checksum is made of the message contents, and the check sum is encrypted with the senders private key. The sender’s public key, the encrypted checksum, and the original payload are put into a message and put to a queue. This can be sent to a remote queue manager, or remain on the local queue manager. The recipient, performs the same checksum calculation on the original payload, decrypts the embedded checksum value (using information from the public certificate). If the checksums match – the messages is unchanged we have integrity

With Integrity, the getting queue manager knows the DN of the userid doing the signing.

With integrity, you need to specify which checksum technique to use, and at the getting end, you can optionally specify a list of authorised senders, and AMS will check the DN of the sender is in the list.

Confidentiality

The message contents are encrypted with a symmetric key. The key is then encrypted for the list of recipients, using the each recipient’s public key. A package of [(recipient, encrypted key) (recipient, encrypted key).. encrypted message] is put to a queue. When the message is got, the getter locates the (recipient, encrypted key) for it, uses its private certificate to decrypt the key to the data, and uses this to decrypt the message content.

With encryption you need to specify which encryption technique to use, and the list of recipients. For each recipient, AMS goes to the key store at the putting end to get the public certificate.

Privacy

A combination of Integrity and Confidentiality. The data is encrypted, and signed. The getting queue manager knows the DN of the sender.

setmqspl syntax

The syntax is

  • setmqspl
  • -m name queue manager name
  • -policy name the queue name – known as the policy name
  • -e value which cipher spec to use – or none
  • -r DN a list of 1 or more recipient’s DN
  • -s value the signing algorithm to use or none
  • -a DN a list of 0 or more acceptable signers DN values (the putter’s DN).

I’ll focus on the -e, -r, -s and -a values.

Looking at a distributed MQ environment, should make it clearer

We have a putter queue manager. Messages go to another queue manager (or stay on the same queue manager) where the messages are got.

Integrity

At the putting queue manager you need

  • setmqspl -m qmname -policy queuename -s algorithm
  • Pick an algorithm from MD5, SHA1, SHA256, SHA384, SHA512, where the SHAnnn are more secure (it is all relative) but MD5 may be faster. The algorithm you pick will be used.
  • -r and -a values are ignored from an integrity and signing perspective

At the getting end you need

  • setmqspl -m qmname -policy queuename -s algorithm <-a DN… >
  • You can specify any algorithm, (as long as the value is not NONE). This says expect a signed payload. The value is not used.
  • The -a DN1 -a DN2… the optional list of values are the authorised DNs from the certificate of the userid that put (and signed) the message. If none are specified, then no checking is done. If there is at least one specified, then the signer’s DN is checked. If it is in the list, the message is returned to the application, if it is not in the list, the message get is rolled back, or put on the SYSTEM.PROTECTION.ERROR.QUEUE queue.

Confidentiality

At the putting end you need

  • setmqspl -m qmname -policy queuename -e algorithm -r DN… .
  • Pick a digital encryption algorithm from RC2,DES,3DES,AES128,AES256 , where the AESnnn are better than the others. The algorithm you pick will be used.
  • The -r value is the DN of a potential recipient. The payload key is encrypted for each DN in the list.

At the getting end you need

  • setmqspl -m qmname -policy queuename -e algorithm .
  • You can specify any algorithm (as long as the value is not NONE), then this says expect an encrypted message. The value is not used.
  • Any -r DN values will be ignored, as they are not relevant. The recipients are identified by the presence of the encrypted key in the payload.

Privacy

See above under Integrity and Confidentially

Specifying a DN

The documentation says

  • DN attribute names must be in uppercase.
  • Commas must be used as a name separators.
  • To avoid command interpreter errors, place quotation marks around the DNs.
  • The attribute values in the DN are case sensitive so, for example, CN=USERID1 is different from CN=userid1.

There is a convention (not a standard as such) that there is a hierarchy in the terms, for example starting with CN= on the left, and C= on the right. Some key stores, such as RACF will arrange the parts of the DN into this order.

AMS allows blanks between parts, for example “CN=COLIN , O=SSS”. These padding blanks are not present in certificates, but significant blanks such as within “CN=COLIN PAICE” are kept.

When trying to resolve AMS “missing certificates” problems, I found it easier when I had the parts of the DN in the right order, and did not have extra blanks between terms. It means I could just cut and paste the DN when searching in the key store.

Other AMS blog posts

Brain check and reset, needed for AMS keystores on midrange.

I was working with AMS between midrange and z/OS and could not get my head around key store set up. The concept was relatively easy – it was the practice that made my head hurt.

I was reminded of a conversation with someone from MQ midrange development over the number of users that MQ should support, and whether a linked list, or binary tree would be better. The midrange guy said linked list would be OK, and easier to implement, I said a linked list would not scale. We had a break, and went for a coffee. I asked him “what do you call a large number of userids?” He replied “about 100”. “Ah, I see the problem. On z/OS 1000 is a small number of userids, and 100,000 is a large number of userids”. We then agreed the problem was too hard to solve.

I have the same sort of problem with AMS keystores. If you have 1000 client machines using MQ AMS, how do you keep the key store up to date? If someone joins the department, then the userid cannot use AMS until all of the key stores have been updated to include the userid’s public certificate.

Other applications using TLS and encryption have a key store to contain just your user’s private key information, and a trust store containing the public keys of users, or Certificate Authority (certificates you trust).
The theory being you create someone’s private key, once only, and the file with public keys can be downloaded whenever it changes, as part of the enterprise refresh of machines (think Microsoft updates which gets pushed to machines).

With AMS, there is a weakness, that only one key store is used which contains both the private, and the public keys. It is now harder to keep the key store current.

You cannot just download a key store for every one to use, as it would have all the private keys in it. (I could then use your private key, and pretend to be you.)

I had problems trying to come up with a process for updating all of the key stores, adding new certificates, and removing unwanted ones. This is where my brain had problems trying to find a solution. I gave up for the evening, and came back next morning when my brain had had a wake-up reset, and I could see a solution.

You download the public certificates file whenever there are update to it, then, on the client machine, you import the private key to this key store, rename the files ( current to old, new to current), restart the client, and then delete the old key store. This should be a simple process to implement.

If you allow a day for the change management process to approve the change to the key store, overnight download of the change, and a restart of the client at start of day, you should be able to get a change implemented in a couple of days!

Of course it is not that simple, as you have the .cms key stores used by C applications, and the Java keystores used by Java applications – so you have to worry about two keystores! But this is just a SMOP (a Small Matter Of Programming).

Other AMS blog posts

Overview and challenges of using end to end message encryption (AMS)

MQ provides end to end protection of messages. This covers

  • Integrity – where the message can still be read – but any changes to the message are detected. You can check the id of the person that sent it.
  • Encryption – where the payload is encrypted and only the ids with the correct private key can decrypt it.
  • Privacy – A combination of Integrity and Encryption, where only specified ids can decrypt it, and you can check the id of the person that signed it.

The messages are protected on the network, in memory and on disk. You do not need to use a channel with TLS for protection (but you may want to use TLS for authentication), as the data is protected before sending.

Any headers that are involved in ‘routing’ like MQMD, XQH, RFH2, etc are skipped by AMS because the queue manager still needs to know what to do with the messages in terms of delivery and selection before they are unprotected. Other message headers like PCF command headers and custom application defined headers/formats can still be protected. Sensitive data should not be held in message properties as that data is used in routing/selection. Thanks to Jon Rumsey for the correction to this.s

Background about encryption and signing of data

You may need to read some of the following section more than once, because it reference private keys and public keys and it is hard to work out what is happening.

The encryption techniques used are special. There is a private key and a public key. (Think of your passport as a private key, and your name and address is your public key). You can use functions

  • You do encrypt(private key, payload) to produce an encrypted payload, send it to me, and then I use decrypt(public key, encrypted payload) to recreate the original payload. I know it came from you, because I needed your public key to decrypt it.
  • I do encrypt(public key, payload) to produce an “encrypted” payload, and send it to you. Only you can use decrypt(private key, encrypted payload) with your private key to recreate the original payload.

With this

  • I can encrypt data using your public key that only someone with access to the private key (you) can decrypt
  • I encrypt with my private key, and anyone with my public key can decrypt it – and you know it came from me – because you needed my public key.

Signing is simply taking a checksum of the data, and encrypting the value with the signer’s key. When you come to check the signature, you do the same check sum calculation, decrypt the “signed” value using my public key, and the values should match. The payload would look something like “Signed by CN=Colin,O=SSS,C=GB” encrypted checksum=2294567, my certificate; followed by the original data.

We can take this further. I sign the data with my private key, and then encrypt the data(including the signed data) with your public key, (keep up at the back there – I said it was hard to follow). When you process it, only you can decrypt it because it needs your private key, and you can check the signed data, and see it came from me. I have sent the message encrypted, and have mutual authentication.

With encryption a symmetric key is used to encrypt the data, because it is faster, and about as secure as encryption with an asymmetric key. When trying to break into enciphered data, the more data you have, the easier it is to break. This is why you can reuse the symmetric key for multiple records, but you should change it, for example after 10 MB has been processed (I don’t know the best figure – it is all a matter of risk).

Sending a message, many to many.

Sending a message from one person to another person looks fairly easy. It gets a bit harder when you have potentially many people in my organisation who are allowed to send a protected message to your organisation, and there may be many people who are allowed to get, and decrypt it.

Let the people in my organisation be CN=A,O=MYORG, and CN=B, O=MYORG, and the people in your organisation be CN=X,O=YOURORG, CN=Z,O=YOURORG.

If person CN=A wants to send an encrypted message so it can be decrypted by CN=X or CN=Z the processing is as follows.

  • Do a checksum calculation on the record.
  • Encrypt this checksum using CN=A’s private certificate.
  • Put a new field of CN=A,O=MYORG|| encrypted checksum value, CN=A’s public certificate before the record.
  • Create a symmetric key.
  • Encrypt the (signed data and original payload) with the symmetric key.
  • Take the symmetric key and encrypt it for CN=X,O=YOURORG using CN=X’s public certificate. Create a field of the Distinguished Name || encrypted key.
  • Take the symmetric key and encrypt it for CN=Z,O=YOURORG using CN=Z’s public certificate. Create a field of the Distinguished Name || encrypted key.
  • Build a record of the two “encryption fields” and the encrypted data
  • Put the data in a message and send it to the recipient(s).

Getting an encrypted message

The person with CN=X gets the message.

  • Scan down the message looking for CN=X,O=YOURORG. If found then extract the encrypted symmetric key. Decrypt this value with the CN=X’s private key. This gives you the key that was used to encrypt the customer data.
  • Decrypt the message using the key you just decrypted.
  • Extract the first field and see it is signed data. We can see that the checksum was done by CN=A,O=MYORG.
  • Check this name with your list of IDs that can send you messages.
  • Validate the signer’s certificate. Either use a CA, or if the certificate was self signed, check with the copy in your trust store. Note this certificate includes the public key.
  • Take the checksum value in the message and decrypt it using CN=A,O=MYORG’s public key from the payload. Do the checksum of the remainder of the message and compare the values – they should match.

Setting up your enterprise

You need to know the list of potential recipients of messages for each queue.

At the receiving end, if you want to check the message came from an authorised id, you need to know the list of potential senders.

If you’ve managed to keep up so far – it gets worse!

The management of these lists can be difficult. If someone joins my team, I would need to tell the other organisations that they need to add CN=NEWPERSON, O=MYORG into their list of potential senders, (and remove CN=IVE_RETIRED, O=MYORG. It will take a week to get this change through their change management system. Until this is done, the message could be rejected as coming from an unauthorised person.

In the same way, you need to update the AMS setmqspl definitions to add CN=NEWPERSON, O=MYORG to each queue, and when people join or leave the other enterprises you’ll need to update your list of potential senders.

Do not worry too much – there is a solution, see below.

Creating the policies

With MQ, the security policy is defined using the SETMQ Security PoLicy command (setmqspl) for example to define a queue at MYORG to send message to YOURORG. The -a option is for who the message is for

setmqspl -m PRODA -p SENDQUEUE -s SHA256 -e AES128
-a "CN=X, O=YOOURORG"
-a "CN=Y, O=YOOURORG"
-a "CN=Z, O=YOOURORG"

At YOURORG for the GETQUEUE you need the following. The -r option is who you can receive a message from

setmqspl -m YOQM -p GETQUEUE 
-r "CN=A, O=MYORG"
-r "CN=B, O=MYORG"
-r "CN=JO, C=GB,O=SOMEOTHERORG"

To change an entry you have to run whole command, you cannot add one “-a” or “-r”. I would keep a file for these, update file, and process the whole file. For two queues – use two files.

You also have to consider how to update the definitions for multiple queue managers. On z/OS I would have the MQ commands inline, and use a variable queue manager name &QM, which you set in the JCL. For midrange you could have a shell script and pass the queue manager name as a parameter.

What keys do I need?

The encryption is done using RSA, and so needs a certificate with an RSA private key.

When the message is signed

Each id putting a message needs access to a private key – this could be one key shared by multiple users.

When getting a message that has been signed, the recipients will need to verify the certificate sent within the signing data; either a CA in the signing chain, or the public certificate (for self signed certificates).

When encrypting the message

The id putting the message needs access to the public key of each recipients in the list.

The id getting a message which has been encrypted, needs access to the private key.

Maintaining the public keys

For each person in the -r or -a list you will need the digital certificate, and public key in a keystore or a keyring (on z/OS).

Each id on z/OS will need their own keyring. There is a keyring for the queue manager;s AMS address space with the public certificates on it.

On midrange MQ, each user has a keystore, with a combination of public keys, and private key(s).

Time to step back and think about what you want.

To me, it is clear that having every possible user having their own DN, and certificate quickly gets impractical. If you are working with people outside of your organisation do you want them to know the people’s names within your organisation?

You may want to have a departmental certificate for example CN=PAYROLL,O=MYORG, which members of your payroll team all use. You can control access to the queue using standard MQ security policies.

An intermediate configuration would be to have a proxy or server applications to change input messages from “partner protection” to “myorg protection”, and the reply messages changed from “myorg protection” to “partner protection”

  • You have an AMS protected input queue called EXTERNAL_IN. This has been configured to specify the DNs of the systems that can send you messages, so might specify CN=RECEIPTSPAYABLE,O=YOURORG as a valid sender.
  • If the get is successful, then put to a queue called INTERNAL_IN, specifying the DNs of the individuals in your Accounts department who are allowed to get one of the protected messages. If someone joins the Accounts department, you only need to change the configuration within your queue manager.
  • Someone (or an automated task) gets the message from INTERNAL_IN queue, it is decrypted automatically, the application does the processing and puts a message the AMS protected queue YOUR_BANK_INTERNAL, where it is encrypted by AMS.
  • Another proxy/server application in your enterprise gets the messages from YOUR_BANK_INTERNAL, checks the DN of the sender, and that they were authorised to put messages to the queue.
  • The server then puts to YOUR_BANK_EXTERNAL, using a DN of “CN=PAYROLL,O=MYORG”, and it is encrypted for the recipient.

This way you can isolate internal processing and external processing. Updating the internal list is within your control. You should not need to update the external lists (and certificates) once established.

Handling whoops’s

Some problems will result in the messages removed from the queue and put to a special error queue. You need to have a process in place to handle this. I had messages put to this queue when a userid tried to get an encrypted message, but the message had not been encrypted for this userid’s DN.

Other AMS blog posts

Using runamscred to encrypt the passwords in the keystore.conf file.

If you are using AMS on midrange, your keystore.conf can contain plaintext passwords to the Java private keys and keystores. This is not very secure.

You can encrypt these passwords using runamscred which takes the file, encrypts the passwords and updates the file. For example

Before

JKS.keystore = /home/colinpaice/ssl/ssl2/trust.jks
JKS.certificate = testuser
JKS.encrypted = no
JKS.keystore_pass = zpassword
JKS.key_pass = zpassword
JKS.provider = IBMJCE
amscred.keyfile=/home/colinpaice/mqamsclient/encryption.key

After

JKS.keystore = /home/colinpaice/ssl/ssl2/trust.jks
JKS.certificate = testuser
JKS.encrypted=yes
JKS.keystore_pass=<AMS>1!cu891gavGZlmnYY2DyLgbA==!W3mhqtHazVx/QDWKW/daJg==
JKS.key_pass=<AMS>1!XFS5rLL4JtlSUaAH5DdwCg==!Q6aC1547gtoqi9Q0Twlogw==
JKS.provider = IBMJCE
amscred.keyfile=/home/colinpaice/mqamsclient/encryption.key

This is documented (not very clearly) here.

The steps are

  • Create a file with a string in it. The documentation has Th1sIs@n3Ncypt|onK$y as an example.
  • Update your keystore.conf file to add a line to point to this file
    • amscred.keyfile=/home/colinpaice/mqamsclient/encryption.key .
  • This line is needed when you have Java programs.
  • Encrypt the passwords with
    • /opt/mqm/java/bin/runamscred -f keystore.conf -sp 1
  • This uses the amscred.keyfile entry to locate the encryption key. You can use -sf to specify a different file, but as the file has to have the amscred.keyfile statement. You might just as well use it, and be sure it works.!
  • The command changes JKS.encrypted to yes, and changes the passwords.

Systems management

Because the command changes the file contents, you may want to have a secure master file, master.conf, with the password in plain text in it. Make sure this file is secure (perhaps on a secure machine).

To manage the passwords

  • copy the file cp master.conf ks.conf.
  • encrypt the passwords using /opt/mqm/java/bin/runamscred -f ks.conf -sp 1.
  • send the ks.conf and the encryption key file to your end users.

Other AMS blog posts

pkcs11? pkcs12? .cms? .jks? .p12? .nss? which type of keystore should I use?

There are at least four types of keystore, and after an email exchange with someone, it became clear that the different types of keystore and how to use them, are not widely understood. It is one of those topics, that once you are an expert, this topic is obvious still difficult to understand. When you have little knowledge and just want to get a job done, it can be very confusing.

I’ve learned a lot from writing this post

  • There is no one tool to manage all your keys and certificate
  • You may need one keystore for your queue manager, another keystore for your C clients, another keystore for your Java clients, a keystore for a web browser and something else for curl!
  • You might need to use 3 tools to manage your keystores!

This post has sections on

This post started off as a few lines, then I though I had better explain the pre-req knowledge, and then the pre-req knowledge to the the pre-req knowlegde. I hope it provides a clear story.

What’s the difference between a trust store and a key store – when they are both called key stores?

You need a store to contain the certificates needed to check any certificates sent to the application. These are usually certificate authority certificates, but can also be self signed certificates. This store is known as a trust store – it is used to check that a certificate is trusted. None of this data is confidential, most of the information is in the public domain. One trust store could be shared by all applications, which makes the maintenance of it much easier than a trust store for each application/server.

You need a store to keep the private key used for encryption/decryption, this is known as the key store. This needs to be kept confidential. The keystore may have just one private key. You can use smart keys, or external keystores such as special USB devices.

Some products use just one store with the trust store and the private certificates combined. They also call this the keystore. If you want to isolate the keystores, you need multiple stores. This, in turn, means that if you update one trust certificate, you have multiple stores to update.

What does a store contain?

A keystore can contain

  • private information – such as you secret key which you use for encryption and decryption
  • public information, such as your Distinguished Name (CN=colin,O=SSS), and they key needed to encrypt data for me
  • trust data, if you send me a public key – can I validate it has not been tampered with?

What format is this data in?

This data is typically in one of three formats

  • Plain text
  • Binary
  • Binary portable (64 bit encoding)

Plain text

If you print a certificate or key, it comes out like

Certificate:
  Data:
    Version: 3 (0x2)
      Serial Number: 379 (0x17b)
      Signature Algorithm: ecdsa-with-SHA256
      Issuer: C=GB, O=SSS, OU=CA, CN=SSCA256
      Validity
        Not Before: Feb  9 09:35:07 2021 GMT
        Not After : Apr  9 17:40:01 2022 GMT
        Subject: C=GB, O=cpwebuser, CN=ecec
        Subject Public Key Info:
          Public Key Algorithm: id-ecPublicKey
            Public-Key: (256 bit)
            pub:
               04:5d:15:78:83:e0:ab:af:78:69:2f:14:51:fe:29:
            ...
                ASN1 OID: prime256v1
                NIST CURVE: P-256
...

Binary

The data is encoded using Abstract Syntax Notation(asn). For example you get a string of fields like

  • The following fields are a sequence
  • of length 24
  • it is a string format
  • the first item has length 8
  • the attribute type is 06 03 55 04 06 which means Country
  • the attribute value is GB
  • the second item is …

This would describe the C=GB in C=GB, O=SSS, OU=CA, CN=SSCA256.

Binary portable (base 64 encoding)

With the binary format, the data is a hex string, which is not very portable. For example if you FTP it to a remote site, you may get data conversion, and new lines changed. Often the data is converted to an intermediate form called base 64 encoding. Every 3 hex characters are converted to 4 “printable” characters A-Z,a-z,0-9 and some special characters.

The file looks like

-----BEGIN CERTIFICATE-----
MIICDjCCAbSgAwIBAgICAXswCgYIKoZIzj0EAwIwOjELMAkGA1UEBhMCR0IxDDAK
BgNVBAoMA1NTUzELMAkGA1UECwwCQ0ExEDAOBgNVBAMMB1NTQ0EyNTYwHhcNMjEw
...
hEzE3hSzUvbfLUKnua64AiEAnyKhrmBjXCVQ834VfjhNHYqjaN1PzXBE8Cv3jlWJ
9bA=
-----END CERTIFICATE-----

What are the keystore types?

When Secure Sockets Layer was being developed each major player came up with their own format for storing the certificate and key information.

  • Java had files with type .jks (java key store) for example keystore.jks
  • IBM had files with type cms (certificate management system) with file types like zzserver.crl, zzserver.kdb, zzserver.rdb, zzserver.sth. It also uses intermediate files such as .arm, which has base 64 encoding of a certifcate, as shown above.
  • Netscape had files in a NSS database, for example files in the directory /home/colinpaice/.pki/nssdb, are files cert9.db and key4.db. key4 is the keystore database (storing keys), and cert9 is the trust store database (storing certificates).
  • OpenSSL developed .p12 files which can contain certificates and keys. It also has .pem (base 64 encoded) and .der (binary) files for individual certificate, private key, and public key files .
  • Windows has .pfx files.
  • Smart cards where the keystore is on a special USB type device or other external Hardware Security Module, each have their own format keystore.

Standards were developed to work with keystores

When can different keystore types be used?

With C programs on mid range and z/OS, IBM products use GSKIT from IBM. On z/OS you can store them in the z/OS security manager (for example RACF) or on the Hardware Security Module in the processor.

Java program can use most types of keystore. You may need to configure the java.security configuration file with an entry like

  • security.provider.X=com.ibm.security.cmskeystore.CMSProvider
  • security.provider.Y=com.ibm.crypto.pkcs11impl.provider.IBMPKCS11Impl /home/colinpaice/mq/nitrokey.cfg

The first line includes support for cms files, the second includes support for a pkcs11 external keystore on an HSM (including its configuration file).

For Java programs, you configure the keystore using start-up options. The following example defines the keystore as an external USB HSM keystore, and the trust store is a openssl .p12 store.

-Djavax.net.ssl.keyStore=NONE
-Djavax.net.ssl.keyStoreType=pkcs11
-Djavax.net.ssl.keyStorePassword=12345678

-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/mytrust.p12
-Djavax.net.ssl.trustStorePassword=password
-Djavax.net.ssl.trustStoreType=pkcs12

These definitions along with the java.security, and its override, show the keystore is a pkcs11, and there is an entry in the java.security (above) pointing to the configuration file /home/colinpaice/mq/nitrokey.cfg for the pkcs11 definition. You can override or extend the options in the java.security file using configuration using

-Djava.security.properties=/home/colinpaice/mq/colin.java.properties

Do these keystores share information?

No, if you have a keystore (or a USB HSM) it shares no information with any other keystore. For example you could unplug the USB keystore, take it to another machine and plug it in, and the certificate etc will be available. You could send a .jks file, to yet another machine, and it could be used.

What can you use to administer the keystore.

IBM provided

You can use GSKCapiCmd_64 from IBM GSKIT to manage certificates and keys.

MQ has runmqakm as a command line tool which invokes GSKCapiCMD_64 under the covers.

You can also use the MQ command strmqikm to have a GUI to manage your keys and certificates. strmqikm just invokes the GSKIT ikeyman program.

These solutions feel a little dated, as I could not find if they support modern certificates capabilities; OCSP, and elliptic curves. They had a few bugs as well.

I’ve used openssl to create the “modern” private key and certificates, then imported them into the keystores using the IBM tools.

Oracle provided

With Java there is keytool. This supports a variety of keystores (depending on what has been configured in its java.security file), including

  • Java Key Stores .jks. This keystore can only be processed by java
  • Java Cryptography Extension KeyStore – a stronger version of .jks. This keystore can only be processed by java
  • pkcs12 .p12 and .pfx. These keystores can be used by Java, C and other languages, with the right API. For example Curl uses a C api to access the keystores.
  • pkcs11 – keystores on smart devices
  • nss – netscape security.

It does not support cms format keystores used by GSKIT.

Openssl

This is open source and is active, keeping up with the trends in security. Openssl deals with the building blocks; private keys, certificates etc but does not handle keystores very well. You can create manage certificates, but you may need other tools to put them into keystores.

opensc for managing smart keys and other pkcs#11 devices

(Opensc is open Smart Card.) If you have a smart key, or external keystore on an HSM, there may be hardware specific libraries for accessing the keystore, or the open source code may be supported. This can provide drivers for other tools, such as the java tools. You can also use pkcs11-tool to directly administer keys and certificates on an HSM device.

Other tools

  • pk12util from Netscape is used to mange keys in the NSS keystore databases
  • certutil from Netscape is used to manage keys and certificate in the NSS database.
  • Firefox and Chrome browsers can be used to update the NSS keystore used by the browsers.

What tool would I use when?

Creating the private key

  • When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool -keypairgen
  • To generate an elliptic curve, use openssl ecparam
  • To generate an RSA, use openssl genpkey
  • Use keytool keypairgen to generate an elliptic or RSA key
  • runmqakm and strmqikm(ikeyman) do not seem to support elliptic curves, but support RSA keys as part of a certificate request (see below)

Create a certificate request

The certificate request takes (or creates) the public key, creates a certificate with the DN ( eg CN=Colin,C=GB,O=SSS and creates the certificate request file.

  • openssl req takes a file from the keypairgen
  • keytool can create a certificate request, I could not see how to use the keypair gen private key as part of this
  • runmqakm and strmqikm (ikeyman) can create a request, but I do not think it supports all flavours of private key.

Sign a request

I expect most people will want a tool which you can run as a script

  • openssl ca – you can include the optional attributes, with this.
  • keytool – I could not see how to use this to sign a certificate. The documentation suggest using openssl x509.
  • runmqakm – you can use runmqakm -cert -sign .
  • strmqikm (ikeyman) I could not see how to sign a request using this, and it is a GUI.

Receive the signed certificate into the keystore

  • When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool –write-object
  • Use openssl pkcs12 to create a pkcs12 (.p12) keystore using the private key and signed certificate
  • Use keytool -importcert. This can support most keystore types, depending on the configuration in the java.security file.
  • runmqakm and strmqikm (ikeyman) import ( receive) the certificate and store it in the keystore.

Update the browser’s keystore

The browsers have an nss format store.

  • pk12util -i …imports a .p12 keystores into the nss “sql:” keystore. The sql: keystore is a Netscape internal format store. (you use certutil … to remove entries from the nss keystore).
  • You can use the browser’s facilities to import certificates and keys into the browsers (nss) keystore.

Receiving a public certificate into a keystore.

If you have a Certificate Authority or a self signed certificate you want to put into your trust store.

  • When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool –write-object
  • Openssl is not very good at adding new entries to an existing .p12 file
  • runmqakm -cert -add -file ca256.pem…
  • strmqikm, select Signer Certificates, then select Add
  • Use keytool -importkeystore.  To import a .p12 store, or a .pem file I have a shell script

ks=” -destkeystore mytrust.p12″
dest=”-deststoretype pkcs12 -deststorepass password”
src=”-srcstoretype PKCS12 -srcstorepass password”
keytool -importkeystore $ks $dest -srckeystore ca1024.p12 $src
or
keytool -import $ks $dest -file carsa1024.pem 

Which keystore should I used for which application?

All applications can use smart cards and external keystores through the standard key stores.

  • A queue manager on midrange uses cms format files, so you need to use the runmqakm or strmqikm similar files
  • A queue manager on z/OS can use the z/OS security manager (eg RACF), or the cms keystore files.
  • Firefox and Chrome browsers use the NSS format keystores.
  • Java clients can use a variety of keystores, .jks, .p12, smart cards
  • A C, .Net etc client use cms format keystores
  • Curl can use .pem files (from openssl) and .p12 files
  • A web server has trust stores, and keystores. You configure the supported formats in the java.security file. It can use jks, .p12 and smart cards.

Summary

It all looks a bit of a mess, and you need to know a lot to get your job done.

Debugging external smart cards and external pkcs11 keystores.

There is an open source package (opensc) which provides access to smart cards and external keystores. It provides some good tools for diagnosing problems.

There is a wiki with good information.

Opensc return codes are here, and the printable text is here

Monitor traffic to and from the device.

You can monitor the traffic to and from the device by using an intermediate “spy” module which displays the traffic.

In your configuration (for example a CCDT), where you specified the name of the module /usr/lib64/pkcs11/opensc-pkcs11.so, replace this with /usr/lib64/pkcs11/pkcs11-spy.so. Specify the environment variable

export PKCS11SPY=/usr/lib/x86_64-linux-gnu/pkcs11/opensc-pkcs11.so

The spy module is invoked, prints out the parameters, and then invokes the module specified in the environment variable.

The output is like

19: C_Login
2021-03-10 14:22:47.947
[in] hSession = 0x21fc030
[in] userType = CKU_USER
[in] pPin[ulPinLen] 00000000021fb2a0 / 8
00000000 5B C7 E7 BB E5 FC 6A BE […..j.
Returned: 160 CKR_PIN_INCORRECT

Detailed internal trace

You can specify the environment variable OPENSC_DEBUG to give a very detailed trace. The higher the number the more detailed the trac.

export OPENSC_DEBUG=9

and use unset OPENSC_DEBUG to reset it.

You can use OPENSC_CONF to specify a configuration file with more parameters, such as file name for the output.

The output from this trace (showing a logon with pin number 12345678) is like

0x7f96e2dca740 14:13:16.756 [opensc-pkcs11] framework-pkcs15.c:1494:pkcs15_login: pkcs15-login: userType 0x1, PIN length 8
0x7f96e2dca740 14:13:16.756 [opensc-pkcs11] pkcs15-pin.c:301:sc_pkcs15_verify_pin: called
….
0x7f96e2dca740 14:13:16.757 [opensc-pkcs11] reader-pcsc.c:283:pcsc_transmit: reader ‘Nitrokey Nitrokey HSM (DENK01051600000 ) 00 00’
0x7f96e2dca740 14:13:16.757 [opensc-pkcs11] reader-pcsc.c:284:pcsc_transmit:
Outgoing APDU (13 bytes):
00 20 00 81 08 31 32 33 34 35 36 37 38 . …12345678

GSKIT return codes

If you are using the MQ C Client interface, this uses GSKIT. There is documentation for the z/OS version, and the return codes are here.