Getting a Java program to use a TLS MQ channel, and use an external USB keystore.

This project started off trying to get a java program to use an HSM (external keystore on a USB). This was not well documented, so I have documented my path.

I had set up my keystore on the USB HSM device as described here. This post covers the MQ side of it.

MQ provides some JMS samples.

I could use JmsBrowser using local bindings with the script.

. /opt/mqm/java/bin/setjmsenv64
java -Djava.library.path=/opt/mqm/java/lib64 JmsBrowser -m QMA -d CP0000

To use a client channel (QMACLIENT) was easy, the program is configured to allow a channel name to be passed as a parameter using the -l option.

. /opt/mqm/java/bin/setjmsenv64
java -Djava.library.path=/opt/mqm/java/lib64 JmsBrowser -m QMA -d CP0000 -h localhost -p 1414 -l QMACLIENT

To use a TLS channel (QMAQCLIENTTLS) proved harder

. /opt/mqm/java/bin/setjmsenv64
java -Djava.library.path=/opt/mqm/java/lib64 JmsBrowser -m QMA -d CP0000 -h localhost -p 1414 -l QMAQCLIENTTLS

This gave me

CC=2;RC=2397;AMQ9641: Remote CipherSpec error for channel ‘QMAQCLIENTTLS’ to host ”. [3=QMAQCLIENTTLS]

It was hard to find where to specify the CipherSpec. The documentation said “Using the environment variables MQCHLLIB to specify the directory where the table is located, and MQCHLTAB to specify the file name of the table.” I tried this and got the same error.

I collected a trace using the Java option , but the trace showed the cipher spec was null.

I gave up with this program and used an enhanced version of the program. JMS programs can use JNDI as a file repository of configuration information and the program can extract the information. For example you can configure Connection Factories(cf) which are MQ connections, and Q objects – which are queues.

Use JNDI to store parameters

JMS programs can use a JNDI interface to access configuration parameters stored in an external file.
I used this article to get me started.

My JMSAdmin.config had


I used the following commands to define a connection factory called CF1, using the channel, QMACLIENT, and a queue reference called MYQUEUE pointing to queue CP0000.

mkdir JNDI-Directory
/opt/mqm/java/bin/JMSAdmin -v -cfg JMSAdmin.config
define cf(CF1) transport(client) channel(QMACLIENT)
define q(MYQUEUE) queue(CP0000)

I used a bash script to run the program, using the JmsJndiBrowser sample (JmsBrowser with JNDI support).

. /opt/mqm/java/bin/setjmsenv64
java -Djava.library.path=/opt/mqm/java/lib64 JmsJndiBrowser -i file:///home/colinpaice/mq/JNDI-Directory -c CF1 -d MYQUEUE

This failed with

JMSCMQ0001: IBM MQ call failed with compcode ‘2’ (‘MQCC_FAILED’) reason ‘2400’ (‘MQRC_UNSUPPORTED_CIPHER_SUITE’).

Which was a surprise to me. I looked in the trace, and there was nothing obviously wrong. The channel was not configured for TLS.

I used the following command in the JMS Admin tool, to specify a TLS channel and configure the cipher suite


Note. If I displayed cf(CF1) it gave me SSLCIPHERSUITE(*TLS12).

This gave me unable to find valid certification path to requested target, which was good progress because it meant the cipher spec had been accepted.

I added in the Java parameters to identify the key store and trust store

. /opt/mqm/java/bin/setjmsenv64

ks=”$kss $ksp $kst”
ts=”$tss $tsp $tst”

java $ks $ts -Djava.library.path=/opt/mqm/java/lib64 JmsJndiBrowser -i file:///home/colinpaice/mq/JNDI-Directory -c CF1 -d MYQUEUE

and it worked successfully.

Use the pkcs11 HSM external keystore.

I changed the file to specify the external keystore (with format pkcs11) on a USB device rather than a file on disk (format pkcs12).


but this gave

find / pkcs11 not found pkcs11 KeyStore not available

You have to tell Java where to find the pkcs11 code. This is configured in the file. You can find it using find $JAVA_HOME/ -name . This gave me the file name /usr/lib/jvm/java-8-oracle/jre/lib/security/

It had the list of security providers,

and is missing an entry for pkcs11.

You can edit this file, or override it. To override it, create a file like with

security.provider.10=SunPKCS11 /home/colinpaice/mq/nitrokey.cfg


  • .10 is the next in sequence after the .9 in the file
  • SunPKCS11 says use the Sun module.
  • /home/colinpaice/mq/nitrokey.cfg use this configuration file for the pkcs11 device.

Tell Java to use this override file, by using the Java option

The configuration file name for the pkcs11 is /home/colinpaice/mq/nitrokey.cfg with content

name = nitrokey
library = /usr/lib/x86_64-linux-gnu/

With these parameters the output from the JmsJndiBrowser command was

Initial context found!
Browse starts
No more messages

Success – and it only took me half a day! The hardest part was known what to specify for the security.provider…… parameter.

Using the runmqakm commands and an HSM (but not strmqikm).

I tried to use strmqikm but it gave an exception.

You can use some of the runmqakm commands you know and love, to access a certificate with an HSM. For example

The command to list the database available to the runmqakm command,

runmqakm -keydb -list -crypto /usr/lib/x86_64-linux-gnu/


/usr/lib/x86_64-linux-gnu/ : UserPIN (mytoken)

You can then use the token label UserPIN (mytoken) and password to use the key store, for example

runmqakm -cert -list all -crypto /usr/lib/x86_64-linux-gnu/
-tokenlabel “UserPIN (mytoken)” -pw 12345678


Certificates found
* default, - personal, ! trusted, # secret key
-	my_key3


runmqakm -cert -details -crypto /usr/lib/x86_64-linux-gnu/
-tokenlabel “UserPIN (mytoken)” -pw 12345678
-label my_key3

displays the details of the certificate with label my_key3.

If the -tokenlabel was wrong or the -pw was wrong, I got the unhelpful messages

  • CTGSK3026W The key file “pkcs11” does not exist or cannot be read.
  • CTGSK2137W The label does not exist on the PKCS#11 device.

Create your certificate request

The following command create a new RSA private-public key pair and a PKCS10 certificate request. The documentation for runmqakm says it supports RSA. If you want to use an Elliptic Curve you will need to use an alternative method, for example openssl.

runmqakm -certreq -create -crypto /usr/lib/x86_64-linux-gnu/
-tokenlabel “UserPIN (mytoken)” -pw 12345678
-dn “cn=colin,o=SSS” -file runmq.csr -label runmqlab -size 1024

Sign it

openssl ca -config openssl-ca-user.cnf -policy signing_policy -md sha256 -cert carsa1024.pem -keyfile carsa1024.key.pem -out runmq.pem -in runmq.csr

Store it back into the HSM keystore

I could not get the runmqakm command to receive the signed certificate and store it into the HSM keystore.

runmqakm -cert -receive -crypto /usr/lib/x86_64-linux-gnu/ -tokenlabel “UserPIN (mytoken)” -file runmq.pem -pw 12345678

It failed with

CTGSK3034W The certificate request created for the certificate is not in the key database.

I could use

openssl x509 -inform pem -outform der -in runmq.pem -out runmq.der
pkcs11-tool –write-object runmq.der –type cert –label “runmqlab” -l –pin 12345678

The openssl command converts the file from .pem format, to .der format as .der format is required by pkcs11-tool.

Using strmqikm – the theory

If you want to use the strmqikm GUI, you have to configure the file. For example edit /opt/mqm/java/jre64/jre/lib/security/ and add the next security.provider in the list. /home/colinpaice/mq/nitrokey.cfg

Where /home/colinpaice/mq/nitrokey.cfg is the configuration file, with

name = nitrokey
library = /usr/lib/x86_64-linux-gnu/

You can then use Ctrl+O, which brings up a pop up with “Key database type”. In this list should be PKCS11Config, if not check your file. Select this, leave File Name and Location empty, and click “OK”. It pops up “Open Cryptographic Token” with the “Token Label” value taken from the configuration file name = nitrokey. This is strange as the runmqakm command uses a TokenLabel of “UserPIN (mytoken)”.

In practice…

I then got an exception java.lang.RuntimeException: findSigner(): Failure while executing cobj.getX509Certificate(certFactory, session), and strmqikm ended.

Using a hardware security module USB as a keystore for a browser.

Background to certificates and keystores

When using TLS(SSL) you have two keystores

  • A keystore for holding the public part and private key of your certificate
  • A trust store which holds the public keys of certificate sent to you which you need to authenticate.

Your certificate has two parts

  • The private key which contains information needed to encrypt information you send. This needs to be kept private.
  • The public part,which has information that is needed to decrypt information you have encrypted, along with information such as your Distinguished Dame (DN) such as CN=ColinPaice C=GB,O=StromnessSoftware

The process of creating a signed certificate is

  • Create a private key and public key. This can be done using an external device Hardware Security Module (HSM), such as the Nitrogen HSM USB, or software, for example using OPENSSL. This produces a private key file, and a certificate request file containing the public information.
  • Send the public information to your certificate authority which signs it, and returns it
  • Import the signed public certificate into your keystore.

Creating a certificate using an HSM as the key repository

I used openssl to process my certificates, I’ve discussed the openssl setup here.

I use a bash script because it is easy to parametrize, and makes it easy to rerun until it works. I’ll give the script, then explain what it does

  • enddate=”-enddate 20240130164600Z”
  • name=”hw”
  • rm $name.key.pem
  • rm $name.csr
  • rm $name.pem
  • ca=”carsa1024″
  • pkcs11-tool –keypairgen –key-type rsa:2048 –login –pin 648219 –label “my_key3”
  • OPENSSL_CONF=eccert.config openssl req -new -engine pkcs11 -keyform engine -key label_my_key3 -out $name.csr -sha256 -subj “/C=GB/O=HW/CN=colinpaice” -nodes
  • openssl ca -config openssl-ca-user.cnf -policy signing_policy -md sha256 -cert $ca.pem -keyfile $ca.key.pem -out $name.pem -in $name.csr $enddate
  • openssl x509 -inform pem -outform der -in $name.pem -out $name.der
  • pkcs11-tool –write-object $name.der –type cert –label “my_key3” -l –pin 648219

What does the script do ?

enddate=”-enddate 20240130164600Z”

This sets the end date for the certificate – the end date is set when it is signed.


This is used within the script to ensure the correct files are being used.

Remove old intermediate files
  • rm $name.key.pem
  • rm $name.csr
  • rm $name.pem

Define the name of the CA files to use at signing time. The $ca.pem and $ca.key.pem are both needed.

pkcs11-tool –keypairgen –key-type rsa:2048 –login –pin 648219 –label “my_key3”
  • pkcs11-tool use this tool
  • –keypairgen to create a key pair (private and public pair)
  • –key-type rsa:2048 use this key type and key length
  • –login –pin 648219 login with the pin number
  • –label “my_key3” use this label to identify the key
OPENSSL_CONF=eccert.config openssl req -new -engine pkcs11 -keyform engine -key label_my_key3 -out $name.csr -sha256 -subj “/C=GB/O=HW/CN=colinpaice”
  • OPENSSL_CONF=eccert.config this sets up the openssl config file. Having -config eccert.config does not work. See here.
  • openssl
  • req this is to create a certificate requests – create a .csr.
  • -new it is a new request
  • -engine pkcs11 use the named engine, pkcs11, defined to the system
  • -keyform engine this says use the engine (HSM). Other choices are der and pem
  • -key label_my_key3 go to the engine and look for the my_key3 label
  • -out $name.csr create this request file with this name.
  • -sha256 using this signature
  • -subj “/C=GB/O=HW/CN=colinpaice” the name to go in the certificate. It uses colinpaice as the certificate will be used to authenticate with the mq web server, and this is the userid the mq web server should use.

Send the .csr file to the CA for signing (which is the same machine in my case).

openssl ca -config openssl-ca-user.cnf -policy signing_policy -md sha256 -cert $ca.pem -keyfile $ca.key.pem -out $name.pem -in $name.csr $enddate
  • openssl ca Use this command to sign the certificate
  • -config openssl-ca-user.cnf use this configuration file
  • -policy signing_policy use this policy within the config file
  • -md sha256 use this for the message digest
  • -cert $ca.pem use the public certificate of the CA
  • -keyfile $ca.key.pem use this private key of the CA to encrypt information about the csr request’s certificate
  • -out $name.pem whee to store the output
  • -in $name.csr the input .csr request
  • $enddate specify the certificate expiry date – set at the top of the script

Send the signed certificate back to the requester.b

openssl x509 -inform pem -outform der -in $name.pem -out $name.der

The pkcs11-tool uses .der files so convert the .pem file to .der format

  • openssl x509
  • -inform pem input format
  • -outform der output format
  • -in $name.pem hw.pem
  • -out $name.der hw.der
pkcs11-tool –write-object $name.der –type cert –label “my_key3” -l –pin 648219

Read the signed certificate and write it to the HSM

  • pkcs11-tool
  • –write-object $name.der write onto the HSM the file hw.der coverted above
  • –type cert import type (cert|pubkey|privkey)
  • –label “my_key3” use this name
  • -l –pin 648219 and logon with this pin number

Define the HSM to Chrome browser

Stop the browser because you need to update the keystore.
The command was issued in the home directory, because key store is in the home directory/.pki .

modutil -dbdir sql:.pki/nssdb/ -add “my_HSM” -libfile

  • modutil use this command
  • -dbdir sql:.pki/nssdb/ to up date this keystore (in ~)
  • -add “my_HSM” give it this name
  • -libfile and use this file to communicate to it

Display the contents of the browser’s keystore

modutil -dbdir sql:.pki/nssdb/ -list

This gave me

Listing of PKCS #11 Modules
 NSS Internal PKCS #11 Module
 Mozilla Root Certs
 library name: /usr/lib/x86_64-linux-gnu/nss/
 library name:
    uri: pkcs11:library-manufacturer=OpenSC%20Project;library-description=OpenSC%20smartcard%20framework;library-version=0.17
  slots: 1 slot attached
 status: loaded
 slot: Nitrokey Nitrokey HSM (DENK01051600000         ) 00 00
 token: UserPIN (SmartCard-HSM)
   uri: pkcs11:token=UserPIN%20(SmartCard-HSM);;serial=DENK0105160;model=PKCS%2315%20emulated 

Restart the browser.

Use an URL which needs a certificate for authentication.

The browser prompts for the pin number (twice), and displays the list of valid certificate CNs. Pick one. When I connected to the mqweb server, I had 3 certificates displayed. I had to remember which one I wanted from the Issuer’s CN and serial number. For example

Select a certificate

(Having a CA just for HSM keys, such as SSSCAHSM would make it more obvious.)

Using openssl with an HSM keystore, and opensc pkcs11 engines.

A Hardware Security Module (HSM) is an external device, such as USB plugin which can securely store keystores, and do other encrpyption work. I used a Nitrokey which uses open source software.

Create the key on the HSM

pkcs11-tool –keypairgen –key-type EC:prime256v1 –login –pin 12345678 –label “my_key3”

Create the certificate request using openssl

To use the opensc pkcs11 driver for an HSM you need to pass parameters to the driver.

A typical openssl command to create a certificate request, using a pre existing private key, is

OPENSSL_CONF=hw.config openssl req -new -x509 -engine pkcs11 -keyform engine -key slot_0-label_my_key3 -sha256 -out hw.pem -subj “/C=CB/O=HW/CN=HW”

The fields for the HSM device are:

  • -engine pkcs11 . This tells openssl which exernal device to use. Use the command openssl engine -vvv -tt pkcs11 to display information about the pkcs11 engine
  • -keyform engine it needs to be “engine” to use the HSM.
  • -key xxxx where xxxx can be in the format
    • n:m where n is the slot number (“where the HSM device is plugged into – the first device is usually 0) and m is the id of the certificate, public key, or private key within the device
    • slot_n-id_m where where n is the slot number (“where the HSM device is plugged into – usually 0) and m is the id within the device
    • id_m m is the id within the device (slot_n defaults to slot_0)
    • slot_n-label_name where where n is the slot number (“where the hsm device is plugged into – usually 0) and name is the keyname within the HSM
    • label_name where name is the keyname within the HSM (and slot_n defaults to slot_0)

Note the id and labels are strings, so the id value 0100 is different from the value 100.

The command pkcs11-tool -O gave

Using slot 0 with a present token (0x0)
Public Key Object; RSA 2048 bits
  label:      Private Key
  ID:         0200
  Usage:      encrypt, verify, wrap
Public Key Object; RSA 2048 bits
  label:      my_key
  ID:         0100
  Usage:      encrypt, verify, wrap
Public Key Object; RSA 2048 bits
  label:      my_key2
  ID:         0101
  Usage:      encrypt, verify, wrap
Public Key Object; RSA 2048 bits
  label:      my_key3
  ID:         0fea4632d386a7f2469eec44daeafa84a1dbd8e2
  Usage:      encrypt, verify, wrap

So I could use for -key

  • 0:0100
  • id_0100
  • slot_0-id_100
  • slot_0-label_my_key
  • label_my_key

The code for this is here.

The contents of the openssl config file for HSM devices

I’ve displayed the bits you need for the “engine” or HSM processing.

openssl_conf = openssl_def

engines = engine_section

pkcs11 = pkcs11_section
engine_id = pkcs11
dynamic_path = /usr/lib/x86_64-linux-gnu/engines-1.1/
MODULE_PATH    = /usr/lib/x86_64-linux-gnu/
PIN = 648219
# init = 0

The documentation for the config file is .

The use of an HSM is driven from an “engine”.

The relevant statements in the config file are

  • openssl_conf openssl searches the config file for this entry by default. It describes which other sections should always be processed. If you use openssl req …, openssl will also process the section [req]. My file says use the section called openssl_def.
  • [openssl_def] this contains the names of sections to be processed. In this case it says process the engine_section.
  • [engine_section] this lists all of the engine sections to be processed. For a pkcs11 engine process I have called the section pkcs11_section.
  • [pkcs11_section] this lists the individual entries for the pkcs11 engine.
    • The engine_id is optional
    • The dynamic path is the code for the HSM. If specified, this must come first. For the default see openssl version -e or openssl version -a.
    • You can pass parameters through the HSM. Use the openssl engine -vvv -tt pkcs11 to list them. Note they are in upper case.
    • The MODULE_PATH in upper case is passed through to the device.
    • The PIN in upper case is passed through to the HSM. Without this, you are prompted for the pin
    • Using VERBOSE = EMPTY gives more information from the HSM device.

Problems using the config file.

The following command worked

OPENSSL_CONF=eccert.config openssl req -new -x509 -engine pkcs11 -keyform engine -key id_0101 -sha256 -out hw.pem -subj “/C=CB/O=HW/CN=HW”

The following command failed

openssl req -config eccert.config -new -x509 -engine pkcs11 -keyform engine -key id_0101 -sha256 -out hw.pem -subj “/C=CB/O=HW/CN=HW”

It looks like the config file is being processed twice, so you have to use the OPENSSL_CONF variable rather than the -config option.

The error messages are

  • engine “pkcs11” set.
  • Using configuration from eccert.config
  • Error configuring OpenSSL modules
  • engine routines:engine_list_add:conflicting engine id:../crypto/engine/eng_list.c :63. I think this is reporting it has already been processed and exists in the list of engines.
  • engine routines:ENGINE_add:internal list error:../crypto/engine/eng_list.c :223:
  • engine routines:dynamic_load:conflicting engine id:../crypto/engine/eng_dyn.c :502:
  • engine routines:int_engine_configure:engine configuration error:../crypto/engine/eng_cnf.c :141: section=pkcs11_section, name=dynamic_path, value=/usr/lib/x86_64-linux-gnu/engines-1.1/ These are the parts of my configuration file.
  • configuration file routines:module_run:module initialization error:../crypto/conf/conf_mod.c: 174:module=engines, value=engine_section2, retcode=-1

Some other error messages

With no MODULE_PATH in the configuration file

  • engine “pkcs11” set.
  • No private keys found.
  • PKCS11_get_private_key returned NULL
  • cannot load Private Key from engine
  • pkcs11 engine:ctx_load_privkey:object not found:eng_back.c:876:
  • engine routines:ENGINE_load_private_key:failed loading private key:../crypto/engine/eng_pkey.c:78:
  • unable to load Private Key

Initial setup for using a keystore on a HSM USB stick.

You can use a keystore on disk, but this inherently insecure, as people with administrator access to the machine, can copy the keystore. Using an external device (such as a USB Hardware Security Module) as a keystore, is more secure as you need physical access to the machine to physically access the keystore. If you have 3 failed attempts to access the keystore using a PIN code, the device locks up.

I found this document a good high level introduction to smart keys.

This post describes the initial set up for using the Hardware Security Module from Nitrokey for securely storing my digital certificates. It comes as a USB device. I chose it because it cost under 80 euros. There are other suppliers, such as yubico , and other suppliers but either they did not supply a price, or it was “call us and to discuss it”.

I found the Instructions that came with it via here, and a user blog very useful.

The Nitrokey HSM is open sourced, and uses open source facilities.

Software needed to use the key.

My machine is Linux Ubuntu 18.04.
You need software installed to configure it.

sudo apt install opensc pcscd pcsc-tools

To be able to use openssl you need an “engine” interface.

sudo apt install libengine-pkcs11-openssl

Once install you need to start it

sudo systemctl start pcscd
sudo systemctl status pcscd


● pcscd.service – PC/SC Smart Card Daemon
Loaded: loaded (/lib/systemd/system/pcscd.service; indirect; vendor preset: enabled)
Active: active (running) since Tue 2021-03-02 08:49:27 GMT; 2s ago

Display it (note it is two hypens)

opensc-tool ‐‐list‐readers

Gave me

#Detected readers (pcsc)
Nr. Card Features Name
0   Yes           Nitrokey Nitrokey HSM (DENK01051600000 ) 00 00

This shows the card is in

  • slot number 0. You may need this number when configuring keys, for example in openssl. This number is like a USB port number.
  • it is a physical card
  • it has no features listed
  • it comes from NitroKey and is a Hardware Security Module with the given serial number.

Set up

The device has

  • a device pin (SO-PIN) which is needed for administration, such as reinitialising the device or setting a user pin. This is 16 hex characters
  • a user pin to allow users access to modify keys. The user pin is a 6-15 digit string.

You need to consider how you use your device. You can have it self contained, and the private information is private to the device. This may be acceptable for a test device, but not in production, where you want to securely backup the keystore, and securely shared the key store between different machines. This can be done using Device Key Encryption Key (DKEK). The DKEK key is a 256-Bit AES key.

You can configure this so that you need more than one person to be able to enable a new device with this DKEK. You can configure n out of m people are needed. This is described here under Using key backup and restore.

You can use sc-hsm-tool – smart card utility for SmartCard-HSM, to

  • Initialize token, removing all existing keys, certificates and files.
  • Create a DKEK share encrypted under a password and save it to the file given as parameter.
  • Read and decrypt DKEK share and import into SmartCard-HSM
  • Define device pin for initialization
  • Force removal of existing key, description and certificate.
  • Define the token label to be used in –initialize.
  • Backup a private key to an encrypted external file. (Using the DKEK).
  • Restore a private key from an external encrypted file into the device, using the DKEK

You need to initialise the device see here.

I used

sc-hsm-tool –initialize –so-pin 3537363231383830 –pin 648219 –dkek-shares 1 –label mytoken
sc-hsm-tool –create-dkek-share dkek-share-1.pbe
sc-hsm-tool –import-dkek-share dkek-share-1.pbe

The command pkcs11-tool -L gave

Available slots:
Slot 0 (0x0): Nitrokey Nitrokey HSM (DENK01051600000         ) 00 00
  token label        : UserPIN (mytoken)
  token manufacturer :
  token model        : PKCS#15 emulated
  token flags        : login required, rng, token initialized, PIN initialized
  hardware version   : 24.13
  firmware version   : 3.4
  serial num         : DENK0105160
  pin min/max        : 6/15

The token label : UserPIN (mytoken) has the name I entered in the –label option above. When using MQ and GSKIT with this device,I needed to identify device with “UserPIN (mytoken)” not just “mytoken”.

You can create a private key using

pkcs11-tool –keypairgen –key-type rsa:2048 –id 10 –label “my_key”

Using slot 0 with a present token (0x0)
Key pair generated:
Private Key Object; RSA
label: Private Key
ID: 10
Usage: decrypt, sign, unwrap
Public Key Object; RSA 2048 bits
label: Private Key
ID: 10
Usage: encrypt, verify, wrap

You can omit the -id, and it will generate a (long) id for you. You can list the objects (in-use slots) in the device

pkcs11-tool -O

You can delete the one we just created

pkcs11-tool -l –pin 648219 –delete-object –type privkey –id 10

You can delete -type with privkey, pubkey and cert