pkcs11? pkcs12? .cms? .jks? .p12? .nss? which type of keystore should I use?

There are at least four types of keystore, and after an email exchange with someone, it became clear that the different types of keystore and how to use them, are not widely understood. It is one of those topics, that once you are an expert, this topic is obvious still difficult to understand. When you have little knowledge and just want to get a job done, it can be very confusing.

I’ve learned a lot from writing this post

• There is no one tool to manage all your keys and certificate
• You may need one keystore for your queue manager, another keystore for your C clients, another keystore for your Java clients, a keystore for a web browser and something else for curl!
• You might need to use 3 tools to manage your keystores!

This post has sections on

• What’s the difference between a trust store and a key store – when they are both called keystores?
• What does a store contain?
• What format is the data? plain text, binary, 64 bit encoded.
• What are the keystore types?
• When can different keystore types be used?
• Do these keystores share information? – short answer no.
• What can you use to administer the keystores.
• What tool would I use ,when?
• Which keystore for which application?

This post started off as a few lines, then I though I had better explain the pre-req knowledge, and then the pre-req knowledge to the the pre-req knowlegde. I hope it provides a clear story.

What’s the difference between a trust store and a key store – when they are both called key stores?

You need a store to contain the certificates needed to check any certificates sent to the application. These are usually certificate authority certificates, but can also be self signed certificates. This store is known as a trust store – it is used to check that a certificate is trusted. None of this data is confidential, most of the information is in the public domain. One trust store could be shared by all applications, which makes the maintenance of it much easier than a trust store for each application/server.

You need a store to keep the private key used for encryption/decryption, this is known as the key store. This needs to be kept confidential. The keystore may have just one private key. You can use smart keys, or external keystores such as special USB devices.

Some products use just one store with the trust store and the private certificates combined. They also call this the keystore. If you want to isolate the keystores, you need multiple stores. This, in turn, means that if you update one trust certificate, you have multiple stores to update.

What does a store contain?

A keystore can contain

• private information – such as you secret key which you use for encryption and decryption
• public information, such as your Distinguished Name (CN=colin,O=SSS), and they key needed to encrypt data for me
• trust data, if you send me a public key – can I validate it has not been tampered with?

What format is this data in?

This data is typically in one of three formats

• Plain text
• Binary
• Binary portable (64 bit encoding)

Plain text

If you print a certificate or key, it comes out like

Certificate:
  Data:
    Version: 3 (0x2)
      Serial Number: 379 (0x17b)
      Signature Algorithm: ecdsa-with-SHA256
      Issuer: C=GB, O=SSS, OU=CA, CN=SSCA256
      Validity
        Not Before: Feb  9 09:35:07 2021 GMT
        Not After : Apr  9 17:40:01 2022 GMT
        Subject: C=GB, O=cpwebuser, CN=ecec
        Subject Public Key Info:
          Public Key Algorithm: id-ecPublicKey
            Public-Key: (256 bit)
            pub:
               04:5d:15:78:83:e0:ab:af:78:69:2f:14:51:fe:29:
            ...
                ASN1 OID: prime256v1
                NIST CURVE: P-256
...

Binary

The data is encoded using Abstract Syntax Notation(asn). For example you get a string of fields like

• The following fields are a sequence
• of length 24
• it is a string format
• the first item has length 8
• the attribute type is 06 03 55 04 06 which means Country
• the attribute value is GB
• the second item is …

This would describe the C=GB in C=GB, O=SSS, OU=CA, CN=SSCA256.

Binary portable (base 64 encoding)

With the binary format, the data is a hex string, which is not very portable. For example if you FTP it to a remote site, you may get data conversion, and new lines changed. Often the data is converted to an intermediate form called base 64 encoding. Every 3 hex characters are converted to 4 “printable” characters A-Z,a-z,0-9 and some special characters.

The file looks like

-----BEGIN CERTIFICATE-----
MIICDjCCAbSgAwIBAgICAXswCgYIKoZIzj0EAwIwOjELMAkGA1UEBhMCR0IxDDAK
BgNVBAoMA1NTUzELMAkGA1UECwwCQ0ExEDAOBgNVBAMMB1NTQ0EyNTYwHhcNMjEw
...
hEzE3hSzUvbfLUKnua64AiEAnyKhrmBjXCVQ834VfjhNHYqjaN1PzXBE8Cv3jlWJ
9bA=
-----END CERTIFICATE-----

What are the keystore types?

When Secure Sockets Layer was being developed each major player came up with their own format for storing the certificate and key information.

• Java had files with type .jks (java key store) for example keystore.jks
• IBM had files with type cms (certificate management system) with file types like zzserver.crl, zzserver.kdb, zzserver.rdb, zzserver.sth. It also uses intermediate files such as .arm, which has base 64 encoding of a certifcate, as shown above.
• Netscape had files in a NSS database, for example files in the directory /home/colinpaice/.pki/nssdb, are files cert9.db and key4.db. key4 is the keystore database (storing keys), and cert9 is the trust store database (storing certificates).
• OpenSSL developed .p12 files which can contain certificates and keys. It also has .pem (base 64 encoded) and .der (binary) files for individual certificate, private key, and public key files .
• Windows has .pfx files.
• Smart cards where the keystore is on a special USB type device or other external Hardware Security Module, each have their own format keystore.

Standards were developed to work with keystores

• pkcs11 -the programming interface to create and manipulate cryptographic tokens (a token where the secret is a cryptographic key).
• pkcs12 – defines an archive file format for storing many cryptography objects as a single file. It is commonly used to bundle a private key with its X.509 certificate or to bundle all the members of a chain of trust. Openssl uses one certificate per file, you can have multiple certificates in a pks12 file is you use keytool.

When can different keystore types be used?

With C programs on mid range and z/OS, IBM products use GSKIT from IBM. On z/OS you can store them in the z/OS security manager (for example RACF) or on the Hardware Security Module in the processor.

Java program can use most types of keystore. You may need to configure the java.security configuration file with an entry like

• security.provider.X=com.ibm.security.cmskeystore.CMSProvider
• security.provider.Y=com.ibm.crypto.pkcs11impl.provider.IBMPKCS11Impl /home/colinpaice/mq/nitrokey.cfg

The first line includes support for cms files, the second includes support for a pkcs11 external keystore on an HSM (including its configuration file).

For Java programs, you configure the keystore using start-up options. The following example defines the keystore as an external USB HSM keystore, and the trust store is a openssl .p12 store.

-Djavax.net.ssl.keyStore=NONE
-Djavax.net.ssl.keyStoreType=pkcs11
-Djavax.net.ssl.keyStorePassword=12345678

-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/mytrust.p12
-Djavax.net.ssl.trustStorePassword=password
-Djavax.net.ssl.trustStoreType=pkcs12

These definitions along with the java.security, and its override, show the keystore is a pkcs11, and there is an entry in the java.security (above) pointing to the configuration file /home/colinpaice/mq/nitrokey.cfg for the pkcs11 definition. You can override or extend the options in the java.security file using configuration using

-Djava.security.properties=/home/colinpaice/mq/colin.java.properties

Do these keystores share information?

No, if you have a keystore (or a USB HSM) it shares no information with any other keystore. For example you could unplug the USB keystore, take it to another machine and plug it in, and the certificate etc will be available. You could send a .jks file, to yet another machine, and it could be used.

What can you use to administer the keystore.

IBM provided

You can use GSKCapiCmd_64 from IBM GSKIT to manage certificates and keys.

MQ has runmqakm as a command line tool which invokes GSKCapiCMD_64 under the covers.

You can also use the MQ command strmqikm to have a GUI to manage your keys and certificates. strmqikm just invokes the GSKIT ikeyman program.

These solutions feel a little dated, as I could not find if they support modern certificates capabilities; OCSP, and elliptic curves. They had a few bugs as well.

I’ve used openssl to create the “modern” private key and certificates, then imported them into the keystores using the IBM tools.

Oracle provided

With Java there is keytool. This supports a variety of keystores (depending on what has been configured in its java.security file), including

• Java Key Stores .jks. This keystore can only be processed by java
• Java Cryptography Extension KeyStore – a stronger version of .jks. This keystore can only be processed by java
• pkcs12 .p12 and .pfx. These keystores can be used by Java, C and other languages, with the right API. For example Curl uses a C api to access the keystores.
• pkcs11 – keystores on smart devices
• nss – netscape security.

It does not support cms format keystores used by GSKIT.

Openssl

This is open source and is active, keeping up with the trends in security. Openssl deals with the building blocks; private keys, certificates etc but does not handle keystores very well. You can create manage certificates, but you may need other tools to put them into keystores.

opensc for managing smart keys and other pkcs#11 devices

(Opensc is open Smart Card.) If you have a smart key, or external keystore on an HSM, there may be hardware specific libraries for accessing the keystore, or the open source code may be supported. This can provide drivers for other tools, such as the java tools. You can also use pkcs11-tool to directly administer keys and certificates on an HSM device.

Other tools

• pk12util from Netscape is used to mange keys in the NSS keystore databases
• certutil from Netscape is used to manage keys and certificate in the NSS database.
• Firefox and Chrome browsers can be used to update the NSS keystore used by the browsers.

What tool would I use when?

Creating the private key

• When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool -keypairgen
• To generate an elliptic curve, use openssl ecparam
• To generate an RSA, use openssl genpkey
• Use keytool keypairgen to generate an elliptic or RSA key
• runmqakm and strmqikm(ikeyman) do not seem to support elliptic curves, but support RSA keys as part of a certificate request (see below)

Create a certificate request

The certificate request takes (or creates) the public key, creates a certificate with the DN ( eg CN=Colin,C=GB,O=SSS and creates the certificate request file.

• openssl req takes a file from the keypairgen
• keytool can create a certificate request, I could not see how to use the keypair gen private key as part of this
• runmqakm and strmqikm (ikeyman) can create a request, but I do not think it supports all flavours of private key.

Sign a request

I expect most people will want a tool which you can run as a script

• openssl ca – you can include the optional attributes, with this.
• keytool – I could not see how to use this to sign a certificate. The documentation suggest using openssl x509.
• runmqakm – you can use runmqakm -cert -sign .
• strmqikm (ikeyman) I could not see how to sign a request using this, and it is a GUI.

Receive the signed certificate into the keystore

• When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool –write-object
• Use openssl pkcs12 to create a pkcs12 (.p12) keystore using the private key and signed certificate
• Use keytool -importcert. This can support most keystore types, depending on the configuration in the java.security file.
• runmqakm and strmqikm (ikeyman) import ( receive) the certificate and store it in the keystore.

Update the browser’s keystore

The browsers have an nss format store.

• pk12util -i …imports a .p12 keystores into the nss “sql:” keystore. The sql: keystore is a Netscape internal format store. (you use certutil … to remove entries from the nss keystore).
• You can use the browser’s facilities to import certificates and keys into the browsers (nss) keystore.

Receiving a public certificate into a keystore.

If you have a Certificate Authority or a self signed certificate you want to put into your trust store.

• When using a smart device, for example an external keystore on an USB, use the device driver or pkcs11-tool –write-object
• Openssl is not very good at adding new entries to an existing .p12 file
• runmqakm -cert -add -file ca256.pem…
• strmqikm, select Signer Certificates, then select Add
• Use keytool -importkeystore.  To import a .p12 store, or a .pem file I have a shell script

ks=” -destkeystore mytrust.p12″
dest=”-deststoretype pkcs12 -deststorepass password”
src=”-srcstoretype PKCS12 -srcstorepass password”
keytool -importkeystore $ks $dest -srckeystore ca1024.p12 $src
or
keytool -import $ks $dest -file carsa1024.pem 

Which keystore should I used for which application?

All applications can use smart cards and external keystores through the standard key stores.

• A queue manager on midrange uses cms format files, so you need to use the runmqakm or strmqikm similar files
• A queue manager on z/OS can use the z/OS security manager (eg RACF), or the cms keystore files.
• Firefox and Chrome browsers use the NSS format keystores.
• Java clients can use a variety of keystores, .jks, .p12, smart cards
• A C, .Net etc client use cms format keystores
• Curl can use .pem files (from openssl) and .p12 files
• A web server has trust stores, and keystores. You configure the supported formats in the java.security file. It can use jks, .p12 and smart cards.

Summary

It all looks a bit of a mess, and you need to know a lot to get your job done.