Whoops my certificate has expired – what do I need to do?

The amount of work depends on what has expired.

  • If the root CA has expired then you need to reissue all certificate which include the root CA, and update trust stores
  • If the enterprise CA has expired you need to renew all certificates signed by the enterprise CA, and update trust stores with the new certificate
  • If your personal certificate has expired, you need to renew that certificate.

The amount of work also depends on the size of your enterprise.

I’ll list the work need if the certificate is going to expire next week – as this is more work than if it has already expired

If your personal certificate is going to expire

You need to

  • renew it, create a new private key and certificate request
  • or extend the date of the existing certificate  – by re-signing it
  • update the keystores which include it.  This could be those used by java programs ( .p12, or .jks) as well as any browser keystore (.nssdb)

To renew your personal certificate

You can just recreate it.  If you are considering to use a stronger algorithms, such as Elliptic Curves, bear in mind that the servers need to be able to support what you specify.

Depending on how your openssl has been configured, if you have unique_subject = yes you may need to use openssl revoke… to remove the old personal certificate from the configuration.   This is to ensure there is only one certificate with the same distinguished name, and online checking of certificate validity (Certificate Revocation Lists and Online Status Certificate Protocol) can be used.  If you have unique_subject = no, you can recreate the certificate without revoking it.

  • openssl genpkey  to generate a new private key
  • openssl req to create a request

You do not need to recreate it – you can just sign it again.

You then need to do normal process of update the users of the certificate.

  • openssl ca to sign the request
  • openssl pkcs12 -export to create the .p12 file
  • You may want to deploy it only when the machine is attached to the local network.  If you download it over wireless, or email, these may be compromised, and the bad guys can just use the new copy.  Downloading it over a wired connection eliminates this risk.
  • certutil -D $sql -n $shortName to remove the certificate from the browser’s keystore
  • pk12util -i $shortName.p12 $sql -W password to add the certificate to the browser’s keystore
  • /opt/mqm/bin/runmqckm -cert -delete  to remove a certificate from a keystore (.jks)
  • /opt/mqm/bin/runmqckm -cert -import to add a certificate to a keystore(.jks)
  • restart the servers to pick up the new certificate

Rather than deleting the certificate from the keystore and adding the new one with the same short name,  you may want to make a copy of the old certificate from the keystore, or simply backup the whole keystore – as it should not change once frequently.  If there are problems, restore from the backup.

To renew your enterprise certificate

This requires a lot more work than updating a personal certificate.

Just like the personal certificate you can recreate it, or just sign it again.

You need to phase in the new certificate as it may take days or weeks to deploy it successfully.

The client’s trust store needs the CA certificate matching the CA used by the server’s certificate.

If you have two servers, and change servers to use the new CA certificate, the client’s trust store content will change over time.

  • The old CA certificate – before any work is done
  • Both old and new CA certificates during the migration period.  The server using the old CA will use the old CA in the client.   The server which has been migrated to use the new CA will use the new CA in the client.
  • Just the new CA certificate.  Once the migration has finished. The old certificate can be removed when all of the servers have been migrated, or the certificate has expired.

A key store and a trust store can have certificate with the same distinguished names, but with a different short name or alias name.  My enterprise is called SSS, and the CA for my enterprise is CASSS… .  My trust store has

  • the CA with C=GB,O=SSS,OU=CA,CN=CASSS and short name CASSS2016 and
  • the CA with C=GB,O=SSS,OU=CA,CN=CASSS and short name CASSS2020 and

When you maintain a keystore you use the short name, for my example CASSS2016 or CASSS2020.

During the TLS handshake, when the server’s certificate is sent to the client, it is checked against all of the trusted CA certificates in the keystore, so having two certificates with the same DN does not matter.

The steps to deploy your new CA are

  • Get the new CA.  Either create a new CA using the latest algorithms and cipher suites (the better solution), or have the old certificate resigned.
  • For each user and server
    • Deploy the CA certificate into the trust stores with a new name – eg CASSS2020
  • For each user and server
    • Renew or recreate each personal and server certificate and sign it with the new CA
    • Deploy it securely to each user and server and replace the old personal certificate
    • You may want to deploy the private key only when attached to the local network.  If you download it over wireless, or email, these may be compromised, and the bad guys can just use the new copy.  Downloading it over a wired connection eliminates this risk.
  • For the server create java.security rules to disable weaker algorithms and cipher specs.   It is easy to undo this change.
    • This should identify any user not using the latest cipher suites and certificates.  These rules can be relaxed while problems are fixed.
  • For each user and server
    • Make a copy of the old CA certificate prior to deletion
    • Delete the old CA certificate from the trust stores eg CASSS
    • After a validation period delete the copy of the old certificate

Start all over again!

You can see how much work you need to do when you renew your enterprise CA, so make sure you  renew it in plenty of time – months rather than days.

Setting up the end user self signed certificate for mqweb

This post describes how to set up self signed certificates to authenticate end user’s access to mqweb.

You can use self signed, which is fine for test and small environments, or use signed certificate which are suitable for production,  and typical environments.  Using certificates means you do not need to specify userid and password.

The userid is taken from the CN part of the subject, and this userid is used to grant access depending on the configuration in the mqwebuser.xml file.

Information about certificates used for authentication are stored in the trust store.  For a CA signed certificate, you only need the CA certificates, not the individual certificates.  With self signed, you need a copy of the individual self signed certificate in the mqweb trust store.

Create the trust store if required.

/opt/mqm/bin/runmqckm -keydb -create -db trust.jks -pw zpassword -type jks

You need to do this once.

Create the self signed certificate for the end user

openssl req -x509 -config openssl-client2.cnf -newkey rsa:2048 -out testuser.pem -keyout testuser.key.pem -subj “/C=GB/O=aaaa/CN=testuser” -extensions ss_extensions -passin file:password.file -passout file:password.file

  • openssl req -x509  –x509 makes this a self signed request
  • -config openssl-client2.cnf – use this config file
  • -newkey rsa:2048 – create a new private key with 2048 bits rsa key
  • -out testuser.pem – put the request in this file
  • -outform PEM – with this format
  • -keyout testuser.key.pem – put the key in this file
  • -subj “/C=GB/O=aaaa/CN=testuser” – this is the DN.   The CN= is the userid used by mqweb to determine the role.  It must match the case of userid
  • -extensions ss_extensions – see below
  • -passin file:password.file -passout file:password.file – passwords are in this file
[  ss_extensions  ]

subjectKeyIdentifier = hash
#Note: there is a bug in Chrome where it does 
# not accept certificates if basicConstraints
# is specified
# basicConstraints   = CA:false
subjectAltName       = DNS:localhost, IP:127.0.0.1
nsComment            = "OpenSSL Self signed Client"
keyUsage             = critical, keyEncipherment
extendedKeyUsage     = critical, clientAuth

Create an intermediate pkcs12 keystore so it can be imported to your browser.

You need to import the certificate and private key into the browser’s keystore.   The only way I found of doing this was via an intermediate pkcs12 keystore (with extension .p12).   If you import the certificate and key from the web browser, it will expect a .p12 file.

openssl pkcs12 -export -in testuser.pem -inkey testuser.key.pem -out testuser.p12 -name “testuser” -passin file:password.file -passout file:password.file

  • openssl pkcs12 – request to process a pkcs12 keystore
  • -export – to create it
  • -inkey testuser.key.pem – this private key
  • -in testuser.pem – this certificate returned from the CA
  • -out ssks.p12 – the name of the key store which is created
  • -name testuser – create this name in the keystore
  • -passout file:password.file -passin file:password.file – use these passwords

Import the intermediate keystore into the trust store

/opt/mqm/bin/runmqckm -cert -import -target trust.jks -target_type jks -file testuser.p12 -label testuser -pw password -target_pw zpassword

  • /opt/mqm/bin/runmqckm – run this command
  • -cert – we want to process a certificate
  • -import – w want to import a file
  • -target trust.jks – this is the mqweb trust store
  • -target_type jks – it is a jks store
  • -file testuser.p12 – input file
  • -label testuser – this is the label in the trust.jks keystore
  • -pw password – the password of the testuser.p12
  • -target_pw zpassword – the password of the trust.jks keystore

In the message.log I had

E CWPKI0022E: SSL HANDSHAKE FAILURE: A signer with SubjectDN CN=testuser, O=aaaa, C=GB was sent from the target host. The signer might need to be added to local trust store /home/colinpaice/ssl/ssl2/trust.jks, located in SSL configuration alias defaultSSLConfig. The extended error message from the SSL handshake exception is: PKIX path validation failed: java.security.cert.CertPathValidatorException: The certificate issued by CN=testuser, O=aaaa, C=GB is not trusted; internal cause is:
java.security.cert.CertPathValidatorException: Certificate chaining error

When I restarted mqweb and the certificate was accepted.

I had the same message when I did not import the certificate into the trust store.

Import the temporary keystore into the Chrome keystore

pk12util -i testuser.p12 -d sql:/home/colinpaice/snap/chromium/986/.pki/nssdb/ -W password

  • pk12util   – this command
  • -i  testuser.p12  – from this keystore
  • -d sql:/home/colinpaice/snap/chromium/971/.pki/nssdb/ – into this key store
  • -W password  – using this password (for the temporary .p12 file)

Remove the intermediate file

rm testuser.p12

Update the mqweb configuration

<webAppSecurity allowFailOverToBasicAuth="false" />
<keyStore id="defaultKeyStore" 
          location="/home/colinpaice/ssl/ssl2/mqweb.p12" 
          type="pkcs12" 
          password="password"/>

<keyStore id="defaultTrustStore" 
          location="/home/colinpaice/ssl/ssl2/trust.jks" 
          type="JKS" 
          password="password"/>

<ssl     id="defaultSSLConfig" 
         keyStoreRef="defaultKeyStore" serverKeyAlias="mqweb" 
         trustStoreRef="defaultTrustStore" sslProtocol="TLSv1.2"
         clientAuthentication="true" 
         clientAuthenticationSupported="true" 
/>

Stop mqweb

You need to restart mqweb so it picks up any changes to the trust store.

/opt/mqm/bin/endmqweb

Start mqweb

/opt/mqm/bin/strmqweb

No messages are produced in  /var/mqm/web/installations/Installation1/servers/mqweb/logs/messages.log if the trust store was opened successfully.

Use a command like grep ” E ” messages.log  and check for messages like

CWPKI0033E: The keystore located at /home/colinpaice/ssl/ssl2/trust.jks did not load because of the following error: Keystore was tampered with, or password was incorrect

Try using it in Chrome

You need to restart Chrome to pick up the changes to its keystore.  Use the url chrome://restart/

Use the url chrome://settings/certificates , to check your certificate is present under “Your certificates”. If not use url chrome://version to display the profile being used, and that it matches the store used in the pk12util command above.

Try connecting to mqweb using a url like https://127.0.0.1:9443/ibmmq/console/ .

You should be logged on with no password request. In the top right hand corner of the screen you should have a black circle with a white “i” in it.   This shows you are logged on with certificates.

Setting up the end user CA signed certificate for mqweb

You want to use certificates to authenticate access to a mqweb server.  You can use self signed, which is fine for test and small environments, or use signed certificate which are suitable for production,  and typical environments.  Using certificates means you do not need to specify userid and password.

The userid is taken from the CN part of the subject, and this userid is used to grant access depending on the configuration in the mqwebuser.xml file.

This section assumes you have set up your mqweb using a certificate authority.

Information about certificates used for authentication are stored in the trust store.  For a CA signed certificate, you only need the CA certificates, not the individual certificates.  With self signed, you need a copy of the individual self signed certificate.

Create the trust store if required.

/opt/mqm/bin/runmqckm -keydb -create -db trust.jks -pw zpassword -type jks

You need to do this once.

Add the CA certificate to the trust store

/opt/mqm/bin/runmqckm -cert -add -db trust.jks -file cacert.pem -label CACert -type jks -pw zpassword

You need to do this for each CA certificate you want to add, giving each CA a unique label.

You need to restart mqweb so it picks up any changes to the trust store, but as you will be changing the mqwebuser.xml – the restart can wait will later.

Create the certificate request for the end user

openssl req -config client.cnf -newkey rsa:2048 -out colinpaice.csr -outform PEM -keyout colinpaice.key.pem -subj “/C=GB/O=cpwebuser/CN=colinpaice” -extensions client_extensions -passin file:password.file -passout file:password.file

  • openssl req – the absence of -x509 makes this a certificate request
  • -config client.cnf – use this config file
  • -newkey rsa:2048 – create a new private key with 2048 bits rsa key
  • -out colinpaice.csr – put the request in this file
  • -outform PEM – with this format
  • -keyout colinpaice.key.pem – put the key in this file
  • -subj “/C=GB/O=cpwebuser/CN=colinpaice” – this is the DN.   The CN= is the userid used by mqweb to determine the role.  It must match the case of userid
  • -extensions client_extensions – see below
  • -passin file:password.file -passout file:password.file – passwords are in this file
[ client_extensions ]

subjectKeyIdentifier = hash
# basicConstraints = CA:FALSE
subjectAltName       = DNS:localhost, IP:127.0.0.1
nsComment = "OpenSSL ColinClient"
keyUsage = critical, nonRepudiation,digitalSignature,
extendedKeyUsage = critical, clientAuth

You need to do this for each user.

Sign it

Send the certificate request to your CA.  You can use the following command to sign it.

openssl ca -config openssl-ca-user.cnf -policy signing_policy -extensions signing_req -md sha256 -keyfile cacert.key.pem -out colinpaice.pem -infiles colinpaice.csr

  • openssl ca – the signing request
  • -config openssl-ca-user.cnf – use this config file
  • -policy signing_policy – defines the requirements for the DN.  See below
  • -extensions signing_req – see below
  • -md sha256 – what encryption to be used for the message digest
  • -keyfile cacert.key.pem – the CA authorities private key
  • -out colinpaice.pem – where the output goes
  • -infiles colinpaice.csr – the input file that needs signing

Send the *.pem file back to the requestor.

You need to do this for each user.

The signing policy allows the CA administrator to define which elements are required in the DN.

[ signing_policy ]
organizationName = supplied
commonName = supplied

The certificate needs extensions which say how the certificate can be used.

[ signing_req ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints    = CA:FALSE
keyUsage            = digitalSignature
extendedKeyUsage    =  clientAuth

Create an intermediate pkcs12 keystore so certificate can be imported

You need to import the certificate and private key into the browser’s keystore.   The only way I found of doing this was via an intermediate pkcs12 keystore (with extension .p12).   If you import the certificate and key from the web browser, it will expect a .p12 file.

openssl pkcs12 -export -inkey colinpaice.key.pem -in colinpaice.pem -out colinpaice.p12 -CAfile cacert.pem -chain -name colinpaice -passout file:password.file -passin file:password.file

  • openssl pkcs12 – request to process a pkcs12 keystore
  • -export – to create it
  • -inkey colinpaice.key.pem – this private key
  • -in colinpaice.pem – this certificate returned from the CA
  • -out colinpaice.p12 – the name of the temporary key store which is created
  • -CAfile cacert.pem – use this CA certificate
  • -chain – include any CA certificates with the certificate and key
  • -name colinpaice – create this name in the keystore
  • -passout file:password.file -passin file:password.file – use these passwords

Import the temporary keystore into the Chrome keystore

pk12util -i colinpaice.p12 -d sql:/home/colinpaice/snap/chromium/986/.pki/nssdb/ -W password

  • pk12util   – this command
  • -i  colinpaice.p12  – from the temporary keystore you just created
  • -d sql:/home/colinpaice/snap/chromium/986/.pki/nssdb/ – into this key store
  • -W password  – using this password (for the temporary .p12 file)

Remove the intermediate file

rm colinpaice.p12

You do not need to import the certificate into the mqweb trust store.

Update the mqweb configuration if required

<webAppSecurity allowFailOverToBasicAuth="false" />
<keyStore id="defaultKeyStore" 
          location="/home/colinpaice/ssl/ssl2/mqweb.p12" 
          type="pkcs12" 
          password="password"/>

<keyStore id="defaultTrustStore" 
          location="/home/colinpaice/ssl/ssl2/trust.jks" 
          type="JKS" 
          password="password"/>

<ssl     id="defaultSSLConfig" 
         keyStoreRef="defaultKeyStore" serverKeyAlias="mqweb" 
         trustStoreRef="defaultTrustStore" sslProtocol="TLSv1.2"
         clientAuthentication="true" 
         clientAuthenticationSupported="true" 
/>

Stop mqweb

It is better to stop and restart mqweb if you change the xml config files, otherwise you can get strange errors.

/opt/mqm/bin/endmqweb

Start mqweb

/opt/mqm/bin/strmqweb

No messages are produced in  /var/mqm/web/installations/Installation1/servers/mqweb/logs/messages.log if the trust store was opened successfully.

Use a command like grep ” E ” messages.log  and check for messages like

CWPKI0033E: The keystore located at /home/colinpaice/ssl/ssl2/trust.jks did not load because of the following error: Keystore was tampered with, or password was incorrect

Try using it in Chrome

You need to restart Chrome to pick up the changes to its keystore.  Use the url chrome://restart/

Use the url chrome://settings/certificates , to check your certificate is present under “Your certificates”. If not use url chrome://version to display the profile being used, and that it matches the store used in the pk12util command above.

Try connecting to mqweb using a url like https://127.0.0.1:9443/ibmmq/console/ .

You should be logged on with no password request. In the top right hand corner of the screen you should have a black circle with a white “i” in it.   This shows you are logged on with certificates.  Click on this to show how you are logged on.