Would you lock your front door and leave the key under the mat? So why do you do it with digital keys?

Where I live it is Island Mentality. Someone said to me that they do not lock their front door. Sometimes, when they come home, they find some eggs or tray-bakes on the kitchen table. They went on a celebration cruise, but could not find the key to the front door, and so left the house unlocked the two weeks they were away.

Digital certificates and keys are used for identification authentication. Often these are stored in a key store, just a file in Windows or Unix. You typically need a password to be able to read the file. If you got hold of a keystore, you could try “password” with an “o”, “passw0rd” with zero etc. There is no limit to the number of attempts you can have. Don’t worry, the password is stored in a stash file , which is just another file. If you have the key store and the stash file you can open the keystore using standard commands. Having both the keystore and the stash file is like finding the front door unlocked.

If someone is an administrator on the machine, they can access any file and so can get the keystore and the stash file. IBM says you need superuser access to install MQ – so the MQ administrator can access these files. I heard that one enterprise was doing backups from the user’s machines to a remote site. The files were encrypted at the remote site, but not the network link to the remote site – whoops! The files could have been stolen en route.

Use external security devices.

You can get round this problem by using an external Hardware Security Module. Instead of storing the keys in a file, they are stored on an external device. You can get USB like devices. Some HSM can store keys, other HSMs can encrypt data. For example my bank gives its user’s a small machine. You put in your debit card, enter your pin. It encrypts the data and generates a one time key which you enter into the bank’s web site.

To steal the keystore you now need access to the physical machine to be able to unplug the USB.

Built in devices that cannot be removed.

On some machines, such as z hardware, they have a tamper resistant “cryptographic chip” built in. If you remove it from the machine, it is useless. When you configure it you need three keys, so you have three people each with their own key. When you install the backup machine, the three people have to go on site, and re enter their keys. They have mechanisms like three wrong passwords and it self destructs (perhaps in a cloud of smoke, as it does in the movies).

“Cloud”

One of the selling points of cloud is flexibility. You can deploy an image anywhere; you can wheel in new machines, and wheel out old machines; and you can have different “tenants” on the same hardware. This makes it difficult to use an HSM device to store your keys, as each machine needs the same keys, and the HSM could have all the keys from all the tenants. So you have the problem, of having your key store as a file with its stash file, and even more people have access to these files.

Would you lock your front door and leave the key under the mat? So why do you do it with digital keys

It is all down to the management of risk. Digital certificates do not give absolute protection. Strong encryption just means it takes longer to crack!

Getting SSL/TLS to work on MQ on z/OS

After I succeeded in getting TLS 1.3 to run on MQ  midrange 9.2, I thought I would try it on z/OS.  I had not used TLS on z/OS for about 10 years, so it was almost like coming to the topic with very rusty knowledge.

I searched the Knowledge centre and found no relevant hits – lots of hits which were not relevant.  I eventually found an SSL related keyword, and this got me to the topic  Working with SSL/TLS on z/OS.   I think this is well documented.  It covered all the things I had to do.

The remained of this post covers the bits not covered by the documentation.

Define SSLTASKS.

You need to define SSLTASKS to be able to use TLS on z/OS.  See the comments here. I used

%CSQ9 ALTER QMGR SSLTASKS(5)

You need to restart the CHINIT if you change the value of SSLTASKS.

Set up the keyring for the queue manager. 

See here.  This post show how to create the keyring and import a CA from z/OS, and import a CA  from a Linux system.

If you alter the keyring or certlabl you just need a refresh security type(SSL) command to pick up the changes.

Defining the channel

I tried to define the channel, as this failed for security reasons, I’ve given the RACF setup I had to do.

In this section I defined the specific commands for example DEFINE.CHANNEL.   I could have defined DEFINE.* to allow all define commands.

I used a channel called TLS, and define the resource CSQ9.CHANNEL.TLS* to allow my ID to define TLS, TLS1 etc

The command %CSQ9 DEF CHL(TLS) CHLTYPE(SVRCONN) gave me

ICH408I USER(CSQOPR ) GROUP(SYS1 ) NAME(COLIN PAICE ) 167
CSQ9.DEFINE.CHANNEL CL(MQCMDS )
WARNING: INSUFFICIENT AUTHORITY – TEMPORARY ACCESS ALLOWED
ACCESS INTENT(ALTER ) ACCESS ALLOWED(NONE )

I used the RACF commands in a batch job.

 /* RDELETE MQCMDS CSQ9.DEFINE.CHANNEL
RDEF MQCMDS CSQ9.DEFINE.CHANNEL UACC(NONE)
PERMIT CSQ9.DEFINE.CHANNEL CLASS(MQCMDS ) –
        ID(COLIN,IBMUSER) ACCESS(ALTER )

I also set up CSQ9.DELETE.CHANNEL and CSQ9.ALTER.CHANNEL in a similar way, so my userid could maintain the channels.

I refreshed MQ security %CSQ9 refresh security to pick up the changes.

I reissued the command %CSQ9 DEF CHL(TLS ) CHLTYPE(SVRCONN) and got

ICH408I USER(COLIN ) GROUP(SYS1 ) NAME(COLIN PAICE )
CSQ9.CHANNEL.TLS CL(MQADMIN )
PROFILE NOT FOUND – REQUIRED FOR AUTHORITY CHECKING
ACCESS INTENT(ALTER ) ACCESS ALLOWED(NONE )

I used the RACF commands in a batch job.

  /* RDELETE MQADMIN CSQ9.CHANNEL.TLS*
RDEF MQADMIN CSQ9.CHANNEL.TLS* UACC(NONE) WARNING
PERMIT CSQ9.CHANNEL.TLS* CLASS(MQADMIN) –    
        ID(COLIN,IBMUSER) ACCESS(ALTER )
SETROPTS RACLIST(MQADMIN) REFRESH

I issued the commands

%CSQ9 refresh security
%CSQ9 DEF CHL(TLS ) CHLTYPE(SVRCONN)

And successfully defined the channel.

I changed the cipher spec.

I selected a cipher spec from the list.

%CSQ9 alter chl(TLS) chltype(SVRCONN) SSLCIPH(ECDHE_RSA_AES_128_CBC_SHA256)

When I started the channel I got

CSQX631E … CSQXRESP Cipher specifications differ,  channel TLS local=ECDHE_RSA_AES_128_CBC_SHA256 remote=TLS_RSA_WITH_AES_256_GCM_SHA384
connection 10.1.0.2 (10.1.0.2)

This was clear;  I love clear messages.

I decided to change the z/OS end

%CSQ9 alter chl(TLS) chltype(SVRCONN) SSLCIPH(TLS_RSA_WITH_AES_256_GCM_SHA384 )

and the client connected successfully.

With MQ 9.2 I could (and did) change this to

%CSQ9 alter chl(TLS) chltype(SVRCONN) SSLCIPH(ANY_TLS12)

and the client worked successfully.  The ANY_TLS12.  provides a wide list of supported cipher specifications, includes TLS_RSA_WITH_AES_256_GCM_SHA384  and ECDHE_RSA_AES_128_CBC_SHA256. 

When I am ready to support TLS 1.3 I will use ANY_TLS12_OR_HIGHER and ANY_TLS13_OR_HIGHER.

Connect a client to it!

I had had my client connect to a midrange queue manager, so I had working client environment.  See here for the journey.

I created a .json file for the CCDT connection to z/OS.  I specified

{ "channel":
  [{
    "name": "TLS",
    "clientConnection":
    {
      "connection":
      [{
        "host": "10.1.1.2",
        "port": 1414
       }],
      "queueManager": "CSQ9"
    },
    "transmissionSecurity":
    {
      "cipherSpecification": "ANY_TLS12",
      "certificateLabel": "rsaca256_client",
      "certificatePeerName": ""
    },
    "type": "clientConnection"
  }]
}

When it connected I got messages

+CSQX511I %CSQ9 CSQXRESP Channel TLS started connection 10.1.0.2
ICH408I USER(COLINPAI) GROUP( ) NAME(??? )
LOGON/JOB INITIATION – USER AT TERMINAL NOT RACF-DEFINED
IRR012I VERIFICATION FAILED. USER PROFILE NOT FOUND.
+CSQX512I %CSQ9 CSQXRESP Channel TLS no longer active connection 10.1.0.2

COLINPAI came from the userid on the Linux machine (colinpaice) upper cased and truncated. This id will be flowed and used as the MCAUSER if you don’t set it to anything else, using CHLAUTH for example  (Thanks to Morag for this information).

Enable chlauth

To be able to map from the DN in a certificate to a z/OS userid you have to use MQ CHLAUTH.  See  Mapping a client user ID to an MCAUSER user ID.

Check it is enabled at the queue manager level and enable it it needed.

%CSQ9 DIS QMGR CHLAUTH
%CSQ9 ALTER QMGR CHLAUTH(ENABLED)

Define a mapping from certificate to userid

I used

//S1 EXEC PGM=CSQUTIL,PARM='CSQ9' 
//STEPLIB  DD DSN=COLIN.MQ921.SCSQLOAD,DISP=SHR 
//         DD DSN=COLIN.MQ921.SCSQANLE,DISP=SHR 
//SYSPRINT DD SYSOUT=* 
//SYSIN   DD * 
 COMMAND DDNAME(COMMAND) 
//COMMAND DD * 
 SET CHLAUTH('TLS') + 
     TYPE(SSLPEERMAP) SSLPEER('O="cpwebuser"') + 
     ACTION(REPLACE)   + 
     MCAUSER(ADCDD ) CHCKCLNT(ASQMGR) 
/* 

This says for channel TLS,  take the Organisation(O=..)  from the certificate, and if it is cpwebuser then set the ID to ADCDD.

Check it works

Once the channel had started I used

%CSQ( DIS CHS(TLS)
it displayed the following, where I have removed lines which are not relevant to TLS and added some comments

  • CHSTATUS(TLS)
  • CHLTYPE(SVRCONN)
  • SECPROT(TLSV12) – this is the level of the protocol
  • SSLCERTI(CN=SSCARSA1024,OU=CA,O=SSS,C=GB)- this is the DN of the issuer of the SSLPEER certificate (below)
  • SSLCERTU(START1) – the IBM documentation says “The local user ID associated with the remote certificate.”  I dont know where this comes from.. how to change it, or where it is used.
  • SSLCIPH(TLS_RSA_WITH_AES_256_GCM_SHA384) – The negotiated cipher spec
  • SSLRKEYS(0) -The number of successful TLS key resets.
  • SSLKEYTI() -The time on which the previous successful TLS secret key was reset.  The secret key has not been reset
  • SSLKEYDA() -The date on which the previous successful TLS secret key was reset.  The secret key has not been reset
  • SSLPEER(SERIALNUMBER=01:90,CN=rsaca256,O=cpwebuser,C=GB, UNSTRUCTUREDNAME=openssl_ca_user_cnf.keyAgreement2, UNSTRUCTUREDNAME=localhost, UNSTRUCTUREDADDRESS=127.0.0.1) . This is information from the certificate at the remote end.
  • MCAUSER(ADCDD) – This is the userid (set by the CHLAUTH above) used by this channel.
  • LOCLADDR(10.1.1.2(1414)) – This is the address the connection came in from.  This value will be different it you have different IP stacks and different listener ports.

Setting up the MQ keyring on z/OS

I wanted to connect some clients to my z/OS queue manager over a TLS channel.  This post describes how I set up the z/OS keyring with the certificates.

Define the keyring.

The CSQ9 CHINIT runs with a userid of START1, so I defined a keyring belonging to that id.

I set up a dataset  called IBM.MQCSQ9.KEYRING to keep all of my JCL in for the CSQ9 queue manager.  This makes it easier to clone the definitions for another queue manager.

The definitions create the keyring, and add the z/OS CA certificate (CERTAUTH ADCD_CA) to it.

//IBMRACF JOB 1,MSGCLASS=H 
//* Use JCL for the RACF definitions
//S1 EXEC PGM=IKJEFT01,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD *
/*RACDCERT DELRING(MQRING) ID(START1)
RACDCERT ADDRING(MQRING) ID(START1)
RACDCERT ID(START1 ) -
   CONNECT(RING(MQRING) LABEL('ADCD-CA') CERTAUTH)
SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh

Define the queue manager’s certificate.   This uses an Elliptic curve with key size of 256.

 /*RACDCERT ID(START1) DELETE(LABEL('CSQ9CERT')) 
RACDCERT ID(START1) GENCERT - 
  SUBJECTSDN(CN('CSQ9CERT')  - 
             O('ADCD') - 
             OU('TEST')) - 
   SIZE(256) - 
   NISTECC - 
   SIGNWITH (CERTAUTH LABEL('ADCD-CA')) - 
ALTNAME(IP(10.1.1.9)) - NOTAFTER( DATE(2021-12-29))- WITHLABEL('CSQ9CERT')
RACDCERT id(START1) ALTER(LABEL('CSQ9CERT'))TRUST RACDCERT ID(START1) CONNECT(RING(MQRING) - LABEL('CSQ9CERT') USAGE(PERSONAL)) RACDCERT LISTRING(MQRING) ID(START1) RACDCERT LIST(LABEL('CSQ9CERT' )) ID(START1) SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh

It need ALTNAME(IP(10.1.1.2)) ( or similar) because some browsers check this name, with the IP address of the server.

 

Configure the queue manager

%CSQ9 ALTER QMGR SSLKEYR(START1/MQRING) CERTLABL(CSQ9CERT)

Note if you use mixed case keyring you need to put the value in quotes.

Export the CA certificate from z/OS and sent to the client machine

RACDCERT CERTAUTH EXPORT(LABEL('ADCD-CA'))- 
DSN('IBMUSER.CERT.ADCDCA.PEM') -
FORMAT(CERTB64) -
PASSWORD('password')

The data set  IBMUSER.CERT.ADCDCA.PEM  contained text, and the first line is —–BEGIN CERTIFICATE—– .   Send this file to the client machine, for example using FTP.  I sent it as zos.adcdca.pem.

Import this to the keystore

runmqakm -cert -add -file zos.adcdca.pem -type cms -stashed -db zzclient.kdb -label zosca

Upload  the certificates from Linux to z/OS.

On Linux, my CA certificiate was in a *.pem file where the first line was —–BEGIN CERTIFICATE—–.  Send this to z/OS.  I used FTP.

Import the CA into the keyring.

The command adds an existing certificate CARSA1024 for userid START1.  The CONNECT USAGE(CERTAUT) defines this as a CA certificate (without the need to have the certificate belong to CERTAUTH userid).

  /*RACDCERT DELETE  ( LABEL('CARSA1024')) ID(START1) 
 SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh 
                                                                   
 RACDCERT ADD('IBMUSER.CARS1024.PEM')  - 
   ID(START1)  WITHLABEL('CARSA1024') 
 RACDCERT ID(START1) CONNECT(RING(MQRING    ) - 
                             USAGE(CERTAUTH) - 
                             ID(START1)  - 
                             LABEL('CARSA1024') 
 racdcert listring(MQRING ) id(start1) 
 SETROPTS RACLIST(DIGTCERT,DIGTRING ) refresh 

Refresh the queue manager

%CSQ9 refresh security type(SSL)

I’m thinking of using MQ MinimumRSAKeySize. What do I need to plan for?

If you enable this option in the qm.ini or mqclient.ini, it can have problems which are hard to diagnose.

This option restricts

  1. the key size of certificates with a type of RSA
  2. the key size of any CA certificates  with a type of RSA, used to sign a certificate (whether the certificate is RSA or not).

For example with MinimumRSAKeySize=2048 you cannot use a certificate or CA generated with openssl genpkey -out $name.key.pem -algorithm RSA -pkeyopt rsa_keygen_bits:1024.

On the client machine in /var/mqm/errors/*01* I got AMQ9633E: Bad SSL certificate for channel ….

On z/OS I got CSQX620E … CSQXRESP System SSL error, channel … connection … function ‘gsk_secure_socket_init’ RC=541.  Code 451 means “bad certificate” was received from the remote end.

How do I check?

There is no easy way of displaying the details of all of the certificates in a key store.

Midrange.

You can use the following command to list the labels all of the certificates

runmqakm -cert -list all -db zzclient.kdb -type cms -stashed -v

Then use the following command to display the details of each label in turn (zosca in the example)

runmqakm -cert -details -db zzclient.kdb -type cms -stashed -label zosca

This displays information like the example below for a CA certificate of type RSA and key size 1024.

Label : zosca
Key Size : 1024
Version : X509 V3
Serial : 00
Issuer : "CN=z/OSCertification Authority,OU=TEST,O=ADCD"
Subject : "CN=z/OSCertification Authority,OU=TEST,O=ADCD"
Not Before : 7 July 2020 00:00:00 GMT+01:00
Not After : 7 July 2021 23:59:59 GMT+01:00
Public Key
    ...
Public Key Type : RSA (1.2.840.113549.1.1.1)
  

or for an Elliptic certificate with key size 256.

Label : ca256
Key Size : 256
Version : X509 V3
Serial : ...
Issuer : CN=SSCA256,OU=CA,O=SSS,C=GB
Subject : CN=SSCA256,OU=CA,O=SSS,C=GB
Not Before : 7 February 2021 11:24:56 GMT
Not After : 7 February 2024 11:24:56 GMT
Public Key
    ... 
Public Key Type :  EC_ecPublicKey 

You can only check the certificates that are in your key store, not certificates that are sent as part of the handshake. 

z/OS

The listring command displays the contents of the ring (owner and label).

RACDCERT LISTRING(MQRING) ID(START1)

The list command displays the details of a certificate.

RACDCERT certauth LIST(label(‘ADCD-CA’))

displays information like, for the RSA certificate with a small key size,

 Label: ADCD-CA                                                       
 Certificate ID: 2QiJmZmDhZmjgcHEw8Rgw8FA                             
 Status: TRUST                                                        
 Start Date: 2020/07/06 23:00:00                                      
 End Date:   2021/07/07 22:59:59                                      
 Serial Number: ...                                                
 Issuer's Name:                                                      
      >CN=z/OSCertification Authority.OU=TEST.O=ADCD<                 
 Subject's Name:                                                      
      >CN=z/OSCertification Authority.OU=TEST.O=ADCD<                 
 Signing Algorithm: sha1RSA                                           
 Key Usage: CERTSIGN                                                  
 Key Type: RSA                                                        
 Key Size: 1024                                                       

Certificates signed by this CA would not work if MinimumRSAKeySize=2048 was specified.

Colins updates to MQ messages for TLS

As I was trying to get TLS to work on midrange, I had many MQ error messages. Sometimes the messages were a bit vague “you’ve had a problem. Resolve it and restart the channel”.

Below is the list of messages I’ve added comments to. I’ve done it as a blog post as well-known search engines are not finding the pages.

Mid range

z/OS

Getting MQ clients to TLS 1.3

King Midas was a greedy king and was granted a wish by Dionysus, the god of wine.   The king wished that every thing he touched turned into gold. His wish was granted, and every thing he touched turned to gold.  As a result he died of starvation.     There is also a saying that you cannot make a silk purse out of a pig’s ear.  I feel I have been gifted with the ability that anything I touch, breaks; as they say, making a pig’s ear out of a silk purse.

I thought changing my queue manager from using TLS 1.2 to TLS 1.3 would be easy.  This blog post describes my journey.  It was easy – I just had one problem which was easy to solve once I had spent a couple of hours work to identify the problem.

A short history of cipher specs in MQ.

In TLS 1.2 a cipher has a name like TLS _ KeyexchangeAlgorithm _  Keytype _ WITH _ BulkEncryption _ HashFunction.   The values have to be consistent at both ends, and the Keytype must match the certificate used during the hand shake. Names like  ECDHE_ECDSA_AES_128_CBC_SHA256 are the same as  of TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, with the TLS_ chopped off and shortened.  See here.

During the TLS 1.2 handshake the client sends a list of cipher suites it supports.  The server has a list of cipher suites it supports.  The lists are merged and any cipher specs in both lists are used in the subsequent processing.   The lists can have 1 or more elements in them.

With the server, it is a bit more subtle than this. The client sends its list of cipher suites to the server.  The server has a list of cipher suites it will accepts.    In MQ 9.2 the qm.ini has

  •  AllowTLSV13 whether to add the TLS 1.3 cipher specs to the standard list.   It defaults to yes.
  • AllowedCipherSpecs which allows you to specify which cipher specs can be used in the handshake.  It has a default list.

The first cipher spec which is in both the client and server lists is passed to the next stage of processing; checking with the svrconn channel’s SSLCIPHR parameter.

In MQ 9.1 and before, the channel lists had only one element in them.  This meant  the cipher spec TLS_A_B_WITH_C_D had to be specified at each end of the channel.  If you wanted to change the cipher spec you had to change the svrconn channel definition, and all of the client’s definitions to use the new cipher spec – and then restart all of the channels.  This was difficult to do, and hard to roll back.

In 9.1.1 the ANY_TLS12 list was implemented.   This is a list of cipher specs, for example TLS_A_B_WITH_C_D, TLS_A_B_WITH_E_F, TLS_X_Y_WITH_C_D, TLS_X_Y_WITH_E_F.

During the handshake the client passed its cipher spec to the server.   If the cipher spec was in this list the handshake would work.   This meant client1 could use TLS_A_B_WITH_C_D and client2 could use a different cipher spec TLS_X_Y_WITH_E_F.
Client1 can easily change to use a stronger cipher spec TLS_X_Y_WITH_E_F.  It just has to change its definition and restart.

This means you can do a phased change of clients using a channel.

You can specify the order of the cipher specification in the AllowedCipherSpecs list.    Usually the order is strong to weak.

I recommend using the ONLY_TLSxx_OR_ABOVE so any supported cipher specs can be used.

Client Server Supported
ANY_TLS12 ANY_TLS12 Yes TLS 1.2
ANY_TLS12 ANY_TLS12_OR_ABOVE Yes TLS 1.2
ANY_TLS12 ANY_TLS13 No
ANY_TLS12 ANY_TLS13_OR_ABOVE No
ANY_TLS12_OR_ABOVE ANY_TLS12 No.  It will try default to TLS 1.3 and fail
ANY_TLS12_OR_ABOVE ANY_TLS12_OR_ABOVE Yes TLS 1.2 or 1.3, Default will be 1.3
ANY_TLS12_OR_ABOVE ANY_TLS13 Yes TLS 1.3 – but not TLS 1.4 when it comes out
ANY_TLS12_OR_ABOVE ANY_TLS13_OR_ABOVE Yes TLS 1.3

How to migrate clients to newer levels of TLS.

Now that MQ supports list of cipher specs the migration should be pretty easy.

Take the scenario where two different customers,  needs to connect to my queue manager.  Before 9.1.x it would have been very difficult to coordinate changes between the customers, who may be in different time zone.

The steps are

  1. Change the svrconn channel to the highest available list, for example
    1. With MQ 9.1.1 you can use ANY_TLS12
    2. With MQ 9.1.4 you can use ANY_TLS_12_OR_HIGHER
    3. With MQ 9.2 you can specify ANY_TLS13_OR_HIGHER (but do not use this till you have finished upgrading to TLS 1.3).
  2. Do this for all queue managers the client connects to.
  3. For the first client, change the client definition to use the server supported level. If the client code is at 9.1.4 or higher then should can use ANY_TLS_12_OR_HIGHER (best), or give a specific TLS 1.x certificate .  See here for the list. The client cipher spec value must not be greater than the svrconn’s SSLCIPH value on all queue managers.
  4. Restart your client.  If there are problems you can revert to the previous value and restart the client.
  5. Use DIS CHS(…) ALL on the server.  It will list information like SSLCIPH(TLS_CHACHA20_POLY1305_SHA256).   This is a TLS 1.3 cipher spec.  Find any cipher specs which are not TLS 1.3 and fix then.
  6. Once you had done this for all clients using the channel, and for all queue managers using this channel, you should then be able to change the server definition to be ANY_TLS_13_OR_HIGHER and it should continue to work.
  7. You do not have to change the client to ANY_TLS13_OR_HIGHER, but you can as part of annual upgrade.  It prevents people creating a server with a TLS lower than 1.3, and having clients connect to it.

What happened to me…

I set ANY_TLS_12_OR_HIGHER at the client, and ANY_TLS_12 at the server – so the wrong way round (but I did not know this at the time).

During the handshake the client agreed with the queue manager to use TLS_CHACHA20_POLY1305_SHA256,  a TLS 1.3 cipher spec.   This was then checked by the channel code which was ANY_TLS12.   The “agreed” cipher spec was not supported so gave me the client AMQ9641E message.

EXPLANATION: The remote end of channel ‘…’ on host …’ has indicated a CipherSpec error ‘SSLCIPH(TLS_CHACHA20_POLY1305_SHA256 ) -> SSLCIPH(… )’. The channel did not start.

and the server message

AMQ9631E: The CipherSpec negotiated during the SSL handshake does not match the required CipherSpec for channel ‘…’.

EXPLANATION:
There is a mismatch between the CipherSpecs on the local and remote ends of channel ‘ECRSA1024’. The channel will not run until this mismatch is resolved. The CipherSpec required in the local channel definition is ‘ANY_TLS12’. The name of the CipherSpec negotiated during the SSL handshake is ‘TLS_CHACHA20_POLY1305_SHA256’.

Action to resolve it

  1. If I changed the svrconn channel to ANY_TLS12_OR_HIGHER it worked, and the session used a TLS 1.3 cipher spec.
  2. I could have changed the client to have ANY_TLS12 but I could potentially have a lot of clients to change.  Changing the svrconn was much simpler.

Setting up TLS on MQ Midrange

The weather is still cold up in Orkney (north of Scotland), and as I had been working on documenting TLS connections to web servers and how to get it working, I thought I would look at migrating my “enterprise” ( two queue managers and a couple of clients) to use TLS 1.3.  It took me a couple of days to set up a TLS 1.2 client to queue manager.  I spent a long time with my head in my hands thinking “why is this not working!”.
There is lots of documentation, from IBM and others, it was hard to find the best set of instructions.  I would classify the information I found as “how to dip your toe into the water” rather than “how to swim”.  At times I felt like this was a puzzle where the instructions say “There are false clues to misdirect you, and some of the clues have spelling mistakes – just to make it more interesting”. 

As well as this post I’ve created a web page of updates and suggestions to the IBM messages, with hints on where to look, and which traps to avoid.

As I was editing this document I found an excellent document on setting up MQ Clients and TLS with a java program.  The blog post below does not start from scratch, it takes an existing queue manager, and existing certificates and gets a C sample program to work.

Initial observations on key and trust stores.

Having been deep into the TLS specification and flows I know that there are two logical stores:

  1. The key store is where you store the private key used by this application.  The store typically has just one private key.  If someone has access to this store they can encrypt decrypted data using the key.
  2. The trust store. This stores the certificates needed to validate any certificates sent to it.  This typically has the Certificate Authorities, and any self signed keys (which you will used only in a sandbox environment).  You can have one trust store for all of your production applications – and other trust stores for your test environment.  Sharing a trust store is good practice, as this means you update a certificate, and every user gets the update.  If you have many trust stores you have to update them all when there is an update for a certificate.

MQ uses one store which is the trust store plus any private keys.  The disadvantage of this you have more work to keep the stores up to date with changes to certificates.

You could have a group of production queue managers sharing the same store.  You could give them each queue manager their own private key – or they could all use the same private key.  In the early days of MQ, each queue manager had a unique  certificate label (alias).  In the keystore for queue manager QMA, the label was  ibmwebspheremqqma.  You can now specify the label(alias) using the CERTLABL queue manager attribute, and on a channel.

In my work with the Liberty Web Server, I used key stores with key store types of .jks (deprecated)  and .p12 (for pkcs12).  On Midrange,  MQ uses a .CMS format used by IBM GSKIT.  On z/OS you use RACF or other z security products.   Java clients can use a variety of stores: .jks, .pem,  .p12; the list is what Java supports. Other clients, using the the GSKIT interface can only use .CMS stores.

You can import key stores into a .CMS store, so it is not a big effort to create a .CMS store – but it is an extra step to keep it up to date.

Before you start planning

(Not to be confused with doing the planning after you have started going in the wrong direction).

You need to think about what certificates and cipher suites names you can use.  See here.

Setting up TLS

I followed the IBM documentation but this was not very helpful.  For example it says “Create a key database file in the location that is specified in the queue manager’s Key repository attribute” but does not tell you how to do it.

I wanted to reuse the certificates I already had, so telling me how to create new certificate was not very useful.

The 30 second hints before you start, to avoid some of the holes in the road that I hit.

  1. Get clients connecting to your queue manager before trying to convert to TLS.
  2. Explicitly specify all  MQ environment variables,  or explicitly unset them.  For example some variables override mqclient.ini parameters, and some do not.  I spent hours debugging a problem because I had an environment variable set globally for another application.
  3. Check you do not have MQSERVER environment variable specified, or ServerConnectionParms in the mqclient.ini.
  4. Specify CERTLABL on each channel, to allow different cipher specs and certifcates to be used on different channels.
  5. Create a client (group) specific CCDT either with JSON or mqscutil for this group of clients.  Using a queue manager generated CCDT  may not give you the channel you wanted.
  6. The server’s certificate must be compatible with the cipher spec and both ends of the channel.    For example all RSA or all Elliptic curve.  This is easy to get wrong.
    1. The cipher spec of ANY_TLS12 etc available in V9.1.x make this much easier, as this uses a list of certificates.
    2. If you can use this at either end then do so.
  7. Each end needs read access to the key store files.
  8. You can create new certificates just for MQ, or reuse existing certificates.
    1. Java support can use a variety of key stores.
    2. Other programs need a .CMS format key store.
  9. Always check after making a change, as runmqsc upper cases values unless they are quoted.
  10. Remember to use refresh security type(ssl) after changing any of the queue manager’s SSL parameters, or changed the keystore.
  11. If you think it has worked, use DIS CHS(…) on the channel and make sure has picked the correct channel.  By default it finds the first channel in the CCDT for the queue manager.  For me this was not configured for TLS – so it appeared worked !!!

My journey.

What tools can help me set up my .cms keystore?

  1. GUI you can use strmqikm (STaRtMQ Interactive KeyManagement).
  2. Command line runmqckm (RUNMQCommandlineKeyManagement).
  3. For strong cipher keys, (eg Elliptic Curve public keys ), and Federal Information Processing Standards (FIPS) use the command line program runmqakm.  If you always use it instead of runmqckm, you will get muscle memory.  It provides a superset of functions over runmqckm.

The command line commands give the options available with the commands.  So you do not need the online documentation.  For example

runmqakm -keydb shows the options available for the -keydb option.

CTGSK3000W An action must be specified for this object.-Command usage-
Object Action Description
------ ------ -----------
-keydb
-changepw Change the password for a key database
-convert Convert the format of a key database
-create Create a key database
-delete Delete a key database
-expiry Display password expiry [Deprecated]
-list Currently supported types of key database.
-stashpw Stash the password of a key database into a file

Create the keystore for the server. 

I used a bash script to make it easier to change a parameter and rerun the commands.

The script

  1. deletes the old keystore
  2. creates the keystore
  3. adds in the Certificate Authority and self signed certificates
  4. imports the certificate and private key used in the handshake
  5. lists the contents of the keystore
  6. sets the authorities to the keystore.

db=”zzserver.kdb “
runmqakm -keydb -delete $db -pw password
# create the cms keystore
runmqakm -keydb -create -db $db -type cms -pw password -stash
# add the CA certificate
runmqakm -cert -add -file /home/colinpaice/ssl/ssl2/ca256.pem -type cms -stashed -db $db -label ca256
# and the self signed  called ss in the ss.p12 keystore
runmqakm -cert -import -file ~/ssl/ssl2/ss.p12 -type pkcs12 -pw password -target $db -target_stashed -label ss -new_label SS_SERVER
# add the certificate and private key into the store
#  it has a label ecc in in a pkcs12 file in ~/ssl/ssl2/ecec.p12
runmqakm -cert -import -file ~/ssl/ssl2/ecec.p12 -type pkcs12 -pw password -target $db -target_stashed -label ecec -new_label ECEC_SERVER
runmqakm -cert -list all -db $db -type cms -stashed
# give mqm access to the files
chown :mqm zzserver.*
chmod g+r zzserver.*

This created 4 files zzserver.crl zzserver.kdb zzserver.rdb zzserver.sth.

Check mqm can access it.  Run as mqm id

sudo -u mqm runmqakm -cert -list all -db zzserver.kdb -type cms -stashed

Define the keystore to the queue manager

Use the mqsc commands

alter qmgr SSLKEYR(‘/home/colinpaice/mq/zzserver’) CERTLABL(‘ECEC_SERVER’)
refresh security type(SSL)

Note the quotes around the values, without them the text is converted to upper case.

Define the channels for the TLS connection

I had an existing client connection QMACLIENT, and wanted to create a new channel QMAQCLIENTTLS.

I used the mqsc commands

  • define chl(QMAQCLIENTTLS) chltype(CLNTCONN) like(QMACLIENT)
  • define chl(QMAQCLIENTTLS) chltype(SVRCONN) like(QMACLIENT)
  • alt chl(QMAQCLIENTTLS) chltype(SVRCONN) SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA256)
  • alt chl(QMAQCLIENTTLS) chltype(CLNTCONN) SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA256)

In theory I could use the system generated CCDT table which contains client information, but this did not work in practice.  See below.

I copied the AMQCLCHL.TAB to my working directory.

cp /var/mqm/qmgrs/QMA/@ipcc/AMQCLCHL.TAB .

Edit the mqclient.ini

I changed the mqclient.ini file to include

CHANNELS:
ServerConnectionParms=QMAQCLIENTTLS/TCP/127.0.0.1(1414)
ChannelDefinitionDirectory=/home/colinpaice/mq
ChannelDefinitionFile=AMQCLCHL.TAB
SSL:
CertificateLabel=ECEC_P
SSLKeyRepository=/home/colinpaice/mq/zzclient

My first run

I used a a bash script which ended up as

# display time - useful for looking at trace 
date
# remove the old client trace files
sudo rm /var/mqm/trace/AMQ*

export MQCLNTCF=/home/colinpaice/mq/mqclient.ini
export MQSSLKEYR=/home/colinpaice/mq/zzclient
export MQCHLLIB=/home/colinpaice/mq
export MQCHLTAB=AMQCLCHL.TAB

strmqtrc -e
/opt/mqm/samp/bin/amqsgetc CP0000 QMA
endmqtrc -e

This gave me

MQCONNX ended with reason code 2393

Problems I experienced

Trying to use ServerConnectionParms

I got 2393 SSL Initialization error, and less /var/mqm/errors/*01* had

AMQ9641E: Remote CipherSpec error for channel ‘QMAQCLIENTTLS’ to host
‘localhost (127.0.0.1)(1414)’.

EXPLANATION:
The remote end of channel ‘QMAQCLIENTTLS’ on host ‘localhost (127.0.0.1)(1414)’ has indicated a CipherSpec error ‘SSLCIPH(‘ ‘) -> SSLCIPH(????)’. The channel did not start.

Note the blank cipher spec. My mqclient.ini had ServerConnectionParms=QMAQCLIENTTLS/TCP/127.0.0.1(1414) which cannot be used for TLS channels.  See ServerConnectionParms. It overrides the ChannelDefinitionFile

I commented out this statement, so the CCDT would be used.

Wrong channel used from the CCDT.

The program seemed to work successfully, until I used DIS CHS(QMA*) when I saw the wrong, a non TLS channel, was being used.

When the CCDT is used, the entries are searched for the first entry which matches the queue manager name.

I found this by taking a trace, see Debugging MQ client connection problems:channel name.

The first channel matching was QMACLIENT which is non TLS.   I had to create a CCDT just for the  TLS channel.

You can have a json format file, see here for the attribute names or a binary ccdt from runmqsc.

Creating a ccdt file using runmqsc

You can use the runmqsc -n command to display or manipulate a binary ccdt file.

I created a file using the same parameters as the queue manager version.

The file it uses depends on the environment variables  for example

export MQCHLLIB=/home/colinpaice/mq
export MQCHLTAB=COLIN2.TAB
runmqsc -n

Set the environment variable in the run shell, and remove the statement from the mqclient.ini .

The wrong keystore was used

I had both an environment variable, and an entry in the mqclient.ini  (SSL: SSLKeyRepository=/home/colinpaice/mq/zzcolin)  for the keystore.

I debugged this using  Debugging MQ client connection problems:keystore.

The environment variable had precedence over the mqclient.ini.

I had mistyped the certificate label for the client

The server’s /var/mqm/qmgrs/QMA/errors/*01* file had

  • AMQ9638E: SSL communications error for channel ‘????’.
  • AMQ9999E: Channel ‘????’ to host ‘127.0.0.1’ ended abnormally.

The clients /var/mqm/errors/*01* had

I had inconsistent Cipher specs.

As part of the handshake there is logic like..

  • Client sends “I want to negotiate in  the following languages…”
  • The Server looks in its list of languages and makes sure there is at least one in common.
  • The server asks its certificate do you speak any of the following languages.

If any of the above fail you get the AMQ9616E: The CipherSpec … proposed by remote host … is not enabled error message.

The client can send up

  • One cipher spec for example specify  SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA256)
  • A list of cipher specs SSLCIPH(ANY_TS12)

The server may have

  • One cipher specified SSLCIPH(ECDHE_ECDSA_AES_128_CBC_SHA256)
  • A list of cipher specs SSLCIPHR(ANY_TS12)

The server’s certificate could be, for example,

  • Subject Public Key Info: Public Key Algorithm: id-ecPublicKey Public-Key: (384 bit)

or

  • Subject Public Key Info: Public Key Algorithm: rsaEncryption  RSA Public-Key: (2048 bit)

The bold text must match the agreed cipher spec.

Display connection information

Once the connection has been established you can use the DIS CHS(…) all command on the server using runmqsc.
This has information like

SECPROT(TLSV12)
SSLCERTI(CN=SSCA256,OU=CA,O=SSS,C=GB)
SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA256)
SSLKEYDA( ) SSLKEYTI( )
SSLPEER(SERIALNUMBER=01:79,CN=rsaca256,O=cpwebuser,C=GB,UNSTRUCTURED...)
SSLRKEYS(0) STATUS(RUNNING)

This shows that the cipher spec used is TLS_RSA_WITH_AES_128_CBC_SHA256 and the DN from the client was had a DN with CN=rsaca256… .

I could not find the equivalent command on the client end.

You can get this from the client trace see debugging mq client connection problems:server cert

Which Cipher Spec and certificate type should I use for the MQ Server TLS 1.2?

Having got a TLS client to MQ working, I know much more about this topic than before I started.    Below is  some information I wish I had known before starting my project.   The discussion applies mainly to TLS 1.2, as there are significant changes and simplification in TLS 1.3.

Planning

Enterprise wide

  1. Do you have any standards you need to follow for TLS definitions, for example what levels of certificate are allowed, what cipher specs are allowed, what cipher specs are disallowed.
  2. Organisation often have multiple queue managers for availability reasons.  If a client can talk to a variety of queue managers the definitions need to be consistent.  Do you give each queue manager a unique certificate with a similar DN, or does each queue manager use the same certificate?  If you give each queue manager their own DN, you may have to do additional work for SSLPEER and CHLAUTH definitions.  If you create a new queue manager then existing clients should not have to change their definitions.
  3. You may have clients with different certificate formats, and DN formats, for example from different enterprises accessing your queue manager.  Your server needs to be able to support these.   You may have no control of the DN’s of clients outside of your domain.
  4. You can have the client authenticating with the server, where the server checks details from the DN in the clients certificate.
  5. You can have the client authenticate the server.    The server sends a certificate to the client, for the client to validate.
    1. You can specify a certificate on the svrconn channel definition at the server, and the DN from this certificate will be used.
    2. If there is no certificate on the channel definition, the queue manager’s certificate is used to provide the DN.
  6. You need to ensure that the CA certificates (and any self signed certificates) are stored in the server’s and/or client’s keystore.

Queue manager specific

  1. Each channel can have a different certificate.    You need to decide if you want the certificate type of RSA or EC.   Before MQ 9.1.1 it was  difficult to change it once it had been established. It does not matter how the certificate is signed (EC or RSA), or what message digest (eg -md SHA256) is used. You can restrict what RSA key sizes you accept for a certificate or CA.  See MinimumRSAKeySize.  It is easier to change your certifcate if you can use the ANY_TLS... cipher specs lists in MQ 9.1.1 and later.
  2. The cipher spec used by each channel must support the server’s certificate type.  Different channels can have different symmetrical encryption and hash algorithm values in the cipher suite.   For example one channel could have TLS_RSA_WITH_AES_256_GCM_SHA384 and another channel have TLS_RSA_WITH_AES_256_CBC_SHA256 both support a certificate type of RSA, but have different symmetrical encryption and hash algorithm values.
  3. Before MQ 9.1 each you could only specify one cipher spec value for a channel.  It had to be the same at each end, and so all clients for a channel had to have the same cipher spec.
  4. With 9.1.1 the server and the clients can support ANY_TLSxx which is a list of available cipher specs.    This means different clients can now have have different cipher specs for the same channel.

Which certificate is used on the server?

A certificate can be used for two purposes

  1. Specifying what key exchange algorithm to use
  2. For authentication

Certificates in an MQ server are used in two places

  1. For inbound channels the CERTLABL(…) certificate is used as part of the handshake to identify how the key exchange will be done. If a channel does not have a CERTLABL then it will use the queue manager defined value.   The certificate could be generated with certificate type RSA or Elliptic Curve (ECDSA).  See ALTER QMGR here.
  2. What is sent down to the client for authentication.  The certificate is checked and the DN extracted. The certificate type is not used.  The documentation says
    1. Channel attribute CERTLABL:Certificate label for this channel to use.  The label identifies which personal certificate in the key repository is sent to the remote peer. If this attribute is blank, the certificate is determined by the queue manager CERTLABL parameter.
    2. Note that inbound channels only send the configured certificate if the IBM® MQ version of the remote peer fully supports (requests) certificate label configuration, and the channel is using a TLS CipherSpec. See Interoperability of Elliptic Curve and RSA CipherSpecs for further information.

Setting up the handshake.

Both ends of a TLS connection need to agree the technique that will be used to agree secret keys.   The value_WITH of the cipher spec names give information about what they support.  For example  

  • TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384  supports ECDSA keys
  • TLS_RSA_WITH_AES_256_GCM_SHA384 supports RSA keys. 

You generate a certificate with a type of RSA or EC.  You can display the certificate type using

runmqakm -cert -details -db zzserver.kdb -type cms -stashed -label RSA_SERVER

This gives information like

  • Public Key Type : EC_ecPublicKey  Parameters: namedCurve: EC_NamedCurve_secp256r1
  • Public Key Type : RSA

The client sends up a list of one or more cipher specs it can support.   The server has a list of one or more cipher specs it supports.   The common subset is used to check the server’s certificate. If the certificate is not compatible with the subset then the request will fail with AMQ9616E: The CipherSpec … proposed by remote host …. is not enabled.

Client Server Certificate Valid?
TLS….RSA_WITH… TLS…RSA_WITH… RSA Yes
TLS….RSA_WITH… TLS…EC_WITH… RSA No
TLS….RSA_WITH… TLS…EC_WITH… EC No
TLS….RSA_WITH…,
TLS…EC_WITH…
TLS…RSA_WITH…

 

 

RSA Yes
TLS….RSA_WITH…,
TLS…EC…
TLS….RSA_WITH…,
TLS…EC_WTH…
RSA Yes
TLS….RSA_WITH..,
TLS…EC_WITH…
TLS….RSA_WTH…,
TLS…EC_WITh…
EC Yes

Once the key exchange parameters are agreed, the other parameters have to match. For example with

    • TLS_RSA_WITH_AES_256_GCM_SHA384
    • TLS_RSA_WITH_AES_256_CBC_SHA256

The symmetric data encryption techniques GCM/CBC do not match, nor do the hashing algorithms SHA256 and SHA384.   This causes message AMQ9631E.

You can specify only one value in the SSLCIPH.   This can be one cipher spec, or the name of a list, for example ANY_TLS12.  You can restrict what is used by using the AllowedCipherSpecs  option.

The documentation, for example Migrating existing security configurations to use the ANY_TLS12_OR_HIGHER CipherSpec is a little simplistic where it says change the value of the CipherSpec for the channel definition at each end, and then restart the channels for the change to take effect. It can be challenging updating and restarting 1000 channels cross multiple organisations; it is better to do a rolling upgrade.

Before MQ V9.1.x you had to specify the same cipher spec at both ends of a channel.  In MQ 9.1.x you can specify ANY_TLS12 at the server to accept any of the cipher specs in a list.

Debugging MQ client connection problems

I had lots of problems trying to get a client using TLS to connect to the queue manager, from set up errors, to unclear documentation.   It took me days to get my first channel set up.
IBM does not provide a program on midrange to format the GSKIT trace.  Someone suggested I use Wireshark  network packet analyser to monitor the traffic on the network.  I’ve given some examples of the handshake for TLS 12, TLS 13.

Below are some things you can do to check your set up is as you expect.

Review the error log

This is in /var/mqm/errors/AMQERR01.LOG
It does not provide all the information needed to identify the parameters used in the handshake, so you need to use the trace.

Turn on the MQ client trace

strmqtrc -e
start your program
endmqtrc -e

Format the trace

The client trace files are in /var/mqm/trace

  • cd /var/mqm/trace
  • dspmqtrc *.TRC

This will create several *.FMT files

Find the trace file for the connection

grep -F PeerName *.FMT

will list the file with the trace entries in it.  My file was  AMQ7232.0.FMT .

Check the channel name.

From the output of the grep -F PeerName *.FMT above, check the Channel Name: is what you expect? Check the remote IP address and port.  If you are using a CCDT it will take the first channel name which matches the queue manager.  This may not be the channel name you were expecting.    I had channels QMACLIENT and QMACLIENTTLS in the CCDT.   QMACLIENT was chosen instead of QMACLIENTTLS.

Check the mqclient.ini being used

grep -F mqclient.ini AMQ7232.0.FMT |grep -F FileName

Check the CCDT, (Channel Definition File) being used

grep -F ChannelDefinition AMQ7232.0.FMT

Adding stanza (ChannelDefinitionDirectory) length (19)
Adding stanza (ChannelDefinitionFile) length (10)
ChannelDefinitionDirectory = '/home/colinpaice/mq'
ChannelDefinitionFile = 'COLIN2.TAB'
Adding stanza (ChannelDefinitionFile) length (10)
Using ChannelDefinitionDirectory / MQCHLLIB value of /home/colinpaice/mq
Using ChannelDefinitionFile / MQCHLTAB value of RE.TAB
Adding stanza (ChannelDefinitionDirectory) length (19)
Adding stanza (ChannelDefinitionFile) length (10)
Using ChannelDefinitionDirectory / MQCHLLIB value of /home/colinpaice/mq
Using ChannelDefinitionFile / MQCHLTAB value of RE.TAB

The last entries show what was used.

Check the keystore being used

grep -Fi SSLKeyR *.FMT

Display the certlabl.

grep -Fi CertificateLabel *.FMT

It will display many records. They key ones are

  • MQCD CertificateLabel ‘rsaca256_client’
  • Saved CertificateLabel ‘rsaca256_client’

If these are missing certlabl has not been specified.

Display the channel definiton

Edit or browse the file and locate “CD “.

This will locate the MQCD (see the MQ documentation).

Interesting offsets in the CD are

  • 0x0000 Client channel name
  • 0x0060 Queue manager name
  • 0x00c0 IP address
  • 0x0698 Cipher Spec
  • 0x0780 TLSCertificate alias name in the client’s keystore.

Display Client Hello

You may have information about the TLS handshake.  This is called Client_hello, and Server_Hello.

Locate <client_hello> if found, it will have data like (some data removed).

client_version 
TLSV12
random ... 
session_id ... 
cipher_suites 
  tls_ri_scsv,tls_rsa_with_aes_128_cbc_sha256
compression_methods ...                              .
Extensions...
signature_algorithms 13
   rsa:sha512,rsa:sha384,rsa:sha256,rsa:sha224,...
server_name 
  qmaqclienttls.chl.mq.ibm.com
End of GSKit TLS Handshake Transcript

You need to understand the TLS handshake to fully understand this.

  • TLSV12 minimum level of TLS supported
  • cipher_suites these are what the client like
  • signature_algorithms  these are what the client will accept
  • server_name is “sni” information identifying the channel

What is my client connected to?

On a queue manager you an issue the DIS CHS(..)  all and get

SECPROT(TLSV12)
SSLCERTI(CN=SSCA256,OU=CA,O=SSS,C=GB)
SSLCIPH(TLS_RSA_WITH_AES_128_CBC_SHA256)
SSLPEER(SERIALNUMBER=01:79,CN=rsaca256,O=cpwebuser,C=GB,…)

I could not find a command to display the same information for the clients perspective.

You can look in the trace file for SSLCERTI for example

SSLCERTI(‘CN=SS,O=SSS,C=GB’)
SSLPEER(‘SERIALNUMBER=73:CB:2B:…,CN=SS,O=SSS,C=GB’)

This gives the server’s DN and Certificate issuer, and peer=subject.  As the peer has the same DN as the SSLCERTI this shows it is a self signed certificate.