Debugging TLS – an easier way

I had been trying to get an IBM product on Linux to talk to LDAP using TLS and certificate authentication; where I give a certificate instead of a LDAP userid and password. Basically I got “computer say no”.

IBM products use GSKIT to manage keystores and TLS between sessions. With Java you can get out a trace of the TLS conversation. You can get out a trace on z/OS, but not on other platforms. After a day I stumbled on a different approach.

Gettting a gskit trace on z/OS.

Getting the trace was easy, understanding it was harder.

In the LDAP environment file I added GSK_TRACE=0xff. Once I had TLS to LDAP working, I used GSK_TRACE=0x04 to trace just errors. Some “errors” are recorded in the GSKTRACE as “INFO”. For example an Elliptic curve is only supported in TLS 1.3.

By default the trace goes to /tmp/gskssl.%.trc. where % is the thread id.

To format it go into USS and use the command gsktrace gskssl.83951642.trc > out, and edit the file “out”.

The linux end.

I found a great program openssl s_client. This starts a TLS handshake and prints out in easy to understand format, what is going on. For example

The command

openssl s_client -connect 10.1.1.2:1389 -cert /home/colinpaice/ssl/ssl2/ecec.pem -key /home/colinpaice/ssl/ssl2/ecec.key.pem -CAfile ~/ssl/ssl2/colinpaice.pem -x509_strict

gave the flow of the data.

stderr output

The stderr output included

depth=1 O = COLIN, OU = TEST, CN = COLIN4Certification Authority
verify error:num=19:self signed certificate in certificate chain

This is the name of the CA.

Following this was an error message. Note: it is at the top of the output – not the bottom as I expected.

SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../ssl/record/rec_layer_s3.c:1528:SSL alert number 80

See below for a discussion about the error.

stdout output

Certificate chain
  0 s:O = SERVER, OU = SSS, CN = ZZZZ
    i:O = COLIN, OU = TEST, CN = COLIN4Certification Authority
  1 s:O = COLIN, OU = TEST, CN = COLIN4Certification Authority 
    i:O = COLIN, OU = TEST, CN = COLIN4Certification Authority 

This says

• the certificate SERVER was signed by COLIN4.Certificate Authority
• the certificate COLIN4Certificate Authority was self signed

The server certificate comes next

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Followed by the identity of the server

subject=O = SERVER, OU = SSS, CN = ZZZZ
issuer=O = COLIN, OU = TEST, CN = COLIN4Certification Authority

Next comes the list of Certificate Authority or self signed certificate in the keyring

Acceptable client certificate CA names
O = ADCD, OU = TEST, CN = MQWEB2 
O = COLIN, OU = TEST, CN = COLIN4Certification Authority
C = GB, O = SSS, OU = CA, CN = SSCARSA1024
C = GB, O = SSS, OU = CA, CN = SSCA256

Any client certificate must be signed by one of those CAs (or self signed certificates).

Then comes the client certificate types the server will accept

• Client Certificate Types: RSA sign, DSA sign

Note: This does not include ECDSA, so elliptic certificates are not supported in this configuration.

After this is the signature algorithms the server will accept

• Requested Signature Algorithms: RSA+SHA512: ECDSA+SHA512: RSA+SHA384: ECDSA+SHA384: RSA+SHA256: ECDSA+SHA256: DSA+SHA256: RSA+SHA224: ECDSA+SHA224: DSA+SHA224: RSA+SHA1: ECDSA+SHA1: DSA+SHA1

It gives information on the session so far

---

SSL handshake has read 2493 bytes and written 1537 bytes
Verification error: self signed certificate in certificate chain

and information about the key sent down. It was defined with SIZE(4096) RSA.

New, SSLv3, Cipher is AES256-SHA
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated

and about the TLS session

SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-SHA
    Session-ID: 0401001A0A0100029...
    Session-ID-ctx: 
    Master-Key: 32D5B4AD162F0403E323DB0...
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1634115022
    Timeout   : 7200 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
    Extended master secret: no

The -msg option gave me

>>> ??? [length 0005]
    16 03 01 01 31
>>> TLS 1.3, Handshake [length 0131], ClientHello
    ...
<<< ??? [length 0005]
    ... 
<<< TLS 1.3, Handshake [length 0051], ServerHello
    ...
<<< TLS 1.2, Handshake [length 0839], Certificate
    ...
<<< TLS 1.2, Handshake [length 0123], CertificateRequest
    ...
<<< TLS 1.2, Handshake [length 0004], ServerHelloDone
    ... 
>>> ??? [length 0005]
    ...
>>> TLS 1.2, Handshake [length 021c], Certificate
    ...
>>> ??? [length 0005]
    ...
>>> TLS 1.2, Handshake [length 0206], ClientKeyExchange
    ...
>>> ??? [length 0005]
    ...
>>> TLS 1.2, Handshake [length 0050], CertificateVerify
    ...
>>> ??? [length 0005]
    ...
>>> TLS 1.2, ChangeCipherSpec [length 0001]
    ...
>>> ??? [length 0005]
    ...
>>> TLS 1.2, Handshake [length 0010], Finished
    ...
<<< ??? [length 0005]
    ...
<<< TLS 1.2, Alert [length 0002], fatal internal_error
    02 50

The fatal errors are described here. 0x02 is fatal error, 0x50= 80 = internal error. internal_error(80). Which is not very helpful.

In the LDAP log I got

GLD1116E Unable to initialize an SSL connection with 10.1.0.2: 434 – Certificate key is not compatible with cipher suite.

In the gsktrace on z/OS I got

cms_validate_certificate_mode_int(): Validating CN=ecec,O=cpwebuser,C=GB ENTRY
INFO get_issuer_certificate(): Using issuer CN=SSCA256,OU=CA,O=SSS,C=GB

ERROR read_v3_certificate(): Client certificate key type 13 not allowed for SSL V3 cipher 0x0035

• Client certificate key type 13. Looking in /usr/include/gskcms.h under X.509 data types, was x509_alg_ecPublicKey.
• SSL V3 cipher 0x0035 is listed here, as TLS_RSA_WITH_AES_256_CBC_SHA = 256-bit AES encryption with SHA-1 message authentication and RSA key exchange.
• x509_alg_ecPublicKey is elliptic public key – which is incompatible with RSA key exchange.

When I used an RSA certificate

openssl s_client -connect 10.1.1.2:1389 -cert /home/colinpaice/ssl/ssl2/rsaca256.pem -key /home/colinpaice/ssl/ssl2/rsaca256.key.pem -CAfile ~/ssl/ssl2/colinpaice.pem -verify_return_error -policy_print -x509_strict

it worked