Setting up CRL on z/OS LDAP

Certificate Revocation Lists (CRLs) are an old (and deprecated) way of checking the validity of certificates. I thought this would be a good way of understanding LDAP on z/OS.

As usual this had some surprises for me,such as having to recreate my CA, and certificates that it had issued, because it was missing the crlSign attribute.

The MQ documentation was a good start (I could not find any other documentation).

How do you know if your certificate is valid?

• The old way of doing this (from about 2008!) was to have a list containing the Certificate Authority’s certificate, and a list of the certificate serial numbers which were revoked. This was called the Certificate Revocation List(CRL). In some cases this might just be a file, but typically this was stored in LDAP. Periodically you would refresh this list.
• If may be that an intermediate CA was revoked. As well as a Certificate Revocation List you had Authority Revocation List(ARL) (from Certificate Authority).
• People realised that refreshing the CRL every day was leaving a certificate exposed for up to a day. Also this file could get very large.
• These days the checks are done “online”, using an Online Certificate Status Protocol(OCSP). A certificate has an extension with information “ask this site… if I am still valid”.
• This has been extended to support the server caching the response to the OCSP request, and sending the OCSP status down to the client so the client knows how long the status of the certificate is valid for. Eventually the OCSP status expires, and the OCSP status has to be renewed.

Revoking a certificate

I wanted to revoke a certificate that I had created on Linux, so it could not be used for authentication.

This was pretty painless, except for a couple of “ahh” moments.

I set up a bash script to revoke one of my client’s certificates, and create the file of revoked certificates.

password=” -passin file:password.file -passout file:password.file”
name=”ecec”
ca=”ca256″ # sign with this
config=”-config openssl-ca-user.cnf”
policy=”-policy signing_policy”
reason=”-crl_reason cessationOfOperation”

openssl ca $config $policy $ext –cert $ca.pem -keyfile $ca.key.pem $reason –revoke $name.pem

openssl ca $config $policy $ext –cert $ca.pem -keyfile $ca.key.pem –gencrl -out crl.pem

This revokes the ecec.pem file, which was signed by ca256.*. I picked a reason cessationOfOperation from the list of reasons.

The index.txt file for the CA, shows the certificate is revoked “R”

R 200229103805Z 211102120448Z, cessationOfOperation 492DF59CE0479E9523C9355077E1DD6F83C4B01E unknown /C=GB/O=cpwebuser/CN=ecec

The second command (–gencrl) produced a file crl.pem (base 64 encoded) with a list of revoked certificates:

—–BEGIN X509 CRL—–
MIIBIzCByQIBATAKBggqhk…
3vs3Bcdkrw==
—–END X509 CRL—–

You can display the file contents using

openssl crl -text -in crl.pem

For example

Certificate Revocation List (CRL):
    Version 2 (0x1)
    Signature Algorithm: ecdsa-with-SHA256
    Issuer: C = GB, O = SSS, OU = CA, CN = SSCA256
    Last Update: Nov 2 15:33:19 2021 GMT
    Next Update: Dec 2 15:33:19 2021 GMT
Revoked Certificates:
  Serial Number: 01
    Revocation Date: Jun 19 10:46:32 2019 GMT
  Serial Number: 023F
    Revocation Date: Nov 2 11:55:32 2021 GMT
  Serial Number: 0240
    Revocation Date: Nov 2 15:33:10 2021 GMT
    CRL entry extensions:
      X509v3 CRL Reason Code:
        Cessation Of Operation
  Serial Number: 492DF59CE0479E9523C9355077E1DD6F83C4B01E
    Revocation Date: Nov 2 12:04:48 2021 GMT
    CRL entry extensions:
      X509v3 CRL Reason Code:
        Cessation Of Operation
  Signature Algorithm: ecdsa-with-SHA256
    30:44:02:20:21:f4:70:11:42:85:22:c8:41:b9:95:d2:3d:9e:
    0a:b4:69:fa:a4:e0:37:81:05:c7:e8:98:43:a1:dc:67:58:6c:
    02:20:14:33:bd:2e:e1:57:3b:76:22:57:b9:b2:c4:a7:29:a3:
    3c:b1:4e:5a:a3:13:bc:13:61:e2:9e:a5:7c:64:b1:b1

I ftped the crl.pem file to z/OS.

Revoking a certificate on RACF

RACF does not have the support for revoking certificates. You are meant to use z/OS® Cryptographic Services PKI Services, to manage your certificates – which looks a non trivial installation.

Configure LDAP.

The CRL information is stored under the DN of the CA. My CA is called

C = GB, O = SSS, OU = CA, CN = SSCA256

So I need an entry in the LDAP tree for cn=SSCA256, ou=CA, o=SSS, c=GB.

My ldap configuration included

database LDBM GLDBLD31/GLDBLD64
useNativeAuth all
suffix “o=Your Company”
databaseDirectory /var/ldap/ldbm

the suffix means that all entries are below “o=Your Company” for example

dn: cn=ibmuser, o=Your Company

To support the CA, I added

suffix “c=GB” to the configuration file and restarted LDAP.

I created the elements for the CA name by using an .ldif file with.

dn: c=GB
objectclass: country
c: GB

dn: o=SSS, c=GB
o: SSS
objectclass: top
objectclass: organization

dn: ou=CA, o=SSS, c=GB
ou: CA

and the CA specific CRL information

dn: cn=SSCA256, ou=CA, o=SSS, c=GB
cn: SSCA256
objectclass: cRLDistributionPoint
certificateRevocationList;binary:< file:///crl1.ldif

This page says the z/OS LDAP client LDIF parser does not support the file:// URL format. 

I modified the ldif to be

dn: CN=SSCA256,OU=CA,O=SSS,C=GB 
cn: SSCA256 
objectclass: cRLDistributionPoint 
objectclass: certificationAuthority 
cacertificate:: 
 MIIB2DCCAX+gAwIBAgI
...
certificateRevocationList:: 
 MIIBRTCB7QIBATA
...

Where the data contents are indented one character.

I copied the contents .pem file from the CA, and the content of the CRL.pem file into the .ldif file.

I used ldapmodify… to update LDAP with the .ldif files.

I tried using the revoked certificate on Linux to issue an LDAPsearch – and it worked! (where it should have failed).

The gsktrace had

Checking revoked status CN=ecec,O=cpwebuser,C=GB

CRL cannot be located

More configuration on z/OS

The CRL processing works as follows

• I use a certificate to connect to LDAP.
• LDAP calls GSKit to verify the certificate, map it to a RACF userid etc.
• If GSKIT has been configured for CRL checking, it issue a request to the CRL server to get the CRL for the CA. This could be my LDAP, or another LDAP somewhere else. It took a little while to work out, my client came in to LDAP, called GSKIT, which called the same LDAP!

Configure GSKIT

In the LDAP environment file I added

GSK_LDAP_SERVER=127.0.0.1
GSK_LDAP_PORT=389
GSK_LDAP_USER=cn=ldapid,o=Your Company
GSK_LDAP_PASSWORD=passwr0d

This does a query asking for the CA certificate, and the CRL data for my Certificate Authority’s DN.

This did not work, I got the message in the LDAP job log

GLD1116E Unable to initialize an SSL connection 8 – Certificate validation error.

In the GSK trace I had

ERROR check_crl_issuer_extensions(): crlSign bit is not set in KeyUsage
ERROR check_revoked(): Unable to verify CRL issuer extensions: Error 0x03353026

My Linux CA was not defined properly. I was missing cRLSign in my CA definition

keyUsage = keyCertSign, digitalSignature,cRLSign

I did the following

• Recreated my CA certificate
• Sent the updated CA to my backend systems.
• Update the certificate on z/OS and added it to the keyrings
• Recreated the user certificate on Linux
• Tried the ldapsearch to z/OS

To my pleasant surprise this failed to work – as I hoped.

My Ubuntu client had

ldap_sasl_interactive_bind_s: Can’t contact LDAP server (-1)
additional info: A TLS fatal alert has been received.

and there was a message in the LDAP joblog

GLD1116E Unable to initialize an SSL connection with 10.1.0.2: 431 – Certificate is revoked.

In my gsktrace file I had

ERROR check_revoked(): RFC 2459 Revoked certificate check failed: Error 0x03353041

and from 03353041 you get

03353041 Certificate is revoked.

Explanation: A certificate is revoked and cannot be used.
User response: Obtain a new certificate from the certification authority.

You have not finished yet!

Now you know the process, of getting a CRL from the CA, and updating LDAP with it you need to consider some more things.

• You need automation to generate the CRL, and update LDAP.
• You may want multiple LDAP servers for your organisation.
• You should consider a DN userid for your GSK, which can only access the CA information, and no other, so even if people saw the userid and password, they could not do much with it.
• You need to protect the data, so the default access is none, but your GSK DN can read it, and you automatic process can update it.

Hindsight.

Since I got it working, I realised that I should have a better name for my CAs, such as O= SSS, OU=CA, c=GB, CN = SSCA256 instead of C = GB, O = SSS, OU = CA, CN = SSCA256.

It is worth going through the whole processed to find out the problems, so you minimise the amount of rework. For example missing keyUsage = cRLSign, caused a lot of rework (several times till I got it right).