Some of the mysteries of Java shared classes

When Java executes a program it read in the jar file, breaks it into the individual classes, converts the byte codes into instructions, and when executing it may replace instructions with more efficient instructions (Jitting). It can also convert the byte codes into instructions ahead of time, so called Ahead Of Time (AOT) compilation.

With shared classes, the converted byte codes, any Jitted code, and any AOT code can be saved in a data space.

  • When the java program runs a second time, it can reuse the data in the dataspace, avoid the overhead of the reading the jar file from the file system, and coverting the byte codes into instructions.
  • The data space can be hardened to a file, and restored to a data space, so can be used across system IPLs.

Using this, it reduced the start-up time of my program by over 20 seconds on my slow zPDT system. The default size of the cache is 16MB – one of my applications needed 100 MB, so most of the benefits of he shared classes could not be exploited if the defaults were used.

This blog post describes more information about this, and what tuning you can do.

Issuing commands to manage the shared classes cache

Commands to manage the shared classes cache are issued like

java -Xshareclasses:cacheDir=/tmp,name=client6,printStats

which can be done using JCL

// SET V=’listAllCaches’
// SET V=’printStats’
// SET C=’/tmp’
// SET N=’client6′
//S1 EXEC PGM=BPXBATCH,REGION=0M,
// PARM=’SH java -Xshareclasses:cacheDir=&C,name=&N,verbose,&V’
//STDERR DD SYSOUT=*
//STDOUT DD SYSOUT=*

Enabling share classes

You specify -Xsharedclasses information as a parameter to the program, for example in the command line or in a jvm properties file.

To use the shared classes capability you have to specify all of the parameters on one line, like

-Xshareclasses:verbose,name=client6,cacheDirPerm=0777,cacheDir=/tmp

Having it like

-Xshareclasses:name=client6,,cacheDirPerm=0777,cacheDir=/tmp
-Xshareclass:verbose

means the name, etc all take their defaults. Only shareclass:verbose would be used.

Changing share classes parameters

You can have more than one cache; you specify a name. You specify a directory were an image is stored when the cache is hardened to disk.

Some of the options like name= and cacheDir= are picked up when the JVM starts, Other parameters like cacheDirPerm are only used when the cache is (re-)created.

You can delete the cache in two ways.

Delete the cache from your your Java program

When you are playing around, you can add reset to the end of the -Xshareclasses string to caused the cache to be deleted and recreated.This gives output like

JVMSHRC010I Shared cache “client6” is destroyed
JVMSHRC158I Created shared class cache “client6”
JVMSHRC166I Attached to cache “client6”, size=20971328 bytes

This was especially useful when tuning the storage allocations.

Delete the cache independently

java -Xshareclasses:cacheDir=/tmp,name=client6,destroy

How to allocate the size of the cache

You specify the storage allocations using -Xsc.. (where sc stands for shareclasses)

If you have -Xsharedcache:verbose… specified then when the JVM shuts down you get

JVMSHRC168I Total shared class bytes read=11660. Total bytes stored=5815522
JVMSHRC818I Total unstored bytes due to the setting of shared cache soft max is 0.
Unstored AOT bytes due to the setting of -Xscmaxaot is 1139078.
Unstored JIT bytes due to the setting of -Xscmaxjitdata is 131832.

This shows the values of maxaot and maxjitdata are too small they were

-Xscmx20m
-Xscmaxaot2k
-Xscmaxjitdata2k

Whem the values were big enough I got

JVMSHRC168I Total shared class bytes read=12960204. Total bytes stored=8885038
JVMSHRC818I Total unstored bytes due to the setting of shared cache soft max is 0.
Unstored AOT bytes due to the setting of -Xscmaxaot is 0.
Unstored JIT bytes due to the setting of -Xscmaxjitdata is 0.

How big a cache do I need?

If you use -Xshareclasses:verbose… it will display messages

for example

JVMSHRC166I Attached to cache “client6”, size=2096960 bytes
JVMSHRC269I The system does not support memory page protection

JVMSHRC096I Shared cache “client6” is full. Use -Xscmx to set cache size.
JVMSHRC168I Total shared class bytes read=77208. Total bytes stored=2038042

Message JVMSHRC096I Shared cache “client6” is full. Use -Xscmx to set cache size, tells you the cache is full – but no information about how big it needs to be.

You can use

java -Xshareclasses:cacheDir=/tmp,name=client6,printStats

to display statistics like

-Xshareclasses persistent cache disabled]                                         
[-Xshareclasses verbose output enabled]                                            
JVMSHRC159I Opened shared class cache "client6"                                    
JVMSHRC166I Attached to cache "client6", size=2096960 bytes                        
JVMSHRC269I The system does not support memory page protection                     
JVMSHRC096I Shared cache "client6" is full. Use -Xscmx to set cache size.          
                                                                                   
Current statistics for cache "client6": 
cache size                           = 2096592                       
softmx bytes                         = 2096592                       
free bytes                           = 0                             
ROMClass bytes                       = 766804                        
AOT bytes                            = 6992                          
Reserved space for AOT bytes         = -1                            
Maximum space for AOT bytes          = 1048576                       
JIT data bytes                       = 212                           
Reserved space for JIT data bytes    = -1                            
Maximum space for JIT data bytes     = 1048576                       
Zip cache bytes                      = 1131864                       
Startup hint bytes                   = 0                             
Data bytes                           = 13904                         
Metadata bytes                       = 12976                         
Metadata % used                      = 0%                            
Class debug area size                = 163840                        
Class debug area used bytes          = 119194                        
Class debug area % used              = 72%

Cache is 100% full  
                                                                             

This show the cache is 100% full, and how much space is used for AOT and JIT. The default value of -Xscmx I had was almost 16MB. I made it 200MB and this was large enough.

I could not find a way of getting my program to issue printStats.

How do I harden the cache?

You can use use the

java -Xshareclasses:cacheDir=/tmp,name=zosmf,verbose,snapshotCache

command to create the cache on disk. Afterwards the listAllCaches command gave

Cache name level        cache-type     feature 
client6    Java8 64-bit non-persistent cr
client6    Java8 64-bit snapshot       cr

Showing the non persistent data space, and the snapshot file.

You can use the restoreFromSnapshot to restore from the file to the data cache; before you start your Java program. You would typically do this after an IPL.

How can I tell what is going on and if shared classes is being used?

The java options “-verbose:dynload,class

reports on the

  • dynamic loading of the files, and processing them,
  • what classes are being processed.

For example

<Loaded java/lang/reflect/AnnotatedElement from /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar>
< Class size 3416; ROM size 2672; debug size 0>
< Read time 1196 usec; Load time 330 usec; Translate time 1541 usec>
class load: java/lang/reflect/AnnotatedElement from: /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar
class load: java/lang/reflect/GenericDeclaration from: /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar

dynload gave

<Loaded java/lang/reflect/AnnotatedElement from /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar>
< Class size 3416; ROM size 2672; debug size 0>
< Read time 1196 usec; Load time 330 usec; Translate time 1541 usec>

this tells you a jar file was read from the file system, and how long it took to process it.

class gave

class load: java/lang/reflect/AnnotatedElement from: /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar
class load: java/lang/reflect/GenericDeclaration from: /Z24A/usr/lpp/java/J8.0_64/lib/rt.jar

This shows two classe were extracted from the jar file.

In a perfect system you will get the class load entries, but not <Loaded….

Even when I had a very large cache size, I still got dynload entries. These tended to be loading class files rather than jar files.

For example there was a dynload entry for com/ibm/tcp/ipsec/CaApplicationProperties. This was file /usr/lpp/zosmf./installableApps/izuCA.ear/izuCA.war/WEB-INF/classes/com/ibm/tcp/ipsec/CaApplicationProperties.class

If you can make these into a .jar file you may get better performance. (But you may not get better performance, as it may take more time to load a large jar file).

I also noticed that there was dynload for com/ibm/xml/crypto/IBMXMLCryptoProvider which is in /Z24A/usr/lpp/java/J8.0_64/lib/ext/ibmxmlcrypto.jar, so shared classes has some deeper mysteries!

What happens if the .jar file changes?

As part of the class load, it checks the signature of the file on disk, matches the signature on the data space. If they are different the data space will be updated.

Using jConsole and with z/OS Liberty web server

I wanted to get some monitoring information out from z/OSMF using jConsole on my Ubuntu machine. Eventually this worked, but I had a few problems on the way. The same technique can be used for base Liberty, MQWeb, z/OSMF and ZOWE all of which are based on Liberty.

Configuring z/OSMF

I changed the z/OSMF configuration to include

<featureManager> 
  <feature>restConnector-2.0</feature> 
</featureManager> 

and restarted the server.

In the stdout (or message log) will be something like

CWWKX0103I: The JMX REST connector is running and is available at the following service
URL: service:jmx:rest://sss.com:10443/IBMJMXConnectorREST

You need the URL. The message above gave service:jmx:rest://sss.com:10443/IBMJMXConnectorREST, but I needed to use service:jmx:rest://10.1.1.2:10443/IBMJMXConnectorREST .

The port number came from the httpEndpoint with id=”defaultHttpEndpoint” . I have another httpEndpoint with port 24993, and this also worked with jConsole.

Set up jConsole

I set up a script for jConsole

k1='-J-Djavax.net.ssl.keyStore=/home/colinpaice/ssl/ssl2/adcdc.p12'
k2='-J-Djavax.net.ssl.keyStorePassword=password'
k3='-J-Djavax.net.ssl.keyStoreType=pkcs12'
t1='-J-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/zca.jks'
t2='-J-Djavax.net.ssl.trustStorePassword=password'
t3='-J-Djavax.net.ssl.trustStoreType=jks'
d='-J-Djavax.net.debug=ssl:handshake'
d=' '
de='-debug'
de=' '
s='service:jmx:rest://10.1.1.2:10443/IBMJMXConnectorREST'
jconsole $de $s $k1 $k2 $k3 $t1 $t2 $t3 $d

Where

  • the -J .. parameters are passed through to java,
  • the -Djava… are the standard set of parameters to define the key stores on the Linux

Running this script gave a pop up window with

Secure connection failed. Retry insecurely?

The connection to service:jmxLret://10.1.1.2:10443/IBMJMXConnectorREST could not be made using SSL.

Would you like to try without SSL?

This was because of the exception

java.io.IOException: jmx.remote.credentials not provided.

I could not see how to pass userid and password to jConsole.

I then used Cntrl+N to create a new connection and entered Username: and Password: which jConsole requires. After a short delay of a few seconds jConsole responded with a graphs of Heap Memory Usage, and Threads in use. You can then select from the Measurement Beans.

The TLS setup

In the keystore I had a certificate which I had used to talk to a Liberty instance before.

This was signed, and the CA certificate had been imported into the key trust keyring on z/OS, for that HttpEndPoint.

The server responded with a server certificate (“CN=SERVER,O=SSS,C=GB”) which had been signed on z/OS. The signing certificate had been exported from z/OS and downloaded to Linux

I created a jks key trust store using this certificate, using the command

keytool -importcert -file temp4ca.pem -keystore zca.jks -storetype jks -storepass password

and used this trust store to validate the server certificate sent down from z/OS.

This worked with jConsole.

I created a pkcs12 keystore using keytool

keytool -importcert -file temp4ca.pem -keystore zca2.p12 -storetype pkcs12 -storepass password

Which also worked.

Problems using a .p12 trust store

I used

runmqakm -keydb -create -db zca.p12 -type pkcs12 -pw password
runmqakm -cert -add -file temp4ca.pem -db zca.p12 -type pkcs12 -pw password -label tempca
runmqakm -cert -details -db zca.p12 -type pkcs12 -pw password -label tempca

to create a pkcs12 keystore and import the z/OS CA certificate. The -details option displayed it.

When I tried to use it, jConsole produced the message (after the Cntl+N)

Secure connection failed. Retry insecurely?

The connection to service:jmxLret://10.1.1.2:10443/IBMJMXConnectorREST could not be made using SSL.

Would you like to try without SSL?

I used Ctrl-N as before, and got the same message.

Using

d=’-J-Djavax.net.debug=ssl:handshake’

and rerunning the script, produced a TLS trace. At the bottom was

VMPanel.connect, handling exception: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

%% Invalidated:…
VMPanel.connect, SEND TLSv1.2 ALERT: fatal, description = internal_error

Using a trace at the server, gave the unhelpful , SEND TLSv1.2 ALERT:

Using openssl also failed. Create the .p12 keystore

openssl pkcs12 -export -out zca.p12 -in temp4ca.pem -name zCA -nokeys

and rerun the jconsole script, and it failed the same way.

Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

It looks like runmqakm and openssl do not create a valid trust store with an imported certificate.

Additional diagnostics

When the trust store created by keytool was used; at the top of the TLS trace output was

System property jdk.tls.client.cipherSuites is set to ‘null’

Ignoring disabled cipher suite: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
trustStore is: /home/colinpaice/ssl/ssl2/zca.p12
trustStore type is : pkcs12

trustStore provider is :
init truststore
adding as trusted cert:
Subject: CN=TEMP4Certification Authority, OU=TEST, O=TEMP
Issuer: CN=TEMP4Certification Authority, OU=TEST, O=TEMP
Algorithm: RSA; Serial number: 0x0
Valid from Tue Jul 14 00:00:00 BST 2020 until Fri Jul 02 23:59:59 BST 2021

keyStore is : /home/colinpaice/ssl/ssl2/adcdc.p12
keyStore type is : pkcs12
keyStore provider is :
init keystore
init keymanager of type SunX509

When the runmqakm or openssl was used, the green entries were missing.

When I used runmqakm to create the pkcs12 keystore

runmqakm -cert -details -db zca.p12 -type p12 -pw password -label tempca

listed the certificate successfully.

When I used keytool to list the contents

keytool -list -keystore zca.p12 -storetype pkcs12 -storepass password
Keystore type: PKCS12
Keystore provider: SunJSSE

Your keystore contains 0 entries

When I created the key store with keytool, both runmqakm and keytool displayed the certificate.

The problem looks like Java is only able to process the imported CA certificates when keytool was used to create the trust store.

How to restrict what certificates and algorithms clients can use to connect to java web servers

As part of your regular housekeeping you want to limit connections to your web server from weak keys and algorithms.   Making changes to the TLS configuration could be dangerous, as there is no “warning mode” or statistics to tell you if weak algorithms etc are being used.  You have to make a change and be prepared to have problems.

In this posting I’ll explain how to do it, then explain some of the details behind it.

How to restrict what certificates and algorithms can be used by web servers and java programs doing TLS.

One way which does not work.

The jvm.options file provided by mqweb includes commented out

-Djdk.tls.disabledAlgorithms=… and  -Djdk.tls.disabledAlgorithms=…..

These is the wrong way of specifying information, as you do it via the java.security file, not -D… .

Create an mqweb specific private disabled algorithm file

Java uses a java.security file to define security properties.

On my Ubuntu, this file if in /usr/lib/jvm/…/jre/lib/security/java.security  .

Create a file mqweb.java.security.  It can go anywhere – you pass the name using a java system property.

Copy  from the java.security file to your file, the lines with  with jdk.tls.disabledAlgorithms=..  and jdk.certpath.disabledAlgorithms=… . 

On my system, the lines are (but your security people may have changed them – if so,  you might want to talk to them before making any changes)

jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 1024,     EC keySize < 224, 3DES_EDE_CBC, anon, NULL

jdk.certpath.disabledAlgorithms=jdk.certpath.disabledAlgorithms=MD2, MD5,    SHA1 jdkCA & usage TLSServer,    RSA keySize < 1024, DSA keySize < 1024, EC keySize < 224

The jvm.options file provided by IBM has

-Djdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, MD5withRSA, DH keySize < 768, 3DES_EDE_CBC, DESede, EC keySize < 224, SHA1 jdkCA & usage TLSServer

So you may want to add this as in your override file ( without the -D), so add “, SHA1 jdkCA & usage TLSServer” to  jdk.certpath.disabledAlgorithms .

Tell mqweb to use this file

Create a java system property in the mqweb jvm.options file

“-Djava.security.properties=/home/colinpaice/eclipse-workspace-C/sslJava/bin/serverdisabled.properties”

Restart your web server.  You have not changed anything – just copied some definitions into an mqweb specific file, so it should work as before.

Limit what can be used

I set up several certificates with combination of RSA and Elliptic Curves, varying keysize, signatures;  and signed with CAs with RSA, and Elliptic Curve, and different signatures.

For example RSA4096,SHA256withECDSA,/EC256,SHA384with ECDSA means

  • RSA4096 certificate is RSA with a key size of 4096
  • SHA256withECDSA signed with this
  • /EC256 the CA has a public key of EC 256
  • SHA384with ECDSA and the CA was signed with this

I then specified different options in the servers’ file, and recorded if they TLS connection worked or not; if not – why not.

certpath: RSA keySize <= 2048

Server EC407,   SHA256withRSA,   /RSA4096,   SHA512withRSA

  • ✅RSA4096,SHA256withECDSA,   /EC256,SW=SHA384with ECDSA
  • RSA2048, SHA256withRSA,      /RSA4096,/SHA512withRSA
  • ✅ EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA
  • ✅EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

certpath: RSA keySize <= 4096

Server EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA

  • RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • RSA2048,SHA256withRSA,     /RSA4096,  SHA512withRSA
  • ❌EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA
  • ✅EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

certpath:EC keySize <= 256

Server EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA

  • ❌  RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • ✅ RSA2048,SHA256withRSA,       /RSA4096,  SHA512withRSA
  • ✅ EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA
  • ❌ EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

tls:EC keySize <= 256

Server EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA

  • ✅ RSA4096,SHA256withECDSA,   /EC256,       SHA384with ECDSA
  • ✅ RSA2048,SHA256withRSA,        /RSA4096,  SHA512withRSA
  • ✅ EC407,      SHA256withRSA,       /RSA4096, SHA512withRSA
  • ✅ EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

certpath: SHA256withRSA

Server EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA

  • ✅ RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • ❌RSA2048,SHA256withRSA,     /RSA4096,  SHA512withRSA
  • ❌ EC407,      SHA256withRSA,    /RSA4096, SHA512withRSA
  • ✅ EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

tls:SHA256withRSA

Server EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA

  • ✅ RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • ❌RSA2048,SHA256withRSA,     /RSA4096,  SHA512withRSA
  • ❌ EC407,      SHA256withRSA,    /RSA4096, SHA512withRSA
  • ✅ EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

either: RSA

Server RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA

All requests failed due to the server’s RSA.   Only 18 out of 50 cipher suites were available.  Server reported javax.net.ssl.SSLHandshakeException: no cipher suites in common

  • ❌RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • ❌ RSA2048,SHA256withRSA,     /RSA4096,  SHA512withRSA
  • ❌ EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA
  • ❌EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

certpath: RSA keySize == 4096

Server RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA

This was a surprise as I did not think this would work!

  • RSA4096,SHA256withECDSA, /EC256,       SHA384with ECDSA
  • RSA2048,SHA256withRSA,     /RSA4096,  SHA512withRSA
  • ❌EC407,      SHA256withRSA,      /RSA4096, SHA512withRSA
  • ✅EC384,       SHA256withECDSA, /EC256,      SHA384withECDSA

Summary of overriding.

You can specify restrictions in the server’s jdk.certpath.disabledAlgorithms and jdk.certpath.disabledAlgorithms. The restrictions apply to the how the certificate has been signed and the CA certificate.

You should check that the server’s certificate is not affected.

More details and what happens under the covers

The section below may be too much information, unless you are trying to work out why something is not working.

In theory jdk.tls.disabledAlgorithms and jdk.certpath.disabledAlgorithms are used for different areas of checking – reading certificates from key files, and what is passed during the handshake – but this does not seem to be true.  I found that it was best to put restrictions on both lines.

A certificate is of type RSA, EC, or DSA.

A certificate is signed for example Signature Algorithm: SHA256withECDSA.   This comes from the CA which signed it, message digest SHA256, and the CA is an Elliptic Curve.  See How do I create a certificate with Elliptic Curve (or RSA).

Signature Algorithms: is a combination of Hash Algorithm and Signature Type.   There are 6 hash algorithms: md5, sha1, sha224, sha256, sha384, sha512, and three types:  rsa, dsa, ecdsa.    These can be combined to to give 14 combinations of Signature Algorithms used in TLSv1.2

You can use java.security to control what TLS does.  On my Ubuntu this file /usr/lib/jvm/java-8-oracle/jre/lib/security/java.security.

This includes

  • jdk.certpath.disabledAlgorithms: Algorithm restrictions for certification path (CertPath) processing:  In some environments, certain algorithms or key lengths may be undesirable for certification path building and validation. For example, “MD2” is generally no longer considered to be a secure hash algorithm. This section describes the mechanism for disabling algorithms based on algorithm name
    and/or key length. This includes algorithms used in certificates, as well as revocation information such as CRLs and signed OCSP Responses.
  • jdk.tls.disabledAlgorithms: Algorithm restrictions for Secure Socket Layer/Transport Layer Security  (SSL/TLS) processing.  In some environments, certain algorithms or key lengths may be undesirable when using SSL/TLS. This section describes the mechanism for disabling algorithms during SSL/TLS security parameters negotiation, including protocol version negotiation, cipher suites selection, peer authentication and key exchange mechanisms.

I found you get better diagnostics if you put the restrictions on both statements.

The TLS Handshake (relating to java.security)

Server starts  up

  • I had 50 available cipher suites
  • Using -Djdk.tls.server.cipherSuites=…,… you can specify a comma separated list of  which cipher suites you want make available.  I recommend you do not specify this and use the defaults.
  • Using jdk.tls.disabledAlgorithm you can specify which handshake information is not allowed.  For example
    •  Any of the following would stop cipher suite TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 from being used
      • java.security.tls = AES_256_CBC
      • java.security.tls= SHA384 – this loses 4 certificate TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 etc
      • java.security.tls=TLS_ECDHE_ECDSA
    • Elliptic curves names Extension elliptic_curves, curve names: {secp256r1, secp384r1, secp521r1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1, secp256k1}.  Specifying EC keySize <= 521 would only allow {sect571k1, sect571r1} to be used
  • The server builds a supported list of cipher suites, signature algorithm, and elliptic curve names

Client starts up

  • As with the server, the client builds up a list of supported cipher suites, signature algorithms, and supported Elliptic curve names.
  • The client sends “ClientHello” and the list to the server.

Server processes

  • The server takes this list and iterates over it  to find the first acceptable certificate ( or for wlp, if <ssl … serverKeyAlias=”…” />, then the specified aliases  is used )
    • if the cipher suite name is like  TLS_AAA_BBB_WITH…   AAA must match the servers certificate type ( RSA, Elliptic Curve, DSA)
    • the signature algorithm (BBB).   This is the algorithm for encrypting the payload, and the algorithm for calculating the hash of the payload
    • if the certificate is EC,  check the  elliptic curve name is valid.  A  server’s certificate created with openssl ecparam -name prime256v1, would be blocked if EC keySize <= 256 was specified in the client resulting.
  • If no certificate was found in the trust store which passed all of the checks, it throws javax.net.ssl.SSLHandshakeException: no cipher suites in common , and closes the connection
  • The server sends “ServerHello” and the server’s public key to the client.
  • The server sends the types of certificate it will accept.   This is typically RSA, DSA, and EC
  • The server sends down the Elliptic Curve names it will accept, if present  – I dont think it is used on the java client
  • The server uses the jdk.certpath.disabledAlgorithm to filter the list of Signature Algorithms, and sends this filtered list to the client.
  • The server extracts the CAs and self signed certificates from the trust store and sends them down to the client.
  • The server sends “ServerHelloDone”, saying over to you to respond.

Client processing:

  • Checks the server’s certificate is valid, including
    • Checks the public certificate of the servers CA chain is allowed according to the client’s jdk.certpath.disabledAlgorithm..  So jdk.certpath.disabledAlgorithm =…, SHA256withECDSA would not allow a  server’s certificate with Signature algorithm:SHA256withECDSA .
  • The client takes the list of certificate types ( RSA, DSA, EC), and CAs and iterates over the keystore and selects the records where
    • the certificate type in the list
    • the signature algorithm is in the list
    • the certificate signed by one of the CA’s in the list
  • Displays this list for the end user to select from.  It looks like the most recently added certificate is first in the list.
  • The client sends “Finished” and  the selected certificate to the server

Server processing

  • The server checks the certificate and any imbeded CA certificates from the client matches.
    • Checks the signature algorithm
    • Checks the constraints, for example RSA keySize < 2048
    • Checks the certificate and CA are valid

End of handshake.

This is a useful link for describing the java.security parameters.

This specification  describes the handshake, with the “ClientHello” etc.

Using java -Djavax.net.debug=… to examine data flows, including TLS

Jan 2021. This page gets many hits. Please can you tell me if the information below is what you were looking for, or if you were looking for more information, such as understanding the trace flow, and finding problems.   Please send me an email or post a comment below with information on what you were looking for, and I’ll see what I can do.

There are different levels you can get from the -Djavax.net.debug option.  You can display its options using -Djavax.net.debug=help .

Ive added a section on -Djava.security.debug=… which looks at keystore access.

Ive also added a examples of what each trace option give you.

I’ve also put up a post which describes the trace, and has an annotated output (from both ends), showing common problems and possible solutions.

With -Djavax.net.debug you get

all            turn on all debugging
ssl            turn on ssl debugging

The following can be used with ssl:

    record       enable per-record tracing
    handshake    print each handshake message
    keygen       print key generation data
    session      print session activity
    defaultctx   print default SSL initialization
    sslctx       print SSLContext tracing
    sessioncache print session cache tracing
    keymanager   print key manager tracing
    trustmanager print trust manager tracing
    pluggability print pluggability tracing

    handshake debugging can be widened with:
    data         hex dump of each handshake message
    verbose      verbose handshake message printing

    record debugging can be widened with:
    plaintext    hex dump of record plaintext
    packet       print raw SSL/TLS packets

and your program exits.

You can use

-Djavax.net.debug=ssl:record or -Djavax.net.debug=ssl:handshake

To display specific levels of detail.

To specify more than one option use -Djavax.net.debug=ssl:keymanager:record

-Djava.security.debug=

Out of all the options below, I found certpath,provider provided the most useful information for seeing what was happening with certificates and the handshake.  For example

certpath: X509CertSelector.match(SN: 1a
Issuer: CN=TEMP4Certification Authority, OU=TEST, O=TEMP
Subject: CN=SERVER, OU=SSS, O=ZZZZ)
certpath: X509CertSelector.match: subject DNs don’t match
certpath: ForwardBuilder.getMatchingCACerts: found 0 CA certs
certpath: SunCertPathBuilder.depthFirstSearchForward(): certs.size=0
certpath: PKIXCertPathValidator.engineValidate()…
certpath: AdaptableX509CertSelector.match: subject key IDs don’t match. Expected: [] Cert’s: []
certpath: NO – don’t try this trustedCert
certpath: X509CertSelector.match(SN: 0
Issuer: CN=TEMP4Certification Authority, OU=TEST, O=TEMP
Subject: CN=TEMP4Certification Authority, OU=TEST, O=TEMP)
certpath: X509CertSelector.match returning: true
certpath: YES – try this trustedCert
certpath: anchor.getTrustedCert().getSubjectX500Principal() = CN=TEMP4Certification Authority, OU=TEST, O=TEMP
certpath: AlgorithmChecker.contains: SHA256withRSA
Provider: MessageDigest.SHA-256 algorithm from: SUN

I grepped with pkcs12: and got

pkcs12: Loading PKCS#7 encryptedData (PBEWithSHA1AndRC2_40 iterations: 2048)
pkcs12: Loading PKCS#7 data
pkcs12: Checking keystore integrity (HmacPBESHA1 iterations: 2048)
pkcs12: PKCS12KeyStore load: private key count: 1. secret key count: 0. certificate count: 1
pkcs12: Retrieved a protected private key at alias ‘ss’ (PBEWithSHA1AndDESede iterations: 2048)
pkcs12: Retrieved a 1-certificate chain at alias ‘ss’

The options…


all              turn on all debugging - this produces millions of lines of
                 output for one handshake! 
access           print all checkPermission results 
certpath        PKIX CertPathBuilder and CertPathValidator debugging 
combiner         SubjectDomainCombiner debugging 
gssloginconfig   GSS LoginConfigImpl debugging 
configfile       JAAS ConfigFile loading 
configparser     JAAS ConfigFile parsing 
jar              jar verification 
logincontext     login context results 
jca              JCA engine class debugging 
policy           loading and granting 
provider        security provider debugging 
pkcs11           PKCS11 session manager debugging 
pkcs11keystore   PKCS11 KeyStore debugging 
sunpkcs11        SunPKCS11 provider debugging 
scl              permissions SecureClassLoader assigns 
ts               timestamping 

The following can be used with access: 

stack            include stack trace 
domain           dump all domains in context 
failure          before throwing exception, dump stack and domain that 
                 didn't have permission 
thread           include the thread name 

The following can be used with stack and domain: 

permission=<classname> 
                only dump output if specified permission is being checked 
codebase=<URL>  only dump output if specified codebase  is being checked 
permname=<name> only dump output if the specified name matches the 
                permission getName() 
permactions=<actions>  only dump output if the specified actions 
                matches the permission getActions() 

The following can be used with provider: 

engine=<engines> only dump output for the specified list of JCA engines.  
                Supported values: 
                   Cipher, KeyAgreement, KeyGenerator, 
                   KeyPairGenerator, KeyStore, Mac, 
                   MessageDigest, SecureRandom, Signature. 

Note: Separate multiple options with a comma  

-Djavax.net.debug=ssl:record gave

records doing read and write
SSLSocketOutputRecord.java:241|WRITE: TLS12 handshake, length = 294
SSLSocketInputRecord.java:214|READ: TLSv1.2 handshake, length = 2948

-Djavax.net.debug=ssl:handshake gave

All of the sections.

-Djavax.net.debug=ssl:handshake:verbose gave

the same as ssl:handshake +

Signature algorithm, ed448, is not supported by the underlying providers
SignatureScheme.java:373|Ignore unsupported signature scheme: ed25519
SignatureScheme.java:373|Ignore unsupported signature scheme: ed448
SignatureScheme.java:393|Ignore disabled signature scheme: rsa_md5

-Djavax.net.debug=ssl:keygen gave

No records.

-Djavax.net.debug=ssl:session gave just

Session initialized:  Session(1611922238363|TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384)

-Djavax.net.debug=ssl:defaultctx gave

No records.

-Djavax.net.debug=ssl:sslctx gave

javax.net.ssl|DEBUG|01|main|2021-01-29 12:29:48.847 GMT|X509TrustManagerImpl.java:79|adding as trusted certificates (
  "certificate" : {
      "subject"            : "CN=DANCA1, OU=DAN, O=DAN",
...
  "certificate" : {
    "subject"            : "CN=SS, O=SSS, C=GB",
... 
  "certificate" : {
    "subject"            : "CN=ecec, O=cpwebuser, C=GB",
...
System property jdk.tls.client.cipherSuites is set to 'null'
System property jdk.tls.server.cipherSuites is set to 'null'
jdk.tls.keyLimits:  entry = AES/GCM/NoPadding KeyUpdate 2^37. AES/GCM/NOPADDING:KEYUPDATE = 137438953472
Ignore unsupported cipher suite: TLS_AES_256_GCM_SHA384
found key for : ecec (
  "certificate" : {
       "subject"            : "CN=ecec, O=cpwebuser, C=GB",
 
    "subject"            : "CN=SSCA256, OU=CA, O=SSS, C=GB",
  ...
found key for : mqweb (
  "certificate" : {...   
trigger seeding of SecureRandom
done seeding of SecureRandom
...
Produced ClientHello handshake message (
"ClientHello": {
...
Consuming ECDH ServerKeyExchange handshake message (
"ECDH ServerKeyExchange": {
  "parameters": {
    "named group": "secp256r1"
    "ecdh public": {
      0000: 04 5F 30 8E 64 45 55 C0   
      0010: 55 23 1B 69 CA FF 78 8D  
      0020: B5 59 72 BA B4 95 8A 4C  
      0030: 2B 37 55 3D F2 A5 3D 71  
      0040: 34                                               
    },
  },

-Djavax.net.debug=ssl:sessioncache gave

No records.

-Djavax.net.debug=ssl:keymanager gave

found key for : ecec (
“certificate” : {
“version”            : “v3”,

found key for : mqweb (
“certificate” : {

-Djavax.net.debug=ssl:trustmanager gave

adding as trusted certificates (
“certificate” : {

“subject” : “CN=mqweb8, O=cpwebuser, C=GB”,

“certificate” : {

“subject” : “CN=DANCA1, OU=DAN, O=DAN”,

Found trusted certificate (
“certificate” : {

“issuer” : “CN=DANCA1, OU=DAN, O=DAN”,

“subject” : “CN=DANCA1, OU=DAN, O=DAN”,

ss private or public key is not of EC algorithm

Where adding as …  is what the local trust store found, and found trusted is what as sent from the server as part of the certificateRequest.

-Djavax.net.debug=ssl:pluggability

No records.