Tag: JMS

MQ, JMS and Opentelemetry

I wanted to use Opentelemetry with MQ. One option is to use JMS, and the “code free” Otel support.

This code-free-support intercepts the send and receive methods, intercepts them, and sets up the Otel support (in MQ).

See here for information on the Java Otel instrumentation.

My code

I used the IBM MQ sample JMS programs JMSProducer and JMSConsumer.

export CLASSPATH=/opt/mqm/java/lib/com.ibm.mq.allclient.jar:$CLASSPATH
export CLASSPATH=/opt/mqm/samp/jms/samples:$CLASSPATH
export CLASSPATH=/home/colin/Downloads/opentelemetry-javaagent.jar:$CLASSPATH
export OTEL_JAVAAGENT_ENABLED=true
export OTEL_JAVAAGENT_LOGGING=simple
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_ENDPOINT=http://10.1.0.2:4317

java -javaagent:/home/colin/Downloads/opentelemetry-javaagent.jar
-Dotel.resource.attributes=service.name=ColinsJMS
-Dotel.traces.exporter=console
/opt/mqm/samp/jms/samples/JmsProducer.java \
-m MQPA -d RSERVER -l SYSTEM.DEF.SVRCONN -h 172.26.1.2 -p 2414

This produced

Jaeger output

Which shows the message passed through MQ and was got (by the server program).

Because the sample JMS program does not allow you to specify a ReplyToQueue manager name, the server cannot send the reply back.

I changed the JMSProducer to specify a replyToQueue, and pass this as part of the message.
I used the JMSConsumer program unchanged.

The updated application

export CLASSPATH=/opt/mqm/samp/jms/samples:$CLASSPATH
export CLASSPATH=/home/colin/Downloads/opentelemetry-javaagent.jar:$CLASSPATH
export OTEL_JAVAAGENT_ENABLED=true
 # export OTEL_JAVAAGENT_LOGGING=simple # default
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_LOGS_EXPORTER=console
export OTEL_METRICS_EXPORTER=console
export OTEL_EXPORTER_OTLP_ENDPOINT=http://10.1.0.2:4317	
java -javaagent:/home/colin/Downloads/opentelemetry-javaagent.jar \
     -Dotel.resource.attributes=service.name=ColinsJMSProducer \
     -Dotel.traces.exporter=console \
     /opt/mqm/samp/jms/samples/colinProducer.java   \
-m MQPA -d RSERVER -l SYSTEM.DEF.SVRCONN -h  172.26.1.2 -p 2414 -r CCOLIN

echo "===================================================="

java -javaagent:/home/colin/Downloads/opentelemetry-javaagent.jar \
     -Dotel.resource.attributes=service.name=ColinsJMSConsumer \
     -Dotel.traces.exporter=console \
     /opt/mqm/samp/jms/samples/JmsConsumer.java \
    -m MQPA -d CCOLIN -l SYSTEM.DEF.SVRCONN -h  172.26.1.2 -p 2414

When this ran, the Otel data displayed in Jaeger was

Comments on the output.

  • The chart shows
    • A message is put to an MQ remote queue called RSERVER on queue manager MQPA. This operation takes 20.2 milliseconds.
    • The message is got from a queue called CSQ9 (a transmission queue). This flows over the network to queue manager CSQ9.
    • The message is put to a queue CSERVER on queue manager CSQ9.
    • An application on CSQ9 gets the message, does some processing and puts a reply to queue CCOLIN ( on queue manager MQPA). You cannot see this program in the trace.
    • The mover on CSQ9 gets the message from the transmission queue YMQPA and sends it over the network
    • The mover on CSQ9 puts the message to queue CCOLIN
    • The above processing is very quick.
    • The putting JMS program ends, and starts the JMSConsumer program.
    • 3.2 seconds after the initial put, the JMSConsumer program gets the message (and the MQGET takes 3.6 ms). The start of the second Java program takes a long time (most of this 3.2 seconds)
  • There are no entries for the JMS programs themselves. There are only entries for the MQ on z/OS.
  • Although there were two JMS Java programs involved – the Jaeger output just shows the MQ processing

What next?

Although this has shown that you can easily add Otel processing to your JMS program, the output is lacking.

You can manually instrument your Java program, as described by the Opentelemetry documentation.

What’s the difference between an MQ Message and a JMS Message

I had problems using the MQI Interface  to create a message for a JMS program to receive.

To see what was in the JMS message,  I used a Java program using JMS to write a message, and used my trusty C program to display it.

I could see that there were message properties in the message

Property 0 name <mcd.Msd> value <jms_text>
Property 1 name <jms.Dst> value <queue:///JMSQ1>
Property 2 name <jms.Rto> value <queue:///JMSQ2>
Property 3 name <jms.Tms> value <1571902099742>
Property 4 name <jms.Dlv> value <2>

These are described here.

The mcd.Msd value is one of jms_none, jms_text, jms_bytes, jms_map, jms_stream, jms_object.   This depends on whether you use Message message, BytesMessage message etc to define your message type.  The jms program receiving the message may be expecting a particular type

The jms.Rto comes from the message.setJMSReplyTo(…).  It was set in the MQMD.ReplyToQ  as well as the message property.

It took me some time to find how to specify value such as for deliveryMode.  I found it here.  For example  message.setDeliveryMode(DeliveryMode.NON_PERSISTENT).   (This comes from javax.jms.DeliveryMode.NON_PERSISTENT,not a com.ibm…. file).

I converted my simple program from JMQI to JMS, in a couple of hours, and was surprised to find it used fewer lines of code than using the JMQI.   Of course I may find I omitted some work, such as error handling, but it seems to be working OK.

How do I make my MDB transactional?

I found from the application trace  that my MDB was doing MQGET, MQCMIT in the listener, and MQOPEN, MQPUT, MQCLOSE and no MQCMIT in my application.    Digging into this I found that the MQPUT was NO_SYNCPOINT, which was a surprise to me!

My application had session = connection.createSession(true, 1); // true = transactional. So I expected it to work.

The ejb-jar.xml had

enterprise-beans
  message-driven
    transaction-type Container
...
assembly-descriptor
  container-transaction
    trans-attribute NotSupported

I changed NotSupported to Required and it worked.

 

The application trace for the Listener part of the MDB gave me

Operation      CompCode MQRC HObj (ObjName) 
MQXF_XASTART            0000 -
MQXF_GET       MQCC_OK  0000    2 (JMSQ2 )
MQXF_XAEND              0000 -
MQXF_XAPREPARE          0000 -
MQXF_XACOMMIT           0000 -

The trace for the application part of the MDB gave me

Operation                    CompCode MQRC HObj (ObjName)
MQXF_XASTART                             0000         –
MQXF_OPEN             MQCC_OK   0000         2 (CP0000 )
MQXF_PUT                MQCC_OK   0000          2 (CP0000 )
MQXF_CLOSE           MQCC_OK   0000          2 (CP0000 )
MQXF_XAEND                                0000         –
MQXF_XAPREPARE                       0000 –
MQXF_XACOMMIT                        0000 –

and the put options had _SYNCPOINT.

I had read documentation saying that you needed to have XAConnectionFactory instead of ConnectionFactory.  I could not get this work,  but found it was not needed for JMS;  it may be needed for JDBC.