Avoiding the random cut and paste approach to configuring JMS in a liberty web server

Why these JMS blog posts?

I had a “quick” question from someone, “can I configure JMS to reduce CPU usage and improve performance?”. It was was wet Monday in Orkney (north of Scotland) and I thought I would spent an hour looking into it. A few weeks later, I am much wiser, and have some answers to the question. The Knowledge Center has a lot of information, mostly is useful, mostly accurate, some information is missing and some assumes that you are very familiar with the product.

I also found that that the Java people tend to use different words for familiar concepts, so I had to struggle with this different language.

Below are the blog posts I wrote on getting JMS working on Ubuntu 18.04 with MQ V9.

• Getting JMS samples working in batch on Ubuntu for people who cannot spell Java Massage Service.

• Getting JMS working in a WAS Liberty Web Server on Ubuntu for people who cannot spell Java Massage Service.

• Avoiding the random cut and paste approach to configuring JMS in a Liberty web server.

• JMS performance and tuning in a WAS Liberty environment

WAS Liberty configuration

• Liberty is a cut down IBM WebSphere Application Server (known as WAS)

• There are two messaging engines available to WAS

1. 1. WMQ which all programs can use ( batch, CICS IMS etc)

   2. a messaging engine within the WAS know as the SI bus.

I’ll focus on the WMQ version.

• A Liberty instance can connect to more than one queue manager, and can use local bindings or client attach.

• An application running in Liberty can connect to multiple queue managers. For example, it might create a JMS connection to QM1 to get a message, and then connect to QM2 to send a reply. It usually connects to just one queue manager.

• Different applications running in a Liberty instance can have different requirements, for example they may use different queue managers, or you want to limit how many application instances can connect to MQ at a time

• You can run a program by specifying a URL in the web browser

• You can have “triggering” within Liberty where there is a listening thread which gets messages from a queue, and runs an message-driven bean (MDB) instance on a new thread, passing it the message. . Typically these MDBs connect to MQ to put a reply.

• There is “glue” code between applications and the queue manager. This is provided by IBM and converts the Java requests and issues the request to the queue manager, and handles syncpoint etc. This is the MQ resource adapter (RA), which implements the Java Connector Architecture (JCA) specification.

• In days gone by you had

  • a jmsQueueConnectionFactory for generating a handle for working with queues,

  • a jmsTopicConnectionFactory for generating a handle for working with topics.

  • there is now a jmsConnectionFactory for working with either. They are not interchangable – I think jmsConnectionFactory is the way to go.

Programs that need a connection to MQ.

These applications

1. Are involved with a URL. It typically connects to MQ, receives (gets) a message from a queue, and sends(puts) a reply message.

2. or an MDB sending a reply back to the originator.

3. or the thread which monitors a queue and invokes the MDB applications

There are several ways of providing the information needed to connect to MQ

1. Provide all the information in the application. This is not very flexible, as you need to change constants, rebuild and redeploy the applications if you want to change the connection information

2. Provide all of the information in a configuration file (server.xml). To change the connection information, you change the configuration and restart the application. You do not need to rebuild, nor redeploy the application. Your application passes in a label, and says give me the information.

3. A mixture of the above.

Different applications may need a different connection profile

You may want different business applications to have a different connection profiles

You may want to specify

• Queue manager name

• TCP port number

• SSL information etc

• The maximum number of application instances that can get a connection to MQ – for a particular application. Different applications may have a different maximum.

Basic configuration

Page JMS Connection Factory (jmsConnectionFactory) describes the jmsConnectionFactory definition, but it is not very clear on how to specify things.
A sample jmsConnectionFactory definition is given below

<jmsConnectionFactory id="QMAJMS" jndiName="QMAJMS">
  <connectionManager maxPoolSize="21" > </connectionManager>
  <properties.wmqJms hostName="localhost"
    port="1414" transportType="CLIENT" 
    applicationName="fromjmsConnectionFactory"
  />
</jmsConnectionFactory>

Kindergarden basics.

<!– ….–> is a comment

For a “name” tag you either need

<name...>
.....
</name>

or <name… />

Where … is data relevant to that tag and ….. are other tags or is empty.

From the doc you can keyword=”value” on the jmsConnectionFactory tag.

<jmsConnectionFactory
id="QMAJMS"
jndiName="QMAJMS"
connectionManagerRef=”...”
containerAuthDataRef=”...”
recoveryAuthDataRef=”...”
>

From the doc, you can have tags between <jmsConnectionFactory …> and </jmsConnectionFactory> tags

• <connectionManager…/>

• <containerAuthData…/>

• <properties.wasJms…/>

• <properties.wmqJms…/>

• <recoveryAuthData…/>

• I found you can have <authData user=”…” password=”….”/> but this is not documented.

properties.wasJms are properties you specify if you are using the SI Bus, or internal queue manager, and properties.wmqJms if you are using MQSeries.

If you want to specify applicationName, this is a property on the <properties.wmqJms> and the <properties.wasJms>.  If you specify it else where, it will be ignored. So you cannot specify if on the <jmsConnectionFactory> tag and expect it be propogated to nested elements.  Make sure you specify the data on the correct tag.  You may not get told if you have misspelt it or added it to the wrong tag (as I frequently did).

Which data is used if there are multiple definitions

if you have multiple statements for the same definition, the last definition value is used. You generally do not get told of any differences.

<jmsConnectionFactory… >
<properties.wmqJms queueManager="QMB" transportType="CLIENT" 
   applicationName="Hello"/>
<properties.wmqJms queueManager="QMA" transportType="BINDINGS"/>
</jmsConnectionFactory>

The parameters used are

• queueManager=”QMA”

• transportType=”BINDINGS”

• applicationName=”Hello”

Similarly 

Will try to use bindings mode and queue manager QMC.

This behavior is not consistent, for examples

<jmsActivationSpec id="wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB"
connectionFactoryLookup="IVTCFB">
…
</jmsActivationSpec>
<jmsActivationSpec id="wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB"
connectionFactoryLookup="IVTCFA" >
... 
</jmsActivationSpec>

Reports the inconsistency and gives

[AUDIT ] CWWKG0102I: Found conflicting settings for 
wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB instance of 
jmsActivationSpec configuration.
Property connectionFactoryLookup has conflicting values:
Value IVTCFB is set in file:.../test/server.xml.
Value IVTCFA is set in file:...test/server.xml.
Property connectionFactoryLookup will be set to IVTCFA.

Server.xml pre-req information to run JMS applications

You need

<featureManager>
<!-- the next two features are needed to install 
the JMS and MDB code -->
<feature>wmqJmsClient-2.0</feature> 
<feature>mdb-3.2</feature> 
</featureManager>
<!-- point to the Resource Adapter- 
  specify the value for your system -->
<variable name="wmqJmsClient.rar.location" 
    value="/opt/mqm/java/lib/jca/wmq.jmsra.rar"/>
<!-- if you use BINDINGS transport you need to -->
<!-- tell the server where the MQ libraries are -->
<wmqJmsClient nativeLibraryPath="/opt/mqm/java/lib64"/>

Basic JMS configuration

Below is an example configuration to provide connection information for two applications, one to connect to QMA, the other to connect to QMB, both using binding mode.

<jmsConnectionFactory jndiName="jms/PAYROLLCF" id="samplej1>
<properties.wmqJms queueManager="QMA" 
  transportType="BINDINGS" "/>
</jmsConnectionFactory>

<jmsConnectionFactory jndiName="INQUIRYCF" id="samplej2>
<properties.wmqJms queueManager="QMB"
  transportType="BINDINGS" "/>
</jmsConnectionFactory>

The PAYROLL application issues a request for the JNDI “jms/PAYROLLCF” and will get a bindings connection to queue manager QMA.

The INQUIRY application issues a request for the JNDI “INQUIRYCF” and will get a bindings connection to queue manager QMB.

The properties.wmqJms says this is for an external queue manager. If you had specified. properties.wasJms… this would be for the SI bus queue manager within the WAS server.

You should specify a jmsConnectionFactory for each business application to provide application isolation otherwise they sharer connections.

This should be enough configuration for your applications to connect to MQ, but  quickly moving on to more advanced topics.

Specifying connection parameters

You may want to limit how many concurrent connections a jmsConnectionFactory can use. You specify this information using an <connectionManager.. > tag.

You can either put <connectionManager.. > inside

<jmsConnectionFactory jndiName="PAYROLLCF" id="samplej1>
<connectionManager maxPoolSize=50/>
<properties.wmqJms … />
</jmsConnectionFactory>

or create one outside of the tags, and point to it. This way a definition can be shared by multiple jmsConnectionFactorys.

<connectionManager id=”MYCM” maxPoolSize=50/>

<jmsConnectionFactory connectionManagerRef="MYCM"
     jndiName="IVTCF"...>
  <properties.wmqJms.../>
</jmsConnectionFactory>

<jmsConnectionFactory connectionManagerRef="MYCM" 
    jndiName="IVTCF2"...>
  <properties.wmqJms.../>
</jmsConnectionFactory>

Each jmsConnectionFactory will have its own pool of threads up to the specified maximum of 50.

Specifying default userid information

You can specify a default userid and password for the application You can specify it inline

For example

<jmsActivationSpec jndiName="PAYROLLCF" id="samplej1>
<authData user="colinpaice" password="ret1red"/>
<properties.wmqJms .../>
</<jmsActivationSpec>

The documentation describes containerAuthDataRef on the <jmsConnectionFactory…>

which points to a <authData id=..>
tag outside of the <jmsConnectionFactory..>, but I could not get it to work.

The connection factory pointed to by <properties.wmqJms connectionFactoryLookup=”QMAJMS”.. > can also has an authData, but this is not used for the jmsActivationSpec. It is used when applications use the jmsConnectionFactory – for example the WMQ_IVT sample.

You may have noticed that the password is in clear text! You can use

~/wlp/bin/securityUtility encode xxxxxxx

to encode xxxxxxxx to make it harder to read. For example

./securityUtility encode ret1red

gives

{xor}LTorbi06Ow==

and you use

<authData user=”me” password=”{xor}LTorbi06Ow==”/>

Note:
You have to do other configuration foruserid validationthis to work. For example using runmqsc

dis qmgr connauth    
   QMNAME(QMB) CONNAUTH(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) 
dis authinfo(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) CHCKLOCL   
   AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS)   
   AUTHTYPE(IDPWOS)      CHCKLOCL(REQUIRED)

Check you have CHCKLOCL(REQUIRED).

If the queue manager has been configured not to require userid and password then you can omit the <authdata…./>

How to specify a queue

If the applications are looking up the queue information, you pass a jndiName.

If you are specifying the queue for an MDB listener you specify the id, and refer to it.

<jmsQueue id="AAAA" jndiName="JIVTQueue">
<properties.wmqJms baseQueueName="IVTQueue"/>
</jmsQueue>

Common parameters are

• jndiName, this is what is used in the application to locate the queue definition

• id used by the EJB listener task (jmsActivationSpec)

• baseQueueName, this is the queue in the queue manager

• baseQueueManagerName, this is used to specify a remote queue manager name, so there will typically be transmission queue with this name. This is not the name of the queue manager you need to connect to.

As with the properties.wmqJms above, the last definition is used, so with

<properties.wmqJms baseQueueName="IVT3"/>
<properties.wmqJms baseQueueName="IVT2"/>

the queue IVT2 would be used.

Specifying a MDB listener.

You need to configure the listening task for the queue.

I thought I had configured this properly, but when I tried to test it by using an invalid queue manager – it continued to work! This is because of new functions in <feature>wmqJmsClient-2.0</feature>

I had copied and pasted

<jmsActivationSpec id="wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB"
     maxEndpoints=500 >
  <properties.wmqJms 
     destinationRef="AAAA"
      transportType="BINDINGS"
     queueManager="QMA"/>
  <authData id="auth1" user="colinpaice" password="ret1red"/>
</jmsActivationSpec>

The

• id=”wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB” is the identity of the EJB code, it is coded within the EJB itself. “a/b/c” is

  • a: wmq.jmsra.ivt is the ….ear file in dropins directory.

  • b: WMQ_IVT_MDB is the name of the jar file within the .ear file

  • c: WMQ_IVT_MDB is the name of the MDB within the jar file.

• destinationRef=”AAAA” points to a jmsQueue object described above.

• QueueManager is the queue manager that the listener connects to – QMA

• MaxEndpoints is optional and specifies the maximum number of threads that can be used for EJBs. The default is 500. You should specify the maximum number of MDBs you want to run for this Activation spec.

• destinationType you can specify if you want to get from a queue or subscribe to a topic.

When I changed this to use a different queue manager, it continued to work because the MDB had ConnectionFactoryLookup=”IVTCF” hidden away in the ejb-jar.xml file. To make it even more difficult to understand, parameters can also be coded in @annotations in the java code!

This says ignore the queue manager information specified in the <properties.wmqJms…> within the <jmsActivationSpec…> and use the information in the <jmsConnectionFactory jndiName=”IVTCF“…> definition to find the  queue manager configuration to use.

This creates its own pool of connections based on the information in the jmsConnectionFactory – it does not share the connection pool.

I tried to override it, and specified

<jmsActivationSpec id="wmq.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB" >
<properties.wmqJms connectionFactoryLookup="IVTCFBZ" …./>

where connection factory IVTCFBZ did not exist – but it still worked! ( Because connectionFactoryLookup property in the ejb-jar.xml file took precedence over the connectionFactoryLookup in the activation specification definition).

To be able to configure the jmsActivationSpec how I wanted it, I had to remove ConnectionFactoryLookup and DestinationLookup from the ejb-jar.xml file.

These are the steps I took

1. copy out the .ear file cp wmq_jmsra_ivt.ear ~/temp/

2. go to this file cd ~/temp

3. save a copy in case things go wrong cp wmq_jmsra_ivt.ear wmq_jmsra_ivt.ear.save

4. rename it mv wmq_jmsra_ivt.ear ccp.ear

5. extract the jar file jar -xvf ccp.ear WMQ_IVT_MDB.jar

6. extract the ejb-jar.xml jar -xvf WMQ_IVT_MDB.jar META-INF/ejb-jar.xml

7. edit file to remove the two activation-config-properties

<activation-config>
<activation-config-property>
<activation-config-property-name>DestinationLookup</activation-config-property-name>
<activation-config-property-value>IVTQueue</activation-config-property-value>
</activation-config-property>
<activation-config-property>
<activation-config-property-name>ConnectionFactoryLookup</activation-config-property-name>
<activation-config-property-value>IVTCF</activation-config-property-value>
</activation-config-property>
</activation-config> 

7. save the updates

8. update the jar file jar -uvf WMQ_IVT_MDB.jar META-INF/ejb-jar.xml

9. rebuild the ear file jar -uvf ccp.ear WMQ_IVT_MDB.jar

10. copy the file back to the dropins directory cp ccp.ear…

11. rename the file in the dropins directory mv wmq_jmsra_ivt.ear wmq_jmsra_ivt.ear.old

I found Configuring the resource adapter for inbound communication

and Configuring the resource adapter for inbound communication useful.

The document describes destinationLookup , which is new – but I could not get it to work. I needed destinationRef= pointing to a <jmsQueue..>.

If I removed destinationRef=… I got

CWWKG0095E: The element properties.wmqJms is missing the required attribute destinationRef.

Without < properties.wmqJms…> I got no console messages and it didnt work!

Can my MDB access this activationSpec?

I could not find any documentation on having your MDB access information from the ActivationSpec, so I dont think you can specify useful information such as which connectionfactory the MDB  should use for sending replies.

I want to use the same MDB for more than one queue or queue manager.

The <jmsActivationSpec id=…> maps to one queue. If you have the same id in two definitions, you get a combined definition, not two. Each major element has to have a unique id, and so you need a different .ear/.jar/mdb combination.

To use a second queue, with the same MDB I had to create a second ear file, ccpdup.jmsra.ivt.ear. This is not simply a copy of the wmq.jmsra.ivt.ear file because the wmq…ear file defines the web application http://192.168.1.222:9080/WMQ_IVT/, and Liberty complained because I had duplicate definitions.

WAS Traditional allows you to create a single activation spec definition, and share that across multiple MDBs. It is not possible to do that with Liberty though, so I had to adopt this approach

I did the following steps to create a copy of the ear file.

1. create a temporary directory

2. copy wmq.jmsra.ivt.ear info this directory.

3. expand it into its constituent pieces. jar -xvf wmq.jmsra.ivt.ear

4. remove the web application .war file I didnt need rm WMQ_IVT.war

5. edit META-INF/application.xml

6. remove the WMQ_IVT.war reference

7. change <display-name>wmq.jmsra.ivt</display-name> to <display-name>ccp.jmsra.ivt</display-name>

8. create a new jar file jar -cvf ccpdup.jmsra.ivt.ear WMQ_IVT_MDB.jar

9. add the meta data to the new jar file. jar -uvf ccp.jmsra.ivt.ear META-INF/*

10. copy ccpdup.jmsra.ivt.ear to the dropins directory

11. change server.xml to duplication the <jmsActivationSpec and change it to id=”ccpdup.jmsra.ivt/WMQ_IVT_MDB/WMQ_IVT_MDB”…>

12. In the dropins directory I copied ccpdup.jmsra.ivt.ear to zzz.jmsra.ivt.ear, and created an jmsActivationSpec. The liberty instance started – I dont know if it works!

I duplicated the <jmsActivationSpec..> and changed the name to ccpdup…..

This then worked.

You do not have to specify …

Under the covers the wmq.jmsra.rar logically generates a <resourceAdapter
id="wmqJms" ..> statement.

You do not need to specify <connectionFactory
...> because the <jmsConnectionFactory… > does this for you.

Here be dragons – take care

In medieval maps you would get pictures of sea monsters and words of warning “Here be dragons” etc.
I think deploying MDBs and jmsActivationSpec is dragon country because of the application development team specifying connectionFactoryLookup in the deployed file which overrides which queue manager parameters the system administrator has specified.  What works in test may not work in production, because the values have been hard coded.

The secret deployment can break you

Typically you have

1. application developers who create the MDB and the .ear, .war and .jar files.

2. System administrators who configure the environments and move deployments from test into production.

When using a MDB there is a risk that the jmsActivationSpec queue manager and queue information carefully crafted by the system administrators will be ignored. This is because the applications people have quietly added <activation-config-property-name>ConnectionFactoryLookup… → jndiname → queue manager name… deep down in the deployment of the jar file, and no one will know until there is a problem because the wrong queue is being processed, or a queue is not being serviced. To fix this you need to rebuild the application!

I could not find an easy set of commands to find if the string is in the ejb-jar.xml files. You have to extract from the .ear file to get the .jar files, then extract from the .jar files to get the ejb-jar.xml file and then use grep on it! So a big check every time you deploy (or redeploy) any MDB into production.

Difficult to deploy MDB more than once.

If you want to use an MDB on more than one queue, you need to have carefully consider your configuration. You may need multiple .ear files, or you may be able to use alias’s to a base .ear file.

Displaying connection manager information

You can use tools like jconsole to display information provided by JMX, such as the max size of a connection pool – but only a little useful information is provided.

I could not find how to display information about the connection factory built for the jmsActivationSpec.

Definitions not working.

I could not get applicationName=”MYAPP” definition to work on any binding connections.

This only works with client connections ( this is not documented). Once I had changed to use client connections, dis Q(IVT*) type(handle) gave me

AMQ8450I: Display queue status details.
 QUEUE(IVT2) TYPE(HANDLE)
 APPLTAG(jmsASIVTCF) PID(8037) 
 USERID(ibmsys1) 

Making it easier to configure your server.xml file

You can use the <include optional="true" location=".."/ statement to help

1. Put the JMS stuff in its own file, so you reduced the complexity of the server.xml file.

2. Put common, or repeated stuff in the file

See WAS Liberty documentation on Include

Location can be

• an explicit path “/home/colinpaice/myxml.xml”

• “${wlp.user.dir}/file.xml”
in my case this is ~/wlp/usr

• “${server.config.dir}/my.xml”.
Where ${server.config.dir} = ${wlp.user.dir}/servers/servername
the
same place as the default server.xml file is stored

For example if you could have a server.xml file with

<server>
<include location=”${server.config.dir}/jmsfeat.xml"/> 
<include location=”${server.config.dir}/jmscf.xml"/> 
</server>

and in the same directory have jmsfeat.xml with

<server>
<featureManager>
<feature>wmqJmsClient-2.0</feature> 
<feature>mdb-3.2</feature> 
</featureManager>
<variable name="wmqJmsClient.rar.location" value="/opt/mqm/java/lib/jca/wmq.jmsra.rar"/>
<wmqJmsClient  nativeLibraryPath="/opt/mqm/java/lib64"/> 
</server>

and jmscf.xml with

<server>  
<jmsConnectionFactory… >  
<properties.wmqJms queueManager="QMB"  transportType="BINDINGS”/> 
   </jmsConnectionFactory>  
</server>

When the server starts it puts out a message

[AUDIT  ] CWWKG0028A: Processing included configuration resource:  /home/colinpaice/wlp/usr/servers/test/jmsfeat.xml 
[AUDIT  ] CWWKG0028A: Processing included configuration resource:  /home/colinpaice/wlp/usr/servers/test/jmscf.xml

so you can see which files are (or are not) being included.

You  can specify

<include optional=”false” location=”${server.config.dir}/z.xml”/>

If this file is not found you get an error messages

[ERROR] CWWKG0090E: The ${server.config.dir}/z.xml configuration resource does not exist or cannot be read.

If  you specify optional=”true” you get no message if the file is not found.

See Customizing the Liberty environment for information on the ${…} variables

Being super efficient failed.

I tried putting the passwords in an includ file, but it did notwork.  So

Being super efficient failed.

<jmsConnectionFactory… >
<properties.wmqJms queueManager="QMB"  transportType="CLIENT" /> >

<include location=”/home/password.xml/password.xml”/> </jmsConnectionFactory>
</server>

did not try to include the file. If I moved the <include..> after the </jmsConnectionFactory> it  copied the file in but that did not help me with keeping the passwords in one place.

Also using Eclipse to validate the data in the server.xml file. Eclipse could check individual files, but it does not use information in other files, and may complain that something is not defined – when it is defined in another file.

Using Eclipse to check the configuration.

• • Start eclipse

• •  Window→ show view → servers

• •  “No servers are available. Click this link to define a new server”

• •  Select IBM → Liberty Server

• • Next

• • Select “Chose an existing installation” and give the directory path of the liberty code (/home/colinpaice/wlp)

• •  “next”

• •  “select an existing server or create a new one”

•  pick your server instance
• Select Servers, and expand your “liberty server at..” and then expand “server configuration server.xml”
• Double click on feature manager, and you should see the top middle pane have data in it.

You can now move through the file and check you definitions.

The “markers” tab has errors and warnings.

If you have used <include.. > then the validation is done on the individual file.

Some of the validation is not at the latest level. I had warning messages about  connectionFactoryLookup not being a valid parameter.

Use of Liberty Adminconsole

The Liberty Admin console provides a way of displaying your server.xml file. I found it often did not work, or did not display all of the information.

Check your liberty startup. If you have a statement like

[AUDIT ] CWWKT0016I: Web application available (default_host): http://…../adminCenter/

You may be able to use this URL to display and manage the server.xml file content.
It sometimes worked for me – sometimes it didnt and I got “One moment please…” and it did not get past this.

When it worked was random.

• Sometime changing the level of java make it work, sometimes closing down and restarting the browser made it work.

• Make sure you do not have the server.xml file open in an editor, or eclipse.

• I also received SRVE8094W: WARNING: Cannot set header. Response already committed.

• Working with one browser (Chrome) worked, when Firefox didn’t.

Good luck