Getting mqweb into production

You’ve got mqweb working,  you can now do administration using the REST API, or use a web browser in your sandbox environment to manage a queue manager.  You now want to get it ready for production – so where do you start?

I’ll document some of the things you need to do.  But to set the the scene, consider your environment

• Production and test
• Two major applications, accounts and payroll
• You have multiple machines for each application, providing high availability and scalability
• Teams of people
  • The MQ administration team who can do anything
  • The MQ RO administration team who can change the test systems, but have read only access to production
  • The applications teams who can change their test environment, but only have read only access to production
• You will use signed certificates (because this is production)  and not use passwords.
• People will get the same dashboard,  to make training and use easier.
• You want to be able to quickly tell if a dashboard is for production or test, and accounts and payroll
• You want to script deployment, so you deployment to production can be done with no manual involvement.
• You want a secure, available solution.

The areas you need to consider are

• the mqwebuser.xml file
• the keystore for the mqweb certificate
• the trust store for the authorisation certificates
• the dashboard for each user
• each user’s certificate store with their private keys
• Displaying the statistics on the mq console and REST requests.

Setting up security

It is better to give access using groups rather than by individual ids.

• If some one joins or leaves a team, you have to update one group, rather than many configuration files.
  • This is easier to do, and is easier to audit
• The control is in the right place.  For example the manager of the accounts team should mange the accounts group.  The MQ team should not be doing userid administration on the accounts group.

You will need groups for

• MQ Systems Administrators who can administer production and test machines
• MQ Systems  RO Administrators,  who can administer test machines, and have read access to production machines.
• Payroll – the applications manager may want more granular groups.
• Accounts  – the applications manager may want more granular groups.

You will need to set up the groups on each machine (you may well have this already).

Queue security

REST users need get and put access to SYSTEM.REST.REPLY.QUEUE.

For example

setmqaut -m QMA -n SYSTEM.REST.REPLY.QUEUE -t q -g test +get +put

then runmqsc refresh security

Set up the mqwebuser.xml file

The same file can be used for the different machines for “Accounts – production”, and a similar file for “Accounts – test” etc.

You may want to use “include”  files, so have one file imbedded in more than one mqwebuser.xml file.

Do not use the setmqweb command.   This will update the copy on the machine, and it will be out of sync with the master copy in your repository.

Define roles

The production environment for payroll may have

 <security-role name="MQWebAdmin">
   <group name="MQSA"/>
</security-role>

<security-role name="MQWebAdminRO">
  <group name="MQSARO"/>
  <group name="PAYROLL"/>
</security-role>

The test environment for payroll may have

<security-role name="MQWebAdmin">
   <group name="MQSA"/>
   <group name="MQSARO"/>
   <group name="PAYROLL"/>
</security-role>

<security-role name="MQWebAdminRO">
  <!-- none -all admin users can change test-->
</security-role>

Define http settings

By default mqweb is set up for localhost only.  You will need to have

• <variable name=”httpHost” value=”hostname” />

You may need to change the port value from

• <variable name=”httpsPort” value=”9443″/>

Define the keystore in mqwebuser.xml

Decide on the names and location of the key stores

• <keyStore id=”defaultKeyStore” location=”/home/mq/payrollproductionkeystore.p12” type=”pkcs12″ password=”{aes}AMsUYgpOjy+rxR7f/7wnAfw1gZNBdpx8RpxfwjeIG8Wj”/>
• <keyStore id=”defaultTrustStore” location=”/home/mq/payrollproductiontruststore.jks” type=”JKS” password=”{aes}AJOmiC8YKMFZwHlfJrI2//f2Keb/nGc7E7/ojSj37I/5″/>

Encrypt the keystore passwords  using the /opt/mqm/web/bin/securityUtility command. See here.

Ensure the deployment process gives the files the appropriate access.  The key store includes the private key, so needs to be protected.  The trust store should only have information in the “public” domain, such as certificates and no private keys, so could be universally read.

Set up the keystores

The keystore has the certificate and private key which identifies the server.  The certificate needs the subjectAltName specified which has a list of valid url names and IP addresses.
You need to decide if you want one certificate per server, and so have several certificates

subjectAltName = DNS:payroll1, IP:10.4.6.1

or several systems in the list, and have one certificate

subjectAltName = DNS:payroll1, DNS:payroll2, IP:10.4.6.1,  IP:10.9.5.4

You may want to create the keystore on your build environment, and securely deploy it to the run time machines, or send the .p12 file across and import it.  I think creating the keystore and deploying it is more secure.

If you change the keystore you have to restart mqweb to pickup changes.

Set up the trust store.

The trust store is used to validate certificates sent from the client for authentication.  In an ideal work, this will have just one CA certificate.  You may have more than one CA.  If you have self signed certificates this creates a management problem.

You may be able to use the same trust store for all your environments.   The access control is done by the security-roles in the mqwebuser.xml, not by the trust store.

The cn from the certificate is used as the userid. So both

cn=colinpaice,ou=test,o=sss and cn=colinpaice,ou=prod,o=sss are valid, and would extract userid colinpaice.

If the trust store is changed, the mqweb server needs to be restarted.

End user certificates

Each user will need a certificate to be able to access the mqweb server.  This needs to be signed by your CA, and needs to be set to trusted.  You should have this set up already.

If you have more than one valid certificate in the browser store, you will be prompted to pick one.   This is used until the browser is restarted.

You can configure mqweb to log off users after a period.   If you are using certificates, the browser will automatically log you on again!

Dashboard

The dashboard is the layout of the mqweb window, the tabs in the window, and the widgets on the tabs.

You will generally want users to have the template you define, and not have to create their own. So the Payroll team use the payroll dashboard, and the MQ admin team use the MQADMIN dashboard.

Create a dashboard and use export to create a json file.   You can store in your configuration repository.   You can change queue manager names as you deploy it for example change QMPayroll1 to QMPayroll2.

On the MQ machines these files are stored in the  /var/mqm/web/installations/Installation1/angular.persistence directory.

You can put your templates for that machine in this directory, and use symbolic links for a userid to their dashboard. For example

ln -s common.json colinpaice.json

If the dashboard.json is made read only, then people will not be allowed to change it online.

Is this dashboard for production or test?

I could not find a way of customise the colours of a page, so you cannot easily tell which is production and which is test etc.

I need a secure available solution.

You can use userids and passwords, or certificates to provide authentication.

You need to protect access to MQ objects

You need to protect the files used by mqweb, especially the key store, and the mqwebuser.xml

If you update the mqwebsuser.xml file, it will pickup up changes a short while later (seconds rather than minute).

If you change the keystore or trust store you need to restart mqweb to pick up the changes.   This should take about 10s of  seconds.

Deploy scripts

All of the configuration can be done with scripts.  For example extract your mqwebuser.xml file, make machine specific changes and deploy it.

You can create the keystores in your secure build environment and deploy them.

House keeping

• You should check /var/mqm/web/installations/Installation1/servers/mqweb/logs/ffdc daily for any files, and raise a PMR with PMR if you get any exceptions.
• Check /var/mqm/web/installations/Installation1/servers/mqweb/ daily.  I was getting large (700MB) dumps in this directory, which caused my machine to go short on disk space.
• Display the server certificate expiry date (any any CA certificates) and put a date in your diary to check (and renew) them.
• Your enterprise should have a process for renewing personal certificates

Someone joins the department

• Connect them to the appropriate group on all machines
• Give them a symbolic link to the appropriate dashboard file, in /var/mqm/web/installations/Installation1/angular.persistence

Collect statistics on the MQ console and REST requests, and the JVM

See these posts

• Getting  statistics on the MQ console and REST requests, and the JVM
  • Getting useful information out of JMX statistics data
  • Accessing JMX data in Liberty server, securely
  • Setting up Liberty to use native JMX
  • Use the Liberty REST API to access the JMX data  – better than using the native JMX interface