mqweb- is it better to use certificate authentication or userid and password?

Like may questions the answer is it depends.

You can set up your mqweb environment

Certificates are required – you cannot use userid and passwords
If a valid certificate is found use that, else prompt for userid and password
Do not use certificates – just use userid and passwords.

This is described in the Liberty documentation.

It says for the <ssl…> tag

If you specify clientAuthentication="true", the server requests that a client sends a certificate. However, if the client does not have a certificate, or the certificate is not trusted by the server, the handshake does not succeed.
If you specify clientAuthenticationSupported="true", the server requests that a client sends a certificate. However, if the client does not have a certificate, or the certificate is not trusted by the server, the handshake might still succeed.
If you do not specify either clientAuthentication or clientAuthenticationSupported, or you specify clientAuthentication="false" or clientAuthenticationSupported="false", the server does not request that a client send a certificate during the handshake.

No matter how you logon, the web server creates a LTPA* cookie with encrypted information about the logon. The mqwebuser.xml attribute ltpaExpiration says how long this cookie is valid for. If the time is exceeded you have to logon again, and it generates a new cookie.

Logging on

Certificate authorisation

If you are using certificates, the server sends down a list of valid self signed certificates or CA certificates, and the browser compares this list with the certificates in its store. If the certificate in the browser’s store is acceptable, it is added to a list

The list is displayed, and the end user selects a certificate. For subsequent requests with the same http://url:port combination, in the same or a different tab within the browser window, the same certificate is used, and the browser does not prompt.

If the ltpa cookie expires then the same certificate is used, invisibly to the end user. A new cookie is generate and sent down.

If there are no valid certificates in the list, then depending clientAuthentication, it may prompt for a userid and password.

You cannot logoff from the page as there is no option to do so.

Userid and password.

When userid and password are passed up in a header (not in the URL as some web servers accept) the ltpa token is returned. When the ltpa token expires, the user is prompted to re-logon. They need to enter the userid and password again. You can also logoff from the web page.

What are the implications of certificates and userid with password

Certificate logon

The web browser can logon, and although the ltpa token expires, it is automatically logged on again. This is useful for a “big screen monitor” in the operations room, where you want the display active all day.
Someone can create a file with some HTML in it to browse a queue (if authorised). So if they are authorised to the queue for their normal job, they could easily create a web page to display the queue contents. This could be the end user, or an evil person saying “click this link”.

userid logon

If the user is using the mqconsole web pages, then periodically they will get logged off – and they will have to logon again.
If they try to access a queue from an HTML page, they will be asked for userid and password. Depending on what the end user is doing, this may give them notification than there is some unusual activity. If they are hacking they will enter the userid and password anyway.
The person may have access to the same information without using the mqweb console interface, so there is no change to the threat level.

What do I recommend?

Tricky.

I think long running monitors need to use certificates
End users should logon with userid and password, with an expiry period of about an hour or less.
You may want to have a Certificate Authority just for the mqweb browsers, to stop your corporate certificates from being used to access the mqweb server.

Because it is easy to create an html page and browse a queue, I would allow administration over mqrest, and not allow processing messages over REST.

As I said in another posting

The admin API authorisation is managed through <security-role name=”MQWebAdmin”> and <security-role name=”MQWebAdminRO”> sections in the mqwebuser.xml file.
The messaging API authorisation is managed through <security-role name=”MQWebUser”> sections.

To stop messaging over the rest interface, have no entries in the <security-role name=”MQWebUser”> section. This applies to both certificates, and userid and password.

(~~Personally I would have called<security-role name=”MQWebUser”> as <security-role name=”MQWebMessaging”> to make the roles clearer.~~

Update from Gwydion…

The MQWebAdmin, MQWebAdminRO and MQWebUser roles can all be used for the admin REST API. That’s why MQWebUser is not called MQWebMessaging – it’s not just for messaging. The difference between them is the user ID that’s checked by the qmgr for operations performed via the REST API.

Operations performed by users in the MQWebAdmin* roles take place under the context of the mqweb server user ID.
Operation performed by users in the MQWebUser role take place under the context of the user logged into the REST API.

To stop messaging over the REST API completely, the mqRestMessagingEnabled should be set to false in the mqweb server configuration.

What does mqRestCorsMaxAgeInSeconds in mqweb mean?

Ive blogged about CORS, and how this allows you to list sites that are permitted to use scripts to send request to the mqweb server.

I struggled with understanding what value mqRestCorsMaxAgeInSeconds has, as it did not behave as expected (this was due to a bug).

If you have a CORS transaction there is an OPTIONS request, followed by the actual DELETE/GET/POST requests.
The OPTIONS request checks that the request is coming from an authorised list of URLs, and that the parameters are valid. The OPTIONS information can be cached.

If the check is successful then the real request can be issued. If the requests occur in a short time, then you can have OPTIONS, DELETE, DELETE, DELETE, using the cached values. If there is a “long” time interval between the requests you may get OPTIONS, DELETE, gap, OPTIONS, DELETE.

The OPTIONS information can be cached for up to the minimum of the mqweb mqRestCorsMaxAgeInSeconds time, and the browser time.

For Chrome the maximum time interval is 600 seconds. If no time is specified in the OPTIONS response, then 5 seconds is used.

There is a bug in the Liberty base code which sends down the header Access-Control-Allow-Max-Age: …, when the browser is expecting Access-Control-Max-Age. Because of this, the default time of 5 seconds is used in Chrome.

This should not have a major impact. For those applications using scripts to send multiple REST API request, there will be more frequent OPTIONS requests – every 5 seconds instead of up to 600 seconds. These extra flowes are invisible to the scripts and the end user.

What value should I use?

Chrome has a maximum of 600 seconds, with a default of 5 seconds.

Firefox has a maximum of 24 hours (86400 seconds).

Setting it to 600 seconds sounds reasonable to me.

Making changes to mqRestCorsMaxAgeInSeconds

If you change mqRestCorsMaxAgeInSeconds you have to restart the mqweb server.

I do not get caching!

When researching this topic I found every GET request had an OPTIONS request, rather than the OPTIONS, GET, GET. A quick search on the internet showed me the Chrome option ( F1 -> Settings and preferences) “Disable Cache ( while DevTOOLS is open)” was enabled. I deselected this, and I got the caching.

Can evil websites get to your mqweb – using java script to get to the backend server

With HTML and scripting people would write scripts and get you to execute them, and so access your personal information, and steal your money. Over time security has been improved to make this harder, and now you have to explicitly say which web sites can run scripts which use the mqweb interface to access MQ to put and get messages.

One way of protecting the access is using Cross Origin Resource Sharing, or CORS. I explained the basics of CORS here. I struggled getting it to work with a web browser.

browsers have been improved and what worked last year may no longer work now, and the documentation does not reflect the newer browsers.
Chrome carefully changes your hand crafted HTTP headers, so what is sent up may not what you expected.

I’ll go through three examples, and then show how it gets more difficult, and what you can do to identify problems.

Note: If you use a web page from a file:// then the origin is ‘null’, and this will fail any checks in the backend, as the checks compare the passed origin to the list of acceptable urls.

I used Dev Tools in Chrome (Alt+Ctrl+i) to display information including the headers flowing.

Simple HTML

With the following within your web page,

<a href=”https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/…/message ”
> direct link ></a>

It issues the REST request, returns the data from the queue, and displays it.

From the headers we see

Simple HTML: Request headers

The important request headers were

Host: localhost:9445
Referer: http://localhost:8889/

And no origin header.

Simple HTML: Response headers

The response headers have

HTTP/1.1 200 OK
ibm-mq-md-expiry: unlimited
ibm-mq-md-messageId: 414d5120514d412020202020202020204c27165d04a98a25
ibm-mq-md-persistence: persistent
Content-Length: 1024
Set-Cookie: LtpaToken2_1 ….

There are no Allow…. headers, so this indicates the response is not a valid cross origin response. The request came from one page, so there was no cross origin request.

You can see the MQ attributes returned, ibm-mq*.

Invoking request from java script.

My HTML page had

<html
<head>
<script src=”http://localhost:8884/src.js ” text=”text/javascript”></script >
</head>
<body>
<button onclick=”src()””>Get message</button>
</body>

I had the src() script downloaded from http://localhost:8884/src.js, and inline within the html file. Both worked.

function src()
{
  fetch("https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message",
  { 
     method :'DELETE',
     headers: {
                'Origin': 'https://localhost:888499'
                ,'Access-Control-Request-Headers' : 'Content-Type' 
                ,'ibm-mq-rest-csrf-token' : '99'
              } 
   }
  ) // end of fetch
   .then((response) => response.text() )
   .then(x => {document.write("OUTPUT:"+x); } 
  ) 
  .catch(error =>  {console.log("Error from src:" + error);});
}

When the button was pressed, the script was executed, there was an OPTIONS request (and response), and a DELETE request (and response). It returned a message and displayed it.

In more detail, the flows were:

JavaScript OPTIONS request

The OPTIONS request had headers

Access-Control-Request-Method: DELETE
Origin: http://localhost:8889

This is doing a preflight check, saying it intends to issue a DELETE request from Origin http://localhost:8889, the url of the web page.

JavaScript OPTIONS response headers

The OPTIONS response headers were the same as before but with additional ones

Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:8889
Access-Control-Allow-Max-Age: 91
Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE
Access-Control-Allow-Headers: Accept-Charset, Accept-Language, Authorization, Content-Type, ibm-mq-rest-csrf-token, ibm-mq-md-correlationId, ibm-mq-md-expiry, ibm-mq-md-persistence, ibm-mq-md-replyTo, ibm-mq-rest-mft-total-transfers

This means, the script is allowed to issue a request from http://localhost:8889 using a request method in the list {GET, POST, PATCH, PUT, DELETE } and the valid headers that can be used are in the list of headers.

The Access-Control-Allow-Max-Age: 91 came from my mqwebuser.xml file, <variable name=”mqRestCorsMaxAgeInSeconds” value=”91″/>.

After this there was a DELETE request to get the message.

JavaScript DELETE request headers

Host: localhost:9445
ibm-mq-rest-csrf-token: 99
Origin: http://localhost:8889

JavaScript DELETE response headers

The response included the CORS headers, and included the headers from the non CORS situation

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:8889
Access-Control-Expose-Headers: Content-Language, Content-Length, Content-Type, Location, ibm-mq-qmgrs, ibm-mq-md-messageId, ibm-mq-md-correlationId, ibm-mq-md-expiry, ibm-mq-md-persistence, ibm-mq-md-replyTo, ibm-mq-rest-mft-total-transfers
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
ibm-mq-md-expiry: unlimited
ibm-mq-md-messageId: 414d5120514d412020202020202020204c27165d04a98a25
ibm-mq-md-persistence: persistent
Content-Length: 1024
Set-Cookie: LtpaToken2_….

Because the Access-* headers are present, this is a CORS response.

The message content was displayed in a browser window.

Link to another page

I set up a link <a href=”http://localhost:8884/page.html”/>localhost 8884</a> to execute a page on another web server. When this executed, it issued the java script request as before. The Origin was Origin: http://localhost:8884 – so the page where the script executed.

What happens if CORS is not set up?

If the http://localhost:8889 is not in the list in the mqwebuser.xml file,

No data was displayed. The Chrome browser Developer tools ( Alt+Ctrl+i) displays a message

Access to fetch at ‘ https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message ‘ from origin ‘http://localhost:8889 ‘ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. If an opaque response serves your needs, set the request’s mode to ‘no-cors’ to fetch the resource with CORS disabled.

The OPTIONS Request header has, as before

HTTP/1.1 200 OK
Host: localhost:9445
Access-Control-Request-Method: DELETE
Origin: http://localhost:8889
Access-Control-Request-Headers: ibm-mq-rest-csrf-token

but the OPTIONS Response header has no Access-* headers.

No DELETE request was issued.

The HTTP/1.1 200 OK is the same for all cases – it means the request was successful.

Trying to be clever

I read the documentation on the web, and much of it was very helpful, but some of it is no longer true. It was hard to get it working, because every things has to be right for it to work.

Unsupported header.

I added an extra header, which is a valid CORS thing to do – but the back end has to support it. With hindsight it makes no sense to add headers that will be ignored by the server.

headers: {
     'Origin': 'https://localhost:888499'
     , 'Access-Control-Request-Headers' : 'Content-Type' 
     , 'colin':'TRUE' 
     ,'ibm-mq-rest-csrf-token' : '99'
}

This sent up a header with

Access-Control-Request-Headers: colin,ibm-mq-rest-csrf-token

Header “colin” is not in the list of accepted headers, header ibm-mq-rest-csrf-token is in the list:

Access-Control-Allow-Headers: Accept-Charset, Accept-Language, Authorization, Content-Type, ibm-mq-rest-csrf-token, ibm-mq-md-correlationId, ibm-mq-md-expiry, ibm-mq-md-persistence, ibm-mq-md-replyTo, ibm-mq-rest-mft-total-transfers

The Developer tools message was the same as before,

Access to fetch at ‘https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message ‘ from origin ‘http://localhost:8889 ‘ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. If an opaque response serves your needs, set the request’s mode to ‘no-cors’ to fetch the resource with CORS disabled.

I removed the unsupported header and it worked.

Other headers do not work

The request header ‘Access-Control-Allow-Method’ : ‘DELETE’ is a valid header.

When this was used, the request headers included

Access-Control-Request-Headers: access-control-allow-method,ibm-mq-rest-csrf-token
Access-Control-Request-Method: DELETE

As before Access-Control-Request-Method is not in the list of Access-Control-Allow-Headers, so the request fails.

The Access-Control-Request-Method: DELETE is not needed, as the method: DELETE defines what will be used.

Using Access-Control-Request-Headers to add more headers does not work, if the header is not in the list of valid parameters.

The true origin is used

I tried using the header ‘Origin’: ‘https://localhost:888499 ‘ – as it did with my curl – and this was ignored. The true Origin from the web page was used (I am glad to say, otherwise it would make the whole protection scheme worthless)

Some options ignored

I found the “fetch” options credentials:, redirect: , and cache:, were all ignored by Chrome.

Some cookies were used.

The LtpaToken2_… was sent up and down

Problems in mqwebuser.xml file

I misconfigured the configuration file with <variable name=”mqRestCorsMaxAgeInSeconds” value=”AA91″/>

This gave me the familair message

Access to fetch at ‘https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message ‘ from origin ‘http://localhost:8889 ‘ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. If an opaque response serves your needs, set the request’s mode to ‘no-cors’ to fetch the resource with CORS disabled.

So if you get this message it is worth checking the configuration file for problems.

Can I trace it?

I used <variable name=”traceSpec” value=”*=audit:CorsService=finest”/> in the mqwebuser.xml file to provide information about the CORS processing, and what was coming in, and what was being checked.

Can evil websites get to your mqweb – understanding CORS

In the beginning was the html, and the html was good; then we had html and scripts which could only do things on the page, which was also good; then we had scripts which could reach out to other websites – and that’s where the problems began. It was easy for evil developers to get you to click on an innocent looking page, which had a script which jumped into a different tab of your browser where you had your banking window open , or to executed a script ; and steal all your money.

The browsers were improved to stop evil scripts from accessing a server, and then they were improved again so the server could say “stuff coming from this list of web sites is OK, I trust them. One implementation is called CORS. There is a good description here. It says

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP headers to tell browsers to give a web application running at one origin, access to selected resources from a different origin. A web application executes a cross-origin HTTP request when it requests a resource that has a different origin (domain, protocol, or port) from its own.

I do not trust things that “just work”, I like to see evidence of it. This lack of trust comes from working with a group of young testers who came to IBM Hursley to test their code. All of their tests ran cleanly – and thought they had a few days spare to go to London sightseeing. I stopped the server they were meant to be using (without telling them) and the tests carried on running successfully! It turned out they had been testing their code with a dummy program acting as the server. They removed this code, reran their tests and most of them failed – and had to stay an extra week.

I also remember changing a config file, and being surprised when my changed worked first time. After a cup of tea (an invaluable thinking aid) I put some spelling mistakes in the file ; and it carried on running. Why? I was using the wrong config file.

I played with CORS and wanted to get things to fail, as well as to work. This was a good choice, as I had many failures.

I’ll document how I got curl to work and demonstrate CORS , and document how I got a web browser to work – a real challenge

mqweb implements CORS, so you can configure mqweb to give a list of websites which may access your server.

The documentation is not very clear. It says

where allowedOrigins specifies the origin that you want to allow cross-origin requests from. You can use an asterisk surrounded by double quotation marks, “*”, to allow all cross-origin requests. You can enter more than one origin in a comma-separated list, surrounded by double quotation marks. To allow no cross-origin requests, enter empty quotation marks as the value for allowedOrigins.

My observations,

You cannot use generics, so http://127.0.0.1:* is the same as “*” – or allow all cross-origin requests
You must specify {scheme:address:port} so http or https, the url with // at the front, and the specific port number
The match is an string equality test, so the case, spacing and values must be the same
- http://127.0.0.1:8888 is different from http://localhost:8888. The value is what you specify in the browser url or the header, so a browser url of http://localhost:8888 may give different results to http://127.0.0.1:8888 even though they map to the same address

How does an HTTP request work?

When you click on a web page, data is sent to the back end server. The following data is exchanged

the request
request headers
your data going to the server
response headers
response data – such as the content of web page.

In more detail…

Request

The URL for example: https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message
What the request type is – for example GET, DELETE, POST, OPTIONS
Remote address of the server 127.0.0.1:9443

Request Headers

accept: for example text/html, application/xml
accept-languages: en-GB
dnt: 1 this is “do not track me”
user-agent: Chromium
cookie: bcookie=”….”
…

Response headers

status: 200
status: 200 OK
set-cookie ….
server: nginx
….

Response data

This might be the web page to be displayed. It can include script, images etc.

What is the origin of the page?

If your web page, invoked a script, for example from clicking a button, an “Origin” header is added, for example Origin: http://localhost:8884 which is the address of the web server hosting the page. The backend server checks to see if this header is present, and looks up site in the authorised list.

If the Origin is acceptable, it sends down additional response headers (CORS Headers), so the browser (or your program) knows and can use the web site.

As part of the handshake, the browser can send up an “OPTIONS” request (instead of a get/delete etc), with a header saying the browser will be doing a GET/DELETE etc from this origin. If there is a positive response, where the additional CORS headers are send back then the get/delete is allowed. If the CORS headers are not present in the response, then the request will not be permitted. This is called a preflight check – just like having your boarding pass checked at the gate before you get on the plane.

Does it work?

If there is no Origin header in the request, the backend server thinks it is all same domain, and does no CORS checks.

This CORS support is really aimed at web browsers, as the web browsers will automatically add headers for you. If you are using curl or other tools to create your own request, you specify exactly the headers you want, so if you omit the Origins header, the server will not check.

My mqwebuser.xml file had <variable name=”mqRestCorsAllowedOrigins” value=”https://9999.0.0.1:19442,http://localhost:8884”/>

So origins for https://9999.0.0.1:19442 and http://localhost:8884 are permitted.

I used a configuration file for curl (because command parameters did not work passing the headers) and had in curl.parms

cacert ./cacert.pem
cert ./colinpaice.pem:password
key ./colinpaice.key.pem
cookie cookie.jar.txt
cookie-jar cookie.jar.txt
request OPTIONS
header “Access-Control-Request-Method: DELETE”
header “Access-Control-Request-Headers: Content-Type”
header “Origin: https://9999.0.0.1:19442 ”
header “ibm-mq-rest-csrf-token : COLINCSRF”
include

I used the command

curl –verbose –config curl.parms –url https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message

The headers were displayed, and I had

OPTIONS /ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message HTTP/1.1
> Host: localhost:9445
> User-Agent: curl/7.58.0
> Accept: */*
> Cookie: LtpaToken2_….
> Access-Control-Request-Method: DELETE
> Access-Control-Request-Headers: Content-Type
> Origin: https://9999.0.0.1:19442
> ibm-mq-rest-csrf-token : COLINCSRF

The response headers were

< HTTP/1.1 200 OK
< X-Powered-By: Servlet/3.1
< X-XSS-Protection: 1;mode=block
< X-Content-Type-Options: nosniff
< Content-Security-Policy: default-src ‘none’; script-src ‘self’ ‘unsafe-inline’ ‘unsafe-eval’; connect-src ‘self’; img-src ‘self’; style-src ‘self’ ‘unsafe-inline’; font-src ‘self’
< Cache-Control: no-cache, no-store, must-revalidate
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: https://9999.0.0.1:19442
< Access-Control-Allow-Max-Age: 90
< Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE
< Access-Control-Allow-Headers: Accept-Charset, Accept-Language, Authorization, Content-Type, ibm-mq-rest-csrf-token, ibm-mq-md-correlationId, ibm-mq-md-expiry, ibm-mq-md-persistence, ibm-mq-md-replyTo, ibm-mq-rest-mft-total-transfers

The response header Access-Control-Allow-Origin: https://9999.0.0.1:19442 shows that requests with origin https://9999.0.0.1:19442 is acceptable.

When I used a different “Origin”, I did not get any Access-Control-Allow-* headers. So from the absence, I could tell the request was not support from the different origin.

The Access-Control-Allow-Headers is a list of header names which can be sent, so the header ibm-mq-rest-csrf-token : COLINCSRF is valid, but “COLIN:value” is not valid, because ibm-mq-rest-csrf-token is in the Access-Control-Allow-Headers list, and COLIN is not in the list.

The Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE says my application can use any of the methods in the list.

Using a web browser.

If you use HTML in a local file, then the “Origin” is null, and so does not match any elements in the authorised list. I had to set up my own web server – which was easy to do using Python.

Using the web page at the bottom of this posting, I pointed my web browser at the web server. It displayed a button. I pressed it, and it invoked a script. Using the developer mode ( Al+Ctrl +i) in Chrome could see network flows etc.

The request headers had

Host: localhost:9445 This is where my webserver is hosted
ibm-mq-rest-csrf-token : 99 I specified this header value
Origin: http://localhost:8884 this overrode the value I had specified in the headers.

The response headers included

Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:8884
Access-Control-Expose-Headers: Content-Language, Content-Length, Content-Type, Location, ibm-mq-qmgrs, ibm-mq-md-messageId, ibm-mq-md-correlationId, ibm-mq-md-expiry, ibm-mq-md-persistence, ibm-mq-md-replyTo, ibm-mq-rest-mft-total-transfers
ibm-mq-md-expiry: unlimited
ibm-mq-md-messageId: 414d5120514d412020202020202020204c27165d04a98a25
ibm-mq-md-persistence: persistent

We can see from

the Access-Control-* headers we know this has validated for http://localhost:8884
the Access-Control-Expose-Headers we can see what headers will be accepted
ibm-mq-md-persistence: persistence. For the returned messages, it was a persistent message

The web page used

<HTML>
<HEAD>
<TITLE>Call a  mqweb rest API </TITLE>
<script>
  function local()
  {
    fetch("https://localhost:9445/ibmmq/rest/v1/messaging/qmgr/QMA/queue/DEEPQ/message" 
      {
         method :'DELETE',
         headers: {
                    'Origin': 'https://localhost:888499'
                    , 'Access-Control-Request-Headers' : 'Content-Type'
                    ,'ibm-mq-rest-csrf-token' : '99'
                  }
     }
    )
    .then((response) => response.text() )
    .then(x => {  document.write("OUTPUT:"+x);    } )
    .catch(error => { console.log("Booo:" + error);});
  }
</script> /head> <body> <button onclick="local()"">press me</button> </body> </html>

For information about “fetch(…,…)” see here.

For information about the “.then(…) ” see here.

How do I tell if my puts and gets are in or out of syncpoint?

This question came up from someone who is new to MQ. it is good to find people people who are new to MQ.

As usual there is an easy answer, but some cases you need to think about.

There is no command you can use to show this information. You can find it it out using the accounting information.

Typically an application does

MQCONN
MQOPEN queue
MQPUT|MQGET
Possibly a commit
MQCLOSE
MQDISC

If the accounting data shows that there was a COMMIT request, and the commit took about 1 millisecond, then there is a good chance that the put or get was in syncpoint.

If gets more interesting if you have

MQPUT of messages out of syncpoint (for example and audit trail – we got to this point. Out of syncpoint so it exists even after a rollback)
MQPUT within syncpoint
MQCOMMIT

If the accounting information shows one commit, you cannot tell if it was one or two messages within syncpoint.

If the message is persistent and out of syncpoint, then under the covers it does “MQPUT|MQGET – and commit this requests”. On z/OS a put or get typically take 10 microseconds, a log I/O takes about 1 ms, so if you see an MQPUT or MQGET with a long response time (1 millisecond), then it might have been a put or get of a message outside of syncpoint.
On z/OS you can also get the logging time for a put or get, so if you see the logging time for a put or get, this is a persistent message out of syncpoint.

A non persistent message does not write to the log on disk, all updates are kept in memory, so you cannot tell from the duration of the request if it was in or out of syncpoint. Best practice is that non persistent messages are usually processed out of syncpoint.

If you see persistent messages are put or got, and the commit time is microseconds, then the commit did not do disk IO, so there was no work to commit, and so the put or get was out of syncpoint.

And lastly, you may see no commit request in the accounting data. This is bad practice, as it is assuming that the MQDISC will cause a commit or roll back, so it would happen invisibly. An application should explicitly issue a commit if messages are within syncpoint.

As I said – pretty simple – but with some complexities.

Error connecting to JMX endpoint: Unsupported protocol: rest

I got this error message when I was trying to use JMX into the WLP web server, when using the restConnector-2.0 interface in Liberty.

The documentation was not that helpful. Oneproblem was I suffered from the curse of cut and paste, and used a Windows environment variable %JAVA_HOME% instead of using the Linux $JAVA_HOME. Another problem was caused documentation saying add the jar to the class path, and then the class path was ignored.

True documentation.

When you use the -jar option, the JAR file is the source of all user classes, and other user class path settings are ignored. See here.

Unhelpful documentation

Configuring secure JMX connection to Liberty says

jconsole -J-Djava.class.path=...;%WLP_HOME%/clients/restConnector.jar

This was for Windows – on Linux it would be $WLP_HOME – except on my system $WLP_HOME was not set.

jconsole

Some of the jar files are in $JAVA_HOME, you can use the environment variable, or specify it yourself. Note %JAVA_HOME% is windows, so be careful when you use cut and paste.

jconsole -J-Djava.class.path=$JAVA_HOME/lib/jconsole.jar:$JAVA_HOME/lib/tools.jar:/opt/mqm/web/clients/restConnector.jar
jconsole -J-Djava.class.path=/usr/lib/jvm/java-8-openjdk-amd64/lib/jconsole.jar:/usr/lib/jvm/java-8-openjdk-amd64/lib/tools.jar:/opt/mqm/web/clients/restConnector.jar

If the window displays “Secure connection failed” restart jconsole and use the -debug option. For me this gave “java.io.IOException jmx.remote.credentials not provided. Set to a String[2] {user,password}”. I entered my userid and password, and this connected successfully.

I could not connect using my .jar file.

I was using JMXQuery to extract the data. I was getting the Error connecting to JMX endpoint: Unsupported protocol: rest message.
Adding the jar file to my class path did not help, as the class path is ignored when using java the -jar parameter.

How to fix it

There are two ways of fixing this.

Put the required jar file in the extensions path, not the class path
Use the java -classpath… instead of specifying java -jar

1. You need to have the jar for in the extensions path, not in the class path.

See How classes are found in the Java documentation. It says

Bootstrap classes – Classes that comprise the Java platform, including the classes in rt.jar and several other important jar files.
Extension classes – Classes that use the Java Extension mechanism. These are bundled as .jar files located in the extensions directory.
User classes – Classes defined by developers and third parties that do not take advantage of the extension mechanism. You identify the location of these classes using the -classpath option on the command line (the preferred method) or by using the CLASSPATH environment variable.

There is a system property java.ext.dirs which gives the location of the Extension classes. On my system this was

/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext

The /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext: is java dependent. You should not put your files in this directory
/usr/java/packages/lib/ext This is for “user” extensions.

The directory /usr/java/packages/lib/ext did not exist on my system, so I had to do, create it, copy the web connection jar file to it, and grant permissions on it.

sudo mkdir -p /usr/java/packages/lib/ext
sudo cp /opt/mqm/web/clients/restConnector.jar /usr/java/packages/lib/ext/
sudo chmod 444 /usr/java/packages/lib/ext/restConnector.jar

2. Use a -classpath – not a -jar

In a .jar file there is a META-INF/MANIFEST.MF file which includes information on the entry point.

Manifest-Version: 1.0
Main-Class: src.Client

Instead of using the -jar option to point to the jar, you can use the -classpath to point to the jar and explicitly specify the entry point. For example

java -cp ./Client.jar:/opt/mqm/web/clients/restConnector.jar  
src/Client
service:jmx:rest://localhost:9443/IBMJMXConnectorREST

java -cp ./Client.jar:/opt/mqm/web/clients/restConnector.jar – use the classpath option, and specify the needed jar files. Client.jar is my program. /opt/mqm/web/clients/restConnector.jar is the Liberty provided jar.
src/Client – the “entry point” class to use
service:jmx:rest://localhost:9443/IBMJMXConnectorREST – the url to use.
note the absence of a -jar option.

In the manifest of the Client.jar file it had Main-Class: src.Client. This is for the the src/Client.java source file. This ties up with the src/Client.class as seen when you use the command jar -tvf Client.jar .

Use the Liberty REST API to access the JMX data in Liberty

Rather than set up traditional JMX, where you specify the JMX port etc, you can use the REST support provided in LIberty to access the JMX data. The rest support is easier to set up.

The Liberty documentation recommends that you do not have the native JMX support (configured in jvm.options), and the Liberty REST support for JMX configured at the same time.

The REST request to get the statistics worked and was easy to use. I could not get the “traditional JMX interface”, such as jconsole to work with the REST interface. See below.

Configure the server: mqwebuser.xml

In mqwebuser.xml add the support

<featureManager>
  <feature>restConnector-2.0</feature>
</featureManager>

Set up authorisation with

<administrator-role>
   <user>colinpaice</user>
   <group>MQADMIN</group>
</administrator-role>

<reader-role>
  <user>John</user>
</reader-role>

As the mqconsole and rest statistics are read only, then it may be better to set up every user as a reader-role.

As with the MQ support, it will use the userid if specified, or the Common Name from the digital certificate.

Using Curl and the rest API

I used

curl --cacert ./cacert.pem --cert-type P12 
--cert colinpaice.p12:password
-url https : //localhost:9443/IBMJMXConnectorREST/mbeans/...

Where … was WebSphere:name=com.ibm.mq.console.javax.ws.rs.core.Application,type=ServletStats/attributes/ResponseTimeDetails . This gives the JMX statistics for the mq.console.

The data comes back as JSON (as you might expect) for example

"name": "ResponseTimeDetails",
  "value": {
    "value": {
      "count": "99",
      "description": "Average Response Time for servlet",
      "maximumValue": "3183755861",
      "mean": "1.116053166969697E8",
      "minimumValue": "1777114",
      "reading": {
        "count": "99",
        "maximumValue": "3183755861",
        "mean": "1.116053166969697E8",
        "minimumValue": "1777114",
        "standardDeviation": "4.360373971884932E8",
        "timestamp": "1580819294060",
        "total": "1.1048926353E10",
        "unit": "ns",
        "variance": "1.90128611746915776E17"
      },
      "standardDeviation": "3.218674128849494E8",
      "total": "1.1048926353E10",
      "unit": "ns",
      "variance": "1.63102693370991648E17"
    ...

As well as the data I have covered before, you also get the time stamp value. This is the value in milliseconds from a well known time.

I used the python to convert the timestamp (1580978634610) to a date time

import datetime
s = 1580978634610 / 1000.0
print(datetime.datetime.fromtimestamp(s).strftime('%Y-%m-%d %H:%M:%S.%f'))

to give 2020-02-04 12:28:14.060000.

Which URL to use for traditional JMX?

The IBM documentation says the url to access the JMX data using “traditional JMX” is in file /var/mqm/web/installations/Installation1/servers/mqweb/log/state/com.ibm.ws.jmx.rest.address. For me this was service:jmx:rest://localhost:9443/IBMJMXConnectorREST .

Client set up: Using the “traditional JMX interface” did not work for me

The Configuring secure JMX connection to Liberty page says you can use this url in jconsole and other tools. I could not get this to work. I got messages like Error connecting to JMX endpoint: Unsupported protocol: rest. This page gives a lot of information on JMX, and towards the end of the second page it says Error connecting to JMX endpoint: Unsupported protocol: xxxx is likely to be a problem with the class path. I used -verbose=class, and did not see the jar file being loaded.

What else can you do with the REST interface?

Show what is available

https://localhost:9443/IBMJMXConnectorREST/graph shows a picture of the facilities available with the JMX connector, and how the facilities are connected
https://localhost:9443/IBMJMXConnectorREST/api . This gives you a small pull down to select what you want to display. For example
- “MBean info”gives “GET https://localhost:9443/IBMJMXConnectorREST/mbeans/{objectName}”
- “Query single attribute” gives “GET https://localhost:9443/IBMJMXConnectorREST/mbeans/{objectName}/attributes/{attribute}“
https://localhost:9443/IBMJMXConnectorREST/mbeans list all the beans

You may get data like

WebSphere%3Aname%3Dcom.ibm.mq.console.javax.ws.rs.core.Application %2C type%3DServletStats

The punctuation has been “escaped” so you need to change

%3A- to :
%3D to :
%2C to ,

and the string becomes the familiar

WebSphere: name=com.ibm.mq.console.javax.ws.rs.core.Application, type=ServletStats

Setting up Liberty(as used in mqweb) to use native JMX

Setting up the server side is well documented in the Oracle Monitoring and Management Using JMX Technology documentation. Using it from a client is not so well documented.

Server set up

The Liberty jvm.options file needs parameters. Note the port=9010 is used by clients accessing the data.

To provide insecure access from only the local machine

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=9010 
-Dcom.sun.management.jmxremote.local.only=true 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.ssl.need.client.auth=false

To provide securer access using TLS

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.ssl=true
-Dcom.sun.management.jmxremote.ssl.need.client.auth=true

# the following statements point to the same key store as
# used by mqweb server.   This could be different.
-Djavax.net.ssl.keyStoreType=PKCS12
-Djavax.net.ssl.keyStore=/home/colinpaice/ssl/ssl2/mqweb.p12
-Djavax.net.ssl.keyStorePassword=password
# the following statements point to the same trust store as
# used by mqweb server.   This could be different.
# if you used self signed certificates you could have a keystore
# just for the JMX users
-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/trust.jks
-Djavax.net.ssl.trustStorePassword=zpassword
-Djavax.net.ssl.trustStoreType=JKS

# The following defines the userid and password file
# Only the owner can have access to it
-Dcom.sun.management.jmxremote.password.file=/home/colinpaice/ssl/ssl2/jmxremote.password

# The following defines the access a userid can have
# Only the owner can have access to it
-Dcom.sun.management.jmxremote.access.file=/home/colinpaice/ssl/ssl2/jmxremote.access

jmxremote.password has

# specify actual password instead of the text password
monitorRole password
controlRole password

jmxremote.access has

# The "monitorRole" role has readonly access.
# The "controlRole" role has readwrite access.
monitorRole readonly
controlRole readwrite

Client set up

jconsole

You cannot pass a userid and password when the jconsole command, so you have to disable authentication in the jvm.options file

-Dcom.sun.management.jmxremote.authenticate=false

The parameters for jconsole have -J on them, as in -J-D…. . jconsole removes the -J and uses the rest of the parameters when invoking the JVM.

I could not get jconsole to recognize a config file using the -J-Dcom.sun.management.config.file = /path/to/jmxremote.properties , so I wrote a bash script to make it easier to change parameters.

ssl1="-Djavax.net.ssl.keyStore=/home/colinpaice/ssl/ssl2/colinpaice.p12"
ssl2="-Djavax.net.ssl.keyStorePassword=password"
ssl3="-Djavax.net.ssl.keyStoreType=pkcs12"
ssl4="-Djava.util.logging.config.file=/home/colinpaice/JMXQuery/java/logging.file"
ssl5="-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/trust.jks"
ssl6="-Djavax.net.ssl.trustStorePassword=zpassword"
ssl7="-Djavax.net.ssl.trustStoreType=jks"
ssl8="-J-Djavax.net.debug=ssl:handshake"
jconsole -J$ssl1 -J$ssl2 -J$ssl3 -J$ssl4 -J$ssl5 -J$ssl6 -J$ssl7 $ssl8 127.0.0.1:9010

The option “-J-Djavax.net.debug=ssl:handshake” gives a verb verbose trace of the ssl flows for the handshake.

The option -J-Djava.util.logging.config.file=/home/colinpaice/JMXQuery/java/logging.file enables the jconsole logging. I did not find the output very useful.

There is information the logger in general here, and on the file logger, here.

The logging.file had

Logging.properties

handlers= java.util.logging.FileHandler
// , java.util.logging.ConsoleHandler2

java.util.logging.FileHandler.pattern=/home/colinpaice/JMXQuery/java/log.%g.file
java.util.logging.FileHandler.limit=50000
java.util.logging.FileHandler.count=2
java.util.logging.FileHandler.level=ALL
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

// .level = INFO
// logger.level = FINEST
.level = FINEST
// Use FINER or FINEST for javax.management.remote.level - FINEST is
// very verbose...
javax.level= FINER
javax.management.level = FINER
javax.management.remote.*     = FINER 
javax.management.remote.level = FINER
javax.management.remote.misc.level  = FINER
javax.management.remote.rmi.level= FINER

Using jmxquery

I used a bash shell script to run the command, as it was easier to manage, and I could not find a way of having the java system properties in a file.

ssl1="-Djavax.net.ssl.keyStore=/home/colinpaice/ssl/ssl2/ibmsys1.p12"
ssl2="-Djavax.net.ssl.keyStorePassword=password"
ssl3="-Djavax.net.ssl.keyStoreType=pkcs12"les
ssl4="-Djava.util.logging.config.file=/home/colinpaice/JMXQuery/java/logging.file" 
ssl5="-Djavax.net.ssl.trustStore=/home/colinpaice/ssl/ssl2/trust.jks"
ssl6="-Djavax.net.ssl.trustStorePassword=zpassword"
ssl7="-Djavax.net.ssl.trustStoreType=jks"
ssl8="-Djavax.net.debug=ssl:handshake"
jar="-jar JMXQuery.jar"
user="-username monitorRole -password password"
url="-url service:jmx:rmi:///jndi/rmi://127.0.0.1:9010/jmxrmi"
parms=" -q   WebSphere:*  -count 2 -every 2"
java $ssl1 $ssl2 $ssl3 $ssl4 $ssl5 $ssl6 $ssl7 $ssl8 $jar $url $user $parms

Accessing JMX data in Liberty server, securely.

I thought I would complete the work I did with using JMX in the mqweb server. It was another example of Hofstadter’s Law:

It always takes longer than you expect, even when you take into account Hofstadter’s Law.

I spent a lot of time looking for things on the web, expecting them to be obvious, only to find that the things do not behave as expected. I could not find them, because they were not there. For example I expected to be able to configure the JMX server to use my OS userid and password. I could have a file with userids and passwords, or lookup in LDAP, but not my normal userid.

Getting started

I found there are two ways of getting the JMX data from the mqweb server.

Use of the native JMX support
Using the Liberty REST API

I think the REST API is easier to set up and is more secure.

I’ll document a high level overview of the two approaches, and how to configure them

Overview of using the native JMX support.

To use this, you configure parameters in the jvm.options file, including a port solely for JMX.

You can use TLS certificates to set up a secure link between the client and the server.

You can decide if you want to logon with userid and password. If you do you can set up

A file with userids and passwords; and a file with userids and permitted access. The documentation talks about userids like monitorRole and controlRole. You have to put a process in place to periodically change these passwords.
Use and LDAP server to do userid validation and to get the access.
I could not find how to use your operating system userid and password for authentication.
I could not find how to use the DN as authorization.

If your certificate is valid (either because it is signed by a CA, or there is a copy of a self signed certificate in the trust store), this is good enough for the checking. You can enable userid and password checking, but this solution feels weak, as you have to do extra work to manage it properly; you do not have a single signon. Not all tools support using userid and password, for example I could not pass userid and password on the jconsole command.

Overview of Using the Liberty REST API

As with the MQ REST API you can issue an HTTP request and get data back. See here. For example

curl –cacert ./cacert.pem –cert-type P12 –cert colinpaice.p12:password https://localhost:9443/IBMJMXConnectorREST/mbeans/WebSphere:name=com.ibm.mq.console.javax.ws.rs.core.Application,type=ServletStats/attributes=*

There is a small amount of configuration you need to do – less than with the native JMX support. The data comes back as JSON (as you might expect) and also includes a time stamp, which is very useful when post processing.

You define <administrator-role><user>..</user></administrator-role> in a similar way to setting up authorisation for mqconsole and mqrest. It takes the cn= value from your certificate as the userid, so you can give individual access.

“Securely” is a good laugh.

There are different levels of (in)security.

If you are using the native JMX support

You can have no passwords or access checks needed. The data is read only, and is not sensitive.
You can set up userid(s) and passwords in a file
You cannot use the operating system userid and password
You can use LDAP to check the userid and password, and get the role for that userid
You can use TLS, so anyone with a valid certificate can access the data
You can use TLS and use the userid and password in a file to determine access
You can use TLS and LDAP to get the role for that certificate

If you are using the WLP REST support

You can specify a userid and password
You can use a certificate, and the Common Name is used as the userid
You can specify in the configuration file, what access userids, or groups have

You can use TLS to protect your communications to and from the server.

Java leaks passwords

You need to be aware that your client machine may leak information. For example I ran a Java program to issue JMX requests from a script.

I could use the linux command ps -ef to display information about my request

ps -ef |grep JMX

gave me

colinpa+ 1871 1870 79 10:27 pts/2 00:00:01 java … -Djavax.net.ssl.keyStore=/home/colinpaice/ssl/ssl2/colinpaice.p12 -Djavax.net.ssl.keyStorePassword=password … -username monitorRole -password password …

This exposed the password to my keystore and password to my userid! I could not find a way of having all these java system parameters in a file.

I found export JAVA_TOOL_OPTIONS=”-D…” and this get picked up, but then java displays the variables as in Picked up JAVA_TOOL_OPTIONS: …

jconsole

Some programs have been designed to protect information for example jconsole you can put your system properties in a file

-J-Dcom.sun.management.config.file=ConfigFilePath

and so keep your parameters secure, but I could not get this to work.

Curl can be configured not to display parameters

With curl you have a command like

curl -n –cacert ./cacert.pem –cert-type P12 –cert colinpaice.p12:password

which gives away your password. If you do not specify it inline, you get prompted for it.

You can put your parameters in a config file, for example curl.config,

cacert ./cacert.pem 
cert ./colinpaice.pem:password 
key colinpaice.key.pem 
cookie cookie.jar.txt 
cookie-jar cookie.jar.txt 
url https://127.0.0.1:9443/ibmmq/rest/v1/login

and use

curl –config curl.config

Easy!

Protecting key files

It is important to protect the certificate file (with the important private key) so it is accessible by just the owner. The linux command ls -ltr colinpaice.p12 gives

-rw------- 1 colinpaice colinpaice 4146 Jan 31 17:56 colinpaice.p12

Of course anyone with super user authority has access to this file!

mqweb – displaying the secret statistics

Yes, mqweb does provide statistics; through the standard JMX interface provided as part of the base Liberty function. I expect most people do not know they are available. The data gets less useful over time, for example you get the “average time” since the mqweb started, rather than the last minute. See here on how to extract useful information from the data, and show useful averages.

You need in mqwebuser.xml .

<featureManager>
<feature>monitor-1.0</feature>
</featureManager>

and in jvm.options

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9010
-Dcom.sun.management.jmxremote.local.only=true
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

These options should be suitable in your test environment. You will want to change them for production.

You need the port number (9010 in the above example) when you extract jmx data.

How do you display the data?

For a quick sniff, (no good for extracting data and plotting charts) you can use jconsole. Use remote connection localhost:9010 . it does not display all of the data.

I found jmxquery very useful. I updated the github version to fix a bug which caused a loop. See here.

The query is

java -jar JMXQuery.jar -url service:jmx:rmi:///jndi/rmi://127.0.0.1:9010/jmxrmi -q ‘WebSphere:*’

You get data on

WebSphere:type=JvmStats
WebSphere:type=ThreadPoolStats,name=Default Executor
WebSphere:type=ServletStats,name=com.ibm.mq.console.javax.ws.rs.core.Application/AppName (String) = com.ibm.mq.console
WebSphere:type=ServletStats,name=com.ibm.mq.rest.javax.ws.rs.core.Application/AppName (String) = com.ibm.mq.rest

To get ‘console’ and ‘ResponseTime’ data I used

java -jar JMXQuery.jar -url service:jmx:rmi:///jndi/rmi://127.0.0.1:9010/jmxrmi 
-q 'WebSphere:*' 
-count 20 -every 60 
|grep --line-buffered console
|grep --line-buffered ResponseTime
|python3 mqweb.py

Where
java -jar JMXQuery.jar – invoke the program
-url service:jmx:rmi:///jndi/rmi://127.0.0.1:9010/jmxrmi – with this url and the above port number from the jvm.options
-q ‘WebSphere:*’ – give me only data for this components
-count 20 -every 60 – my extensions giving a record every 60 seconds, and doing 20 of them
|grep –line-buffered console – only pull out the console records, ( ignore the ‘rest’ records). The –line-buffered tells grep to flush it immediately
|grep –line-buffered ResponseTime – only interested in this detailed level
|python3 mqweb.py – pass it into the python program. This calculates the delta between records and prints out the count and mean value for that interval

If you are collecting data in real time from a stream, you need to ensure any processing is unbufferred. Often the default behavior is to accumulate the data in a big buffer, and write the buffer when it is full. Check any filters you use, for example grep –line-buffered.

One line of output from the JMXQuery program for the console activity is

WebSphere:type=ServletStats,name=com.ibm.mq.console.javax.ws.rs.core.Application/ResponseTimeDetails/count (Long) = 5

JvmStats

This comes under WebSphere:type=JvmStats/ See here.

FreeMemory (Long) = 13350536
ProcessCPU (Double) = 0.658040027605245
UsedMemory (Long) = 75131256
GcTime (Long) = 539
UpTime (Long) = 1048277
GcCount (Long) = 118
Heap (Long) = 88473600
FreeMemory (Long) = 13350536
ProcessCPU (Double) = 0.658040027605245
UsedMemory (Long) = 75131256
GcTime (Long) = 539
UpTime (Long) = 1048277
GcCount (Long) = 118

ThreadPoolStats

This comes under WebSphere:type=ThreadPoolStats,name=Default Executor/

PoolSize (Integer) = 8
ActiveThreads (Integer) = 2
PoolName (String) = Default Executor

See here.

com.ibm.mq.console and com.ibm.mq.rest

The data is similar between them. One has name=com.ibm.mq.console.javax.ws.rs.core.Application, the other has name=com.ibm.mq.rest.javax.ws.rs.core.Application

The data (in italics) with my comments in plain font are

AppName (String) = com.ibm.mq.console
RequestCountDetails/currentValue (Long) = 116
RequestCountDetails/description (String) = This shows number of requests to a servlet
RequestCountDetails/unit (String) = ns – this looks like a bug as it is a count not nanoseconds
RequestCount (Long) = 116
ServletName (String) = javax.ws.rs.core.Application
ResponseTimeDetails/count (Long) = 116
ResponseTimeDetails/description (String) = Average Response Time for servlet
ResponseTimeDetails/maximumValue (Long) = 3060146565 – in nanoseconds ( see below for the unit)
ResponseTimeDetails/mean (Double) = 8.796846855172414E7 – in nanoseconds
ResponseTimeDetails/minimumValue (Long) = 793871 – in nanoseconds
ResponseTimeDetails/standardDeviation (Double) = 4.198572684166255E8
ResponseTimeDetails/total (Double) = 1.0204342352E10 – in nanoseconds – used in calculations
ResponseTimeDetails/unit (String) = ns – this is the units. ns is nanoseconds
ResponseTimeDetails/variance (Double) = 1.64064538075292096E17 – used in calculations
Description (String) = Report Servlet Stats for specified Servlet and application.
ResponseTime (Double) = 8.796846855172414E7 – same as the ResponseTimeDetails

So we can see that there were

116 console requests since the mqweb server was started
the units are ns (nano seconds)
the console requests taking an average of 8.796846855172414E7 nanoseconds, 0.0879 seconds with
a standard deviation of 4.136787844033974E7 – nanoseconds = 0.04198 seconds
the maximum value was 3 060 146 565 ns = 3.060 seconds
the minimum time was 793 871 ns or 0.000793 seconds

Some other data, showing how it changed over time

Data	Values	Later values	Much later values
Number of requests	82	3590	22920
Average (seconds)	0.158	0.0108	0.0099
Standard deviation (seconds)	0.487	0.034	0.035
Maximum (seconds)	2.3	2.3	2.3
Minimum(seconds)	0.001	0.0001	0.0001

Notes:

There is data only once a request has been processed, so if you have not run a rest request, there will be no JMX data for rest activity.
These values are from start of the mqweb server. I did not see them reset, so you could have a data for a whole week or more.
The maximum was from the first requests to run. I expect this includes the “warm up” costs, of loading the code and JITing it.
The average values are from the start, so will be impacted by peaks and troughs.

For the each mqconsole window, there are two console counts every 10 seconds. Any charts are refreshed every 10 seconds, so I think this is a “I am still here, please send me any data you have for me”.

Data for rest

I started my mqweb server, and ran a python program which opened a connected and got three messages.

Maximum time 0.3486
Minimum time 0.0026
Calculate the other one 0.0028

Because the first request takes a long time, you can adjust for this in your calculations to get a truer mean.

For example

I reran the script and processed 100 messages. The average time of these was 0.003 seconds.

Maximum 0.3486
Mean 0.00637
Count 103
Total 0.656

The calculations are

Mean * count = 0.656 (which matches Total as expected)
Subtract the maximum, first time value 0.656 – 0.349 = 0.307
Calculate the improved mean value ignoring the first value, 0.307 /(103 -1) = 0.003

So the adjusted mean time is 0.003 seconds – compared to the 0.006 which the JMX stats report.