Category: MQ on z/OS

Problems with mqinqmp

I struggled to get MQ in Python to work with Message properties when running on z/OS. I eventually found that mqinqmp() in 64 bit mode was not always working (31 bit mode was fine).

Also the documentation for mqinqmp() is not very clear.

You pass to mqinqmp(), a property name (or a generic like %), a buffer, the size of the buffer, the length of the data, and some other parameters (such as get first, or get next). It returns the property name, the property value ( possibly truncated) , and the true length of the property..

If your buffer is too small to contain all of the property, you get reason code MQRC_PROPERTY_VALUE_TOO_BIG, and should get the true length of the property, its name, type, and length.

This worked fine when it was a 31 bit program. When I ran it as a 64 bit program, it did not return the true length, and did not return the property name.

The documentation says

DataLength Type: MQLONG – output

This is the length in bytes of the actual property value as returned in the Value area.

If DataLength is less than the property value length, DataLength is still filled in on return from the MQINQMP call. This allows the application to determine the size of the buffer required to accommodate the property value, and then reissue the call with a buffer of the appropriate size.

The DataLength is the actual length of the property. (The value returned in the Value area is truncated at Value Length).

What should happen

After you get the MQRC_PROPERTY_VALUE_TOO_BIG reason code, you reallocate the buffer with a size of DataLength and retry the request. It should work the second time.

For me the DataLength was 0, so every time around the loop it got MQRC_PROPERTY_VALUE_TOO_BIG.

Understanding PCF cmdscope on z/OS

I wanted to check the response to a PCF command when cmdscope=”*” was issued.

It took a while to understand it. It feels like it was implemented by different people with different strategies. I’ve documented what I found in case someone else struggles down this path. As an extra challenge I used Python to issue the command and process the responses.

High level view of the cmdscope reply

A PCF request is sent to a server, and responses are turned to the specified reply-to-queue. These reply messages all have the same correlid, but different msgids.

  • The first message gives information about the remainder of the messages, and how many queue managers are involved.
  • The last message is a message saying “finished”, and loosely ties up with the first messages.
  • In between there are sets of messages from each queue manager. There are one or more messages in the set. Each set has its own identifier.

More information

The first message

  • This has a Response id – it is not used anywhere else.
  • Says there replies from N queue manager. The value of N is given.
  • There is a Response Set structure (A) giving the Response id for the “end of command scope” message.
  • There are one or more Response Set structures giving the Response id for each queue manager. There are N of these structures.

The last message has a Response-id which matches the Response set (A). The content of this last message is “the request (with command scope) has finished”.

Each queue manager returns a set one or more messages.

  • All messages in the set have a Response-id for the set; which is given in the first message above. The name of the queue manager providing the information is given.
  • The last message in the set has the “Last message in the set” flag set in the PCF header.

Response-id values

Each message has a Response-id field. This has a value like “CSQ M801 GM802…” where the … is hex data. The original request was to queue manager M801, and in this case the response was for queue manager M802. There are several field that look similar to this, for example “CSQ M801 RM801…”. The content of this response is not an API.

How many messages should I expect?

The answer is “it depends”.

  • You will get a matching first and last message (2)
  • You will get a set of messages for each queue manager
    • There will be one or more messages with data.
    • The number of messages with data varies. For example the Inquire archive request returns returns the values at startup. Sysp Type : Type Initial.
    • If a Set Archive command has been issue, a second record with Sysp Type : Type Set, will be present, which shows the parameters which have been changed since startup.
    • A last in set message.

You know when you have the last message when you have received the “the request (with command scope) has finished” message.

At a practical level…

I wrote code like that below, to get all the messages in a request. If is more complex than written as the PCF body may not have the structure with the data type.

cmdscopeloop = no
_______________

Loop: Get message with PCF header and PCF body
## If cmdscope was specified, keep looping until end cmdscope message
If header.type ==MQCFT_XR_MSG and
body[MQIACF_COMMAND_INFO] == MQCMDI_CMDSCOPE_ACCEPTED
then cmdscopeloop = yes

If header.type ==MQCFT_XR_MSG and
body[MQIACF_COMMAND_INFO] == MQCMDI_CMDSCOPE_COMPLETED
then cmdscopeloop = no
## Requests like inquire chinit have a command accepted
If header.type ==MQCFT_XR_MSG and
body[MQIACF_COMMAND_INFO] == MQCMDI_COMMAND_ACCEPTED
then state = “CommandAccepted”

## Most messages have
if header.type ==MQCFT_XR_ITEM
then state = “Item”

## The summary record means end of set.
##For many requests this is end of data
if header.type ==MQCFT_XR_SUMMARY
then state = “endset”

## see if we need to keep looping.
if state == “endset” and cmdscopeloop == no
then return
else loop to get more messages

The request and detailed responses

I used PCF with the command MQCMD_INQUIRE_ARCHIVE, cmdscope=”*”. I send the request to queue manager M801. There were three queue managers in the QSG: M801, M802, M803.

The data description like “Response Q Mgr Name”, is decoded from the PCF value, and post processed to make it more readable, using the MQ sample code (provided on MQ Midrange).

Values like b’M801′ is a Python byte string, or hexadecimal string. Other character data may be in UTF-8 code page.

Response (1) meta information

PCF.Type 17 MQCFT_XR_MSG
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 1
PCF.Control 1 Last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 8

  1. Response Id : CSQ M801 RM801…
  2. Response Q Mgr Name : b’M801′
  3. Command Info : Cmdscope Accepted
  4. Cmdscope Q Mgr Count : 3
  5. Response Set : CSQ M801 SM801… This is for the final “end of cmdscope” message
  6. Response Set : CSQ M801 GM801… These are for the data returned from a queue manager
  7. Response Set : CSQ M801 GM802…
  8. Response Set : CSQ M801 GM803…

Response (2), for queue manager M801. The initial values

PCF.Type 18 MQCFT_XR_ITEM
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 1
PCF.Control 0 Not last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 19

  1. Response Id : CSQ M801 GM801… Matching a response set value in the first message
  2. Response Q Mgr Name : b’M801′
  3. Sysp Type : Type Initial
  4. Sysp Archive Unit1 : b’TAPE’
    ….

19. Sysp Quiesce Interval : 5

Response (3), for queue manager M801. The fields which have been set

On queue manager M801 the command SET ARCHIVE UNIT(DISK) was issued. If the DISPLAY ARCHIVE command is issued it will display DISK under the “SET value” column.

PCF.Type 18 MQCFT_XR_ITEM
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 2
PCF.Control 0 Not last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 4

  1. Response Id : CSQ M801 GM801…
  2. Response Q Mgr Name : b’M801′
  3. Sysp Type : Type Set
  4. Sysp Archive Unit1 : b’DISK’

Response(4), for queue manager M801. End of queue manager’s data

PCF.Type 19 MQCFT_XR_SUMMARY
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 3
PCF.Control 1 Last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 2

  1. Response Id : CSQ M801 GM801…
  2. Response Q Mgr Name : b’M801′

Response (5), for queue manager M802. The initial values

PCF.Type 18 MQCFT_XR_ITEM
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 1
PCF.Control 0 Not last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 19

  1. Response Id : CSQ M801 GM802…
  2. Response Q Mgr Name : b’M802′
  3. Sysp Type : Type Initial
  4. Sysp Archive Unit1 : b’TAPE’
  5. …

Response (6), for queue manager M802. End of queue manager’s data

PCF.Type 19 MQCFT_XR_SUMMARY
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 2
PCF.Control 1 Last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 2

  1. Response Id : CSQ M801 GM802…
  2. Response Q Mgr Name : b’M802′

Response (7), for queue manager M803. The initial values

PCF.Type 18 MQCFT_XR_ITEM
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 1
PCF.Control 0 Not last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 19

  1. Response Id : CSQ M801 GM803…
  2. Response Q Mgr Name : b’M803′
  3. Sysp Type : Type Initial
  4. Sysp Archive Unit1 : b’TAPE’

…

Response (8), for queue manager M803. End of queue manager’s data

PCF.Type 19 MQCFT_XR_SUMMARY
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 2
PCF.Control 1 Last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 2

  1. Response Id : CSQ M801 GM803…
  2. Response Q Mgr Name : b’M803′

Response (9), for queue manager M801. End of command.

PCF.Type 17 MQCFT_XR_MSG
PCF.Command 114 MQCMD_INQUIRE_ARCHIVE
PCF.MsgSeq 1
PCF.Control 1 Last message in set
PCF.CompCode 0
PCF.Reason 0
PCF.ParmCount 3

  1. CSQ M801 SM801…
  2. Response Q Mgr Name : b’M801′
  3. Command Info : Cmdscope Completed

What does MQRCCF_PARM_CONFLICT mean?

I’ve been using PCF (from Python) and have been getting MQRCCF_PARM_CONFLICT. For example use MQCMD_CHANGE_Q_MGR and set MQIA_TCP_CHANNELS to 201.

The documentation does not help. It says

Explanation

Incompatible parameters or parameter values.

The parameters or parameter values for a command are incompatible. One of the following occurred:

  1. A parameter was not specified that is required by another parameter or parameter value.
    • The MQIA_TCP_CHANNELS can be used on its own – so it is not this one.
  2. A parameter or parameter value was specified that is not allowed with some other parameter or parameter value.
    • The MQIA_TCP_CHANNELS was specified on its own – so it is not this one.
  3. The values for two specified parameters were not both blank or non-blank.
    • Only one parameter was specified – so not this one.
  4. The values for two specified parameters were incompatible.
    • Only one parameter was specified – so not this one.

I found the problem by issuing the command and seeing what the response was.

QM01 ALTER QMGR TCPCHL(201)

gave

CSQM150I QM01 CSQMAMMS ‘TCPCHL’ AND ‘MAXCHL’ VALUES ARE INCOMPATIBLE
CSQ9023E QM01 CSQMAMMS ‘ ALTER QMGR’ ABNORMAL COMPLETION

So I would add reason

5. The specified value is inconsistent with the configuration.

How do I use SFTP to ftp to a z/OS data set?

You cannot do it directly, you have to do a two step process.

Sending files to z/OS using SFTP

SFTP can send files to Unix Services Subsystem. It cannot send to data sets.

sftp colin@10.99.88.77

You can use commands like

  • cd to change directory on the remote system
  • lcd change directory on the local system
  • put
  • get
  • chmod
  • chown
  • exit

There is no command “bin” nor “quote…”.

Getting from a Unix Services file to a dataset.

You can use the cp command.

To copy a binary file to a dataset

For example as if you were using FTP with BIN; quote site cyl pri=1 sec=1 recfm=fb blksize=3200 lrecl=80; put mp1b.load.xmit ‘COLIN.MP1B.LOAD.XMIT’)

cp -W “seqparms=’RECFM=FB,SPACE=(500,100),LRECL=80,BLKSIZE=3200′” mp1b.load.xmit “//’COLIN.MP1B.LOAD.XMIT'”

Where the seqparms are in upper case. If they are in mixed case you get

FSUM6258 cannot open file “… “: EDC5121I Invalid argument.

I could then do TSO RECEIVE INDSN(‘COLIN.MP1B.LOAD.XMIT’).

To copy a text file to a data set

If you use SFTP to copy a text file to Unix Services, it gets sent in bin, and, on z/OS, looks like

ñà ÈÇÁ ËøÁÄÑÃÑÁÀ….

You can tag a file so Unix Services knows it is an ASCII file, using

chtag -tc ISO8859-1 aaa

This makes the file editable from Unix Services, but you cannot just use cp to copy and create a dataset, as above.

You can convert it from ASCII to EBCDIC using

iconv -f ISO8859-1 -t IBM-037 ascii_file  > ebcdic-file

Then use

cp -W “seqparms=’RECFM=VB,SPACE=(CYL,(1,1)),LRECL=800,BLKSIZE=8000′” ebcdic-file “//’COLIN.EBCDFILE'”

Setting up SMDS in MQ on z/OS

Setting up and using SMDS

MQ on z/OS provides shared queue, where a coupling facility structure is shared amongst queue managers in a sysplex to provide a queue sharing group.

You can set up Shared Message Data Set(SMDS) to handle large messages over 63 KB in size, or to provide additional capacity, so as the structure fills up messages are written to a data set – but available from all queue managers in the QSG. Ultimately the structure just has pointers to messages on the data set.

For messages under 63 KB in size, instead of using SMDS you can use the z/OS structure overflow called Storage Class Memory. I think of this as paging for a structure. If you have many messages on a queue it will “page out” the messages in the middle out of the queue, and as the queue is processed sequentially it will “page in” in the messages before they are needed.

There is lots of good information on SMDS, but I found it hard to locate the information I needed.

This blog post aims to fill some of the holes.

Before you start

The term CFLEVEL is used in two contexts, MQ and z/OS

  • z/OS Coupling Facility Resource Management (CFRM) uses a CFLEVEL. For example ,the CFRM definitions include if the structure is duplexed. If CFRM CFLEVEL > 21 then it supports asynchronous duplexing.
  • MQ has a CFLEVEL. MQ CFLEVEL(5) allows offload, and allows the queue manager to tolerate a loss of connectivity to the CF structure.

SMDS requires the CF structures to have MQ CFLEVEL(5). This is not a major change, but needs to be planned. Increasing the MQ CFLEVEL in the z/OS CF structure definition, will probably increase the size of the structure.

In “Backing up and recovering data in SMDS” below, it discusses having a queue manager within the QSG just for backing up and recovering messages in structures and especially when there are a large number of messages in SMDS. You need to consider this, and decide if you want to implement it. This can be done at any time. You may need to review how often you backup your MQ CF structures.

Backing up (and recovering) data in SMDS

Unlike MQ pagesets you do not recover an SMDS by restoring it from a backup. The CF structure and the SMDS are one logical entity, with the structure pointing to the location on disk. If you restore a backup of an SMDS, the CF will not have the matching information and you will have corrupt data.

Instead of backing up the SMDS you use the MQ BACKUP CFSTRUCT command. This reads the CF and any messages on the SMDSs, and writes the content to the log data sets. Ideally you do this when the CF structure is close to empty. If you have a large SMDS full of messages, this will write a lot of data to the active logs, possibly impacting normal throughput.

When the RECOVER CFSTRUCT command is issued (to recreate the data) , the log is read to recover the data. If you have normal message data logged, this could mean that a lot of log data needs to be read to recover the structure, because the normal message traffic and the backup data will be intermixed on the log.

Some customers have a queue manager dedicated to backing up the CF Structures. This has the following advantages:

  • The backup activity writing to the active log does not impact normal business activity
  • When recover cfstruct is issued, only the data for CF structures is in the logs, and so less data overall needs to be read from the logs, which means that recovery is faster.

Create the SMDS

Each queue manager needs its own SMDS, the queue manager can read and write to its own SMDS.

A queue manager can read the data in another queue manager’s SMDS, (but updates are only made from the owning queue manager).

The SMDS can be of different sizes, for example if two queue managers are “customer facing”, and two queue managers are “back end facing” the “customer facing” SMDS may get a lot of activity, and the “back end facing” may get no activity because these queue managers only do gets.

You allocate an SMDS with a primary size, and can specify a secondary size. If a secondary size is zero or not specified, and expansion is allowed, then the queue manager will try expanding with a secondary space of 10% of the existing size. This means the 100th extent will be larger than the 10th extent.

An SMDS can have up to 255 extents. If the data set is SMS managed and the SMS configuration has Extent Constraint Removal specified in the DFSMS Data Class, then the data set can be on up to 59 volumes, with up to 123 extents per volume (or 7257 extents). See here for more information.

You cannot make the SMDS smaller.

If enabled, automatic expansion is attempted when the SMDS is 90% full. If the message rate is not too high then the automatic expansion may be enough so the SMDS does not get 100% full.

If there is a spike in workload, then the expansion may not be able to keep up, and so some requests will get no space available, until any in-progress expansion has completed. In this case to avoid applications getting the “no space” condition you should allocate the SMDS with a primary extent big enough to hold the expected peak of messages, or allocate a queue on the CF structure, and put many messages to cause expansion (then drain or delete the queue).

Note: The time taken to expand the SMDS depends on the size of the increment. Larger extents mean there are more pages to format, and so take longer.

Allocating the data set

When the CF Structure is not full, then most of the messages may be in the CF Structure. As the CF becomes fuller, more messages are written to the SMDS. If all messages are on an SMDS, and there is a high message rate, the SMDS will have a lot of I/O activity, read and write.

You need to allocate the SMDS on fast disks to minimise response time of processing messages.

If you are not using SMS managed data sets, you need specify additional volumes to allow for expansion.

What logical block size should be used?

You need to know the profile of the size of your messages so you know what value to use. Each page within SMDS is 4KB. The block size defines how many of these 4KB pages are allocated to each message. I/O is done at the block level.

The default DSBLOCK is 256K.

Once an SMDS has been opened the DSBLOCK value cannot be changed, so you need to plan this before you start to use the SMDS.

If you find the value is not optimum, then

  • if the value is too large, the impact is more disk space will be used and more 64 bit buffers are used.
  • if the value is too small, then more I/O operations (of smaller blocks) will be done, and the time to process messages may take longer (possibly just a few milliseconds).

If you have big messages then a DSBLOCK value of 256K may be OK. If you have many small messages (under 64K) then 64K or smaller may be better.

Defining SMDS to MQ

If you define a new CF Structure and specify CFLEVEL(5) then the offload parameter defaults to SMDS.

If you alter an existing CF Structure from a lower CFLEVEL to CFLEVEL(5) the OFFLOAD defaults to DB2. This was done for migration reasons.

Once you have created your SMDS for each queue manager. You use the

  • alter cfstruct.(..) cflevel(5)
  • alter cfstruct(..) DSGROUP(‘data.set.name.*’) DSBLOCK(…)
  • alter cfstruct(…) offload(SMDS)

The number of buffers (of size DSBLOCK) is specified in DSBUFS. This can be set on a queue manager using the ALTER SMDS DSBUFS command.

This value can be changed dynamically, it defaults to 100.

You need to work with the z/OS systems programmer before using this to make sure there is enough resources. For example I defined the block size as 1MB, and defined 9999 buffers. This is over 9 GB of virtual storage and z/OS ran out of auxiliary storage and real storage. I had to shutdown the QSG and reset the size in CSQINP2.

When reading from a different queue managers’ SMDS, a buffer is allocated from the pool, data is read in, and the buffer is released back to the pool.

When a queue manager is writing to its own SMDS, a buffer is obtained, written to the SMDS and the buffer released(still with its content). If an MQGET is issued, the data may be in a buffer, and so avoid disk I/O.

Generally SMDS works ok with a small number of buffers – as long as there are enough buffers for the concurrent requests. Having more buffers may make little difference to performance. You need to look at the statistics to see if there were buffer shortages.

Other SMDS activities

If you need to work with the SMDS, you can use the RESET SMDS command to disable access to the SMDS. Message processing which does not use the SMDS can continue.

The RESET SMDS ACCESS(DISABLED) causes all of the queue managers to close it normally and deallocate it. When the data set is ready to be used, it can be altered to ACCESS(ENABLED) allowing the queue managers to access it again.

Media recovery

If there is a problem with shared queue, the CF structure may need to be rebuilt, and the messages restored from the log data sets.

One queue manager does the Media recovery. This needs access to the log of all the queue managers in the QSG, and needs update access to the SMDS data sets.

Automatic expansion

You can enable automatic expansion of the SMDS using DSEXPAND(YES) .

You can disable automatic expansion of the SMDS using DSEXPAND(NO). You may want to do this is you have an exceptional peak, and do not want the SMDS to expand(because you cannot make an SMDS smaller). If the CF structure or the SMDS is full a put request will get no space available reason code.

Automatic expansion may fail

  • If the number of extents exceeds the limit
  • If there is no space available to expand. You may be able to fix this by making more volumes available to be used. You may need to talk to your data manager about the best way to resolve this problem.

What to monitor?

SMF

You should collect SMF data from a good day, so if you have a bad day, you have some comparison data.

SMDS writes statistics data to SMF. A program to print the SMF data, is available in SupportPac MP1B. This describes the fields, and how to interpret the data. There are other tools which format the data, but they do not tend to interpret the data and give useful information.

Information available includes

  • I/O requests to an SMDS, write to local, read from local, read from other qmgr’s SMDS
    • count of requests,
    • number of pages,
    • average I/O duration,
    • average wait before the I/O could be started.
  • Numbers of buffers
    • Total,
    • In use,
    • Current count of waiting for a free buffer,
    • Highest count in the interval of waiting for a free buffer,
    • Current count of waiting for a busy buffer,
    • Highest count in the interval of waiting for a busy buffer,
    • Number of times there were no buffers available.

Display command

SupportPac MP16: Capacity Planning and Tuning for IBM MQ for z/O describes SMDS. It gives information on DSBLOCK and impact of message size, and other good information.

If there are insufficient buffers to handle the maximum concurrent number of I/O requests, then requests will have to wait for buffers, causing a significant performance impact. This can be seen in the CSQE285I message (issued as a response to the “DISPLAY USAGE TYPE(SMDS)” command) when the “lowest free” is zero or negative and the “wait rate” is non-zero. If the “lowest free” is negative, increasing the DSBUFS parameter by that number of buffers should avoid waits in similar situations.

MP16

The number of extents

You can use MQ commands (or MQ SMF data) to monitor the size of SMDS. You can also automate the expansion messages as expansion should be an exception rather than an every day occurrence.

You might prepare for SMDS expansion deciding in advance if you want to use SMDS expansion, or disable expansion, and prepare the system operations to handle this.

MQ commands relating to SMDS

Some of the commands for SMDS are dissimilar to normal MQ commands. For example

DIS smds(*) CFSTRUCT(CSQSYSAPPL)

gets a reply from all queue managers in the QSG. You can issue the following command from queue manager QM01, about queue manager QM02

QM01 DIS SMDS(QM02) CFSTRUCT(CSQSYSAPPL)

Or you could just issue the command to QM02

QM02 DIS SMDS(QM02) CFSTRUCT(CSQSYSAPPL)

I forget this every time!

  • ALTER CFSTRUCT. Define OFFLOAD parameters, data set name, thresholds for offloading messages to SMDS. Change numbers of buffers for QSG specific value, the number of buffers for individual queue managers, can be overwritten by ALTER SMDS.
  • ALTER SMDS. Change the number of buffers for a queue manager; Change DSEXPAND().
  • DISPLAY CFSTATUS. Display the status of one or more CF application structures.
    • DISPLAY CFSTATUS TYPE(SMDS). Display shared message data set information. You get a reply from each queue manager.
  • DISPLAY CFSTRUCT. Display the attributes of one or more CF application structures.
  • DISPLAY SMDS. Display the parameters of existing SMDSs associated with a specified application structure.
  • DISPLAY SMDSCONN. Display status and availability information about the connection between the queue manager and the shared message data sets for the specified CFSTRUCT
  • RECOVER CFSTRUCT. Recreate the persistent messages in the CF (and SMDS) from the logs.
  • RESET SMDS. Modify availability or status information relating to one or more shared message data sets associated with a specific application structure. For example make SMDS not available to queue managers, while it is made larger.

Hell is PCF.

In Dante’s book “Inferno”, the gates of hell have “Abandon all hope, ye who enter here” inscribed upon them. The book describes Dante’s journey to the deepest levels of hell.

I had a similar journey, not so far, trying to use PCFs in common code.

I thought I knew how to use MQ PCFs to display and extract information about MQ and its objects, but “fools rush in where angels fear to tread”.

Thanks to Morag Hugson of MQGEM for her help in getting me through this journey.

The kindergarten course on using PCF

Do not use PCF, use the display commands and parse the output. Parsing the output is a simpler problem to solve than trying to use PCF.

The high school course.

Instead of using a command DIS Q(…) you use PCF; create a message with a series of control blocks, send the request to a system queue, and get the data back in a similar format.

There is a header control block to define the function, and 0 or more control blocks each of which define a field.

  • The first control block, the header block, defines the request. It has an integer request field, such as MQCMD_INQUIRE_Q.
  • The second and following control blocks are field control blocks, they have
    • A field to define the type of data, for example character string, byte string (may contain hex 00s), integer, 64 bit integer, or arrays of data.
    • A field to define which field is defined, for example the integer MQCA_Q_NAME
    • For strings and byte strings, the length of the data.
    • For strings the code page of the data.
    • The data.

The request is put to a system queue and the response comes back in the reply_to queue you specified. This has a similar format as the request:

  • There is a header giving the request type, (MQCMD_INQUIRE_Q), the condition and reason code, and a count of the fields returned.
  • There are 0 or more field control blocks – the same as above.

Your program can chain down the field control blocks and can display the fields.

The program from the first year high school will report data like

field type 20 has value 1

which is not very usable.

Using the supplied header file CMQSTRC, you can call MQIA_STR(20) and get back “MQIA_Q_TYPE”. So now you know this field is the Queue Type data.

Now you know it is a Q_Type you can use the function MQQT_STR(1) and get back the string “MQQT_LOCAL”. You can determine the queue type value as a string using

if (field_type=MQIA_Q_TYPE ) 
    queue_type= MQQT_STR(value)
else if (field_type=MQIACF_PUBSUB_PROPERTIES,)                
    property_type=MQPSPROP_STR(value)
else if (field_type...)

You need a bit of code to match up the field type to the routing you need to call to get the data back. I am working on a function getValue(20,0) to get back “MQIA_Q_TYPE”, and “MQQT_LOCAL”. One call which returns, the data type and its character representation of the value, which will make life much easier for the programmer. There are over 200 field types and it is hard to find the mapping for the value.

In early days of MQ there were two choices of header. You issued a header with type MQCFT_COMMAND and got back one reply with a header type MQCFT_RESPONSE, and all of the data.

Not too difficult so far.

Going down to the next level of hell with z/OS. The request “inquire_chinit” goes to the command server, which sends back “Request received”. The command server then sends the request to the chinit, which sends the response back to the command server, which sends it on to your reply to queue.

You now have to manage at least two reply messages, one for the “Request received”, and one or more for the data.

The next complication which takes you down to the next level of hell, is Shared Queue on z/OS.

You can issue the PCF request saying “hello all queue managers, please give me information about the queue status as you see it for queue COLINPAICEQ”. This request can now go to multiple queue managers, and you get multiple replies back which you have to manage.

This is not well documented in the official documentation. It jumps straight to the PhD class.

I’ll try to explain the data you get back with some examples.

Example 1 Inquire queue manager.

I sent the PCF request with MQCMD_INQUIRE_Q_MGR.

Two messages were returned. They had the same msgid and correlid

The first message was type MQCFT_XR_ITEM with sequence number 1, the second message was MQCFT_XR_SUMMARY with sequence number 2.

First message

  • Type MQCFT_XR_ITEM: Message is an extended response to an Inquire command. It contains item data.
  • Command Request MQCMD_INQUIRE_Q_MGR
  • Condition code 0
  • Reason code 0
  • Sequence number 1
  • It is not last message in set.
  • Parameter count 107

Data fields

  1. MQBACF_RESPONSE_ID … a 24 character string with hexadecimal values. Think of this as a correlator field, so you can match all the responses for this request.
  2. MQCACF_RESPONSE_Q_MGR_NAME : ‘CSQ9’. This is the queue manager which sent the response.
  3. MQCA_Q_MGR_NAME : ‘CSQ9’
  4. MQCA_Q_MGR_DESC : ‘CSQ9, IBM MQ for z/OS – V9.0.1’
  5. MQIA_PLATFORM : MQPL_ZOS
  6. MQIA_CPI_LEVEL : 100
  7. MQIA_COMMAND_LEVEL : MQCMDL_LEVEL_924
  8. …

Second message

Header

  • Type: MQCFT_XR_SUMMARY . Message is an extended response to a command. It contains summary information.
  • Request INQ_QMGR
  • Condition code 0
  • Reason code 0
  • Sequence number 2
  • It is the last message in set.
  • Number of parameters 2
  1. MQBACF_RESPONSE_ID: …. same as above.
  2. MQCACF_RESPONSE_Q_MGR_NAME: ‘CSQ9’

These two fields are used to correlate requests. I think …XR_SUMMARY really means end of messages in this set.

Example 2 Inquire chinit.

I sent the PCF request with MQCMD_INQUIRE_CHANNEL_INIT.

Five messages were returned. They all had the same msgid and correlid

The first response message was type MQCFT_XR_MSG with sequence number 1.

The the remaining four messages were part of one set. They had

  1. Queue manager information, such as number of dispatchers.
  2. Information about the TCP/IP listener
  3. Information about the LU 62 listener
  4. The “end” or summary record

First set, single message

  • Type 17 MQCFT_XR_MSG: Message is an extended response to a command. It contains informational or error details.
  • Command MQCMD_INQUIRE_CHANNEL_INIT
  • Condition code 0
  • Reason code 0
  • Sequence number 1
  • It is the last message in set.
  • 4 field control blocks were returned.

Field control blocks

  1. MQBACF_RESPONSE_ID: Value: 24 bytes of data (with the queue manager name in it
  2. MQCACF_RESPONSE_Q_MGR_NAME: 48 character queue manager name
  3. MQIACF_COMMAND_INFO : MQCMDI_COMMAND_ACCEPTED
  4. MQBACF_RESPONSE_SET: Same value as the first field control block

Set 2, first message

Header

  • Type: MQCFT_XR_ITEM: Message is an extended response to an Inquire command. It contains item data.
  • Request Command MQCMD_INQUIRE_CHANNEL_INIT
  • Reason code 0
  • Sequence number 1
  • It is not the last message in set.
  • Parameter count 20.

Field control blocks – information about the chinit

  1. MQBACF_RESPONSE_ID: Value: 24 bytes of data (with the queue manager name in it)
  2. MQCACF_RESPONSE_Q_MGR_NAME: 48 character queue manager name
  3. MQIACF_CHINIT_STATUS : MQSVC_STATUS_RUNNING
  4. MQIACH_ADAPS_STARTED : 8
  5. MQIACH_ADAPS_MAX :8
  6. MQIACH_DISPS_STARTED :5
  7. MQIACH_DISPS_MAX :5
  8. …

Set 2, second message

Header

  • Type: MQCFT_XR_ITEM Message is an extended response to an Inquire command. It contains item data.
  • Request Command MQCMD_INQUIRE_CHANNEL_INIT
  • Reason code 0
  • Sequence number 2
  • It is not the last message in set.
  • Parameter count 7.

Field control blocks – status of TCP listener

  1. MQBACF_RESPONSE_ID …
  2. MQCACF_RESPONSE_Q_MGR_NAME : b’CSQ9′
  3. MQIACH_LISTENER_STATUS : MQSVC_STATUS_RUNNING
  4. MQIACH_XMIT_PROTOCOL_TYPE : MQXPT_TCP
  5. MQIACH_INBOUND_DISP: MQINBD_Q_MGR
  6. MQIACH_PORT_NUMBER :1414
  7. MQCACH_IP_ADDRESS : b’*’

Set 2, third message

Header

  • Type: MQCFT_XR_ITEM Message is an extended response to an Inquire command. It contains item data.
  • Request Command MQCMD_INQUIRE_CHANNEL_INIT
  • Reason code 0
  • Sequence number 2
  • It is not the last message in set
  • Parameter count 5.

Field control blocks – status of LU62 listener

  1. MQBACF_RESPONSE_ID ….
  2. MQCACF_RESPONSE_Q_MGR_NAME : b’CSQ9′
  3. MQIACH_LISTENER_STATUS : MQSVC_STATUS_STOPPED
  4. MQIACH_XMIT_PROTOCOL_TYPE : MQXPT_LU62
  5. MQIACH_INBOUND_DISP MQINBD_Q_MGR

Set 2, fourth (and last) message

Header

  • Type: 18 MQCFT_XR_SUMMARY Message is an extended response to a command. It contains summary information.
  • Request Command MQCMD_INQUIRE_CHANNEL_INIT
  • Reason code 0
  • Sequence number 4
  • It is the last message in set.
  • Parameter count 2.

Field control blocks

  1. MQBACF_RESPONSE_ID …
  2. MQCACF_RESPONSE_Q_MGR_NAME : ‘CSQ9’

I do not have access to a QSG, so cannot document the records for a QSG wide query.

Creating the data fields

If you want to create a PCF command to change a queue, you need the PCF header to describe the command, and data fields to identify the queue, and changed attributes.

The header is defined with control block MQCFH (not MQPCFH, I keep trying to use). There are several fields, the key one is Command. In this case we need MQCMD_CHANGE_Q. All request begin with MQCMD. There is MQCMD_INQUIRE_Q for displaying information about a queue.

There is a good post on PCF fields, called Morag’s Quirk’s #14: MQCFH Versions which contains some good information on data types and their usage.

There is a variety of data field record types, such as Byte string, Character string, Integer; and different usages

The field can be

  • passed with the command, (a selector), and returned. For example MQIA_, MQCA_
  • returned (but not passed through with a command); for queue manager for example MQCACF, and MQIACF) and for the CHINIT for example (MQCACH and MQIACH)

or to look at it a different way…

  • MQIA_ are the set used on MQINQ – and thus defined in cmqc.h. They can also be used in PCF rather than defining a second ID for the same parameter.
  • MQIACF_  are the set used solely for PCF – and thus defined in cmqcfc.h. They are not used on MQINQ.
  • MQIACH_ are the set used for channel commands, they are defined in cmqcfc.h.

The same logic is for for MQCA_/MQCACF_/MQCACH_.

Each field has a unique number, and the field types are generally grouped within a range, but not all of them. The monitoring integers and 64 bit integers are all mixed up.

To find the string representation of the unique number you can call the formatting functions (*MQBACF_STR, *MQCA_STR, *MQCACF_STR, *MQCACH_STR, *MQIA_STR, *MQIACF_STR, *MQIACH_STR, *MQIAMO_STR, *MQIAMO64_STR) in CMQSTRC.h. These routines return a string matching the number or the empty string ”. I found I have to go through them each in turn until there is a match.

In more detail.

  • Individual fields
    • A byte string (MQCFBS) . This is a string which can include hexadecimal characters, such as a message id or correlid. It does not get converted to a different code page. There are fields
      • MQBACF_ (Byte Parameter Types ) and are in the range 7001 to 7039.
    • A character string(MQCFST). This is a string such as queue name. This has a code page associated with it. This allows you to convert from ASCII to EBCDIC (or other conversion). There are fields
      • MQCA_ (Character Attribute Selectors) and are in the range 2001 to 2999 range (and 4000) for queue manager values .
      • MQCACF_ (Character Queue Manager Parameter Types) in the range 3000 to 3211 + a couple in the 55xx range.
      • MQCACH_ (Character Channel Parameter Types) in the range 3500 to 3573
    • An integer (MQCFIN), such as queue depth, or the value to mean “Enabled” or “Disabled”. There are fields
      • MQIA_, (Integer Attribute Selectors) integers in the range of 1 to 275 ( and 2000).
      • MQIACF_, (Integer Queue Manager Parameter Types) and are number in the range of 1 to 1440.
      • MQIACH_, (Integer Channel Types) and in the range 1501 to 1646.
    • A 64 bit integer(MQCFIN64). For example MQIAMO64_GET_BYTES, the number of bytes got as a integer 64 number. The values are intermixed with MQIAMO_ numbers. These are only used by the monitoring code – there are no MQIA*64 for anything else
  • Lists of fields
    • 64 bit integers(MQCFIL64). These are used in the midrange monitoring
    • integers(MQCFIL)
    • character strings(MQCFSL)
  • A group of (sub) parameters (MQCFGRP).
    • For example with application activity. This has a several groups. One of these groups is “Operations” which contains a set of PCF data for Operation type, Operation time, QMGR name).
    • They are also used in command events – but not used in command server messages.
    • These have a type field of MQGACF_ and are in the range 8001 to 9000
  • Filters – which provides the same function as the “WHERE” clause on a command.
    • Passing in a byte string (MQCFBF),
    • Passing a string (MQCFSF)
    • Passing in an integer string(MQCFIF).

Using PCF

It takes a while to understand how to use PCF. I found I had to write a lot of code to handle the PCF data and covert from numeric values to useful character strings so as to produce useful output. It does all work – it is just hard to get it to work.
To make it just a touch more complex, you may need to worry about data conversion and strings having different length when doing conversion, but I’ll leave this “as an exercise for the reader”1.

(1)Quote: The phrase is sometimes used as a joke. When a writer gets to the difficult part of a problem and he doesn’t want to take the trouble to solve it, or doesn’t know how to solve, he says this is “left as an exercise for the reader”, as if it was too trivial to waste space in the book discussing, when really the issue is that he can’t figure out the answer himself.

What was new in the MQ API?

I wanted to know what new features were available in different releases of MQ, but could not find the information documented.

I’ve taken the CMQC C header file from different releases on z/OS and shown the differences!

From MQ V8.0 to V9.0.1

CMQC

MQCNO_CURRENT_VERSION          5      
MQCNO_VERSION_6                6 
MQCNO_CURRENT_VERSION          6

MQCNO_CURRENT_LENGTH           188    
MQCNO_LENGTH_6                 208 
MQCNO_CURRENT_LENGTH           208 

MQAT_MCAST_PUBLISH             36 
MQAT_AMQP                      37 
MQAT_DEFAULT                   2 

MQOT_PROT_POLICY               1019 
MQOT_TT_CHANNEL                1020 
MQOT_AMQP_CHANNEL              1021 
MQOT_AUTH_REC                  1022 
MQ_AMQP_CLIENT_ID_LENGTH       256
MQRC_STORAGE_MEDIUM_FULL       2192
MQRC_ADMIN_TOPIC_STRING_ERROR  2598 
MQRC_AMQP_NOT_AVAILABLE        2599 
MQRC_CCDT_URL_ERROR            2600
MQADOPT_CHECK_CHANNEL_NAME     8  

MQCMDL_CURRENT_LEVEL           800                         
MQCMDL_LEVEL_801               801 
MQCMDL_LEVEL_802               802 
MQCMDL_LEVEL_900               900 
MQCMDL_LEVEL_901               901 
MQCMDL_LEVEL_902               902 
MQCMDL_LEVEL_903               903 
MQCMDL_LEVEL_904               904 
MQCMDL_LEVEL_905               905 
MQCMDL_CURRENT_LEVEL           905 

MQPL_APPLIANCE                 28
MQCAP_EXPIRED                  2 

 /* Media Image Scheduling */ 
MQMEDIMGSCHED_MANUAL           0 
MQMEDIMGSCHED_AUTO             1 
                                                                 
 /* Automatic Media Image Interval */ 
MQMEDIMGINTVL_OFF              0 
                                                                 
 /* Automatic Media Image Log Length */ 
MQMEDIMGLOGLN_OFF              0 
                                                                 
 /* Media Image Recoverability */ 
MQIMGRCOV_NO                   0 
MQIMGRCOV_YES                  1 
MQIMGRCOV_AS_Q_MGR             2

MQCA_AMQP_SSL_CIPHER_SUITES    2137 
MQCA_AMQP_VERSION              2136  
  
MQCA_LAST_USED                 2135     
MQCA_LAST_USED                 2137   

MQIA_ADVANCED_CAPABILITY       273 
MQIA_AMQP_CAPABILITY           265 

MQIA_AUTHENTICATION_METHOD     266
MQIA_KEY_REUSE_COUNT           267 

MQIA_LAST_USED                 264               
MQIA_LAST_USED                 273   

/* Key reuse count */ 
MQKEY_REUSE_DISABLED           0 
MQKEY_REUSE_UNLIMITED          (-1)  

struct tagMQCNO { 
  ...
  PMQCHAR    CCDTUrlPtr;           /* Address of CCDT URL string */ 
MQLONG     CCDTUrlOffset;        /* Offset of CCDT URL string */ 
MQLONG     CCDTUrlLength;        /* Length of CCDT URL */ 
MQBYTE8    Reserved;             /* Reserved */ 
  /* Ver:6 */  
}
...
MQCNO_DEFAULT ....

CMQCFC

MQCMD_INQUIRE_AMQP_CAPABILITY  216 
MQCMD_AMQP_DIAGNOSTICS         217

MQRCCF_CLWL_EXIT_NAME_ERROR    3374 
MQRCCF_SERVICE_NAME_ERROR      3375 
MQRCCF_REMOTE_CHL_TYPE_ERROR   3376 
MQRCCF_TOPIC_RESTRICTED        3377 
MQRCCF_CURRENT_LOG_EXTENT      3378 
MQRCCF_LOG_EXTENT_NOT_FOUND    3379 
MQRCCF_LOG_NOT_REDUCED         3380 
MQRCCF_LOG_EXTENT_ERROR        3381 
MQRCCF_ACCESS_BLOCKED          3382 

MQ_ENTITY_NAME_LENGTH          64             
MQ_ENTITY_NAME_LENGTH          1024  


MQIAMO_MONITOR_CLASS           839 
MQIAMO_MONITOR_TYPE            840 
MQIAMO_MONITOR_ELEMENT         841 
MQIAMO_MONITOR_DATATYPE        842 
MQIAMO_MONITOR_FLAGS           843 
MQIAMO64_QMGR_OP_DURATION      844 
MQIAMO64_MONITOR_INTERVAL      845 
MQIAMO_LAST_USED               845 

/* Defined values for MQIAMO_MONITOR_FLAGS */ 
MQIAMO_MONITOR_FLAGS_NONE      0 
MQIAMO_MONITOR_FLAGS_OBJNAME   1 
                                                       
/* Defined values for MQIAMO_MONITOR_DATATYPE */ 
MQIAMO_MONITOR_UNIT            1 
MQIAMO_MONITOR_DELTA           2 
MQIAMO_MONITOR_HUNDREDTHS      100 
MQIAMO_MONITOR_KB              1024 
MQIAMO_MONITOR_PERCENT         10000 
MQIAMO_MONITOR_MICROSEC        1000000 
MQIAMO_MONITOR_MB              1048576 
MQIAMO_MONITOR_GB              100000000 

MQIACF_AMQP_ATTRS              1401 
MQIACF_AMQP_DIAGNOSTICS_TYPE   1406 

MQIACF_SYSP_MAX_CONC_OFFLOADS  1412          
MQIACF_LAST_USED               1412          
MQIACF_AUTH_REC_TYPE           1412 
MQIACF_SYSP_MAX_CONC_OFFLOADS  1413 
MQIACF_SYSP_ZHYPERWRITE        1414 
MQIACF_Q_MGR_STATUS_LOG        1415 
MQIACF_ARCHIVE_LOG_SIZE        1416 
MQIACF_MEDIA_LOG_SIZE          1417 
MQIACF_RESTART_LOG_SIZE        1418 
MQIACF_REUSABLE_LOG_SIZE       1419 
MQIACF_LOG_IN_USE              1420 
MQIACF_LOG_UTILIZATION         1421 
MQIACF_LOG_REDUCTION           1422 
MQIACF_LAST_USED               1422 

MQIACH_LAST_USED               1643  
MQIACH_AMQP_KEEP_ALIVE         1644 
MQIACH_SECURITY_PROTOCOL       1645 
MQIACH_LAST_USED               1645

MQCAMO_LAST_USED               2712  
MQCAMO_MONITOR_CLASS           2713 
MQCAMO_MONITOR_TYPE            2714 
MQCAMO_MONITOR_DESC            2715 
MQCAMO_LAST_USED               2715 
                                             
MQCACF_LAST_USED               3206  
MQCACF_AMQP_CLIENT_ID          3207 
MQCACF_ARCHIVE_LOG_EXTENT_NAME 3208 
MQCACF_LAST_USED               3208 

MQCACH_LAST_USED               3570  
MQCACH_TOPIC_ROOT              3571 
MQCACH_LAST_USED               3571

MQGACF_LAST_USED               8014
MQGACF_MONITOR_CLASS           8015 
MQGACF_MONITOR_TYPE            8016 
MQGACF_MONITOR_ELEMENT         8017 
MQGACF_LAST_USED               8017 
  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
MQACT_REDUCE_LOG               10 
MQACT_ARCHIVE_LOG              11 
                                             

MQEVO_REST                     8 

MQRQ_FAILOVER_PERMITTED        30 
MQRQ_FAILOVER_NOT_PERMITTED    31 
MQRQ_STANDBY_ACTIVATED         32 

MQLDAP_AUTHORMD_SRCHGRPSN      3 

/* Authentication Method */ 
MQAUTHENTICATE_OS              0 
MQAUTHENTICATE_PAM             1 
                                                
/* Reduce Log Options */ 
MQLR_ONE                       1 
MQLR_AUTO                      (-1) 
MQLR_MAX                       (-2)

From MQ 9.0.1 to MQ 9.1.1

CMQC

MQCNO_CURRENT_VERSION          6          
MQCNO_VERSION_7                7 
MQCNO_CURRENT_VERSION          7 
 
MQCNO_CURRENT_LENGTH           208            
MQCNO_LENGTH_7                 240 
MQCNO_CURRENT_LENGTH           240 

MQCNO_GENERATE_CONN_TAG        0x00200000

MQAN_NONE 
MQAN_NONE_ARRAY
MQRC_INCOMPLETE_TRANSACTION    2147  
MQRC_Q_MGR_RECONNECT_REQUESTED 2601
MQRC_SUB_JOIN_NOT_ALTERABLE    29440

/* Max queue file size values */ 
MQQFS_DEFAULT                  (-1) 

MQCMDL_CURRENT_LEVEL           905               
MQCMDL_LEVEL_910               910 
MQCMDL_LEVEL_911               911 
MQCMDL_LEVEL_912               912 
MQCMDL_LEVEL_913               913 
MQCMDL_LEVEL_914               914 
MQCMDL_LEVEL_915               915 
MQCMDL_CURRENT_LEVEL           915 

MQIA_LAST_USED                 273          
MQIA_LAST_USED                 274   

MQIA_MAX_Q_FILE_SIZE           274  

struct tagMQCNO { 
  ...   
  MQCHAR28   ApplName;             /* Application name */ 
  MQBYTE4    Reserved2;            /* Reserved */ 
  /* Ver:7 */ 
};
MQCNO_DEFAULT....

CMQCFC

 MQCMD_AMQP_DIAGNOSTICS         217 
 MQCMD_INTER_Q_MGR_STATUS       218 
 MQCMD_INTER_Q_MGR_BALANCE      219 
 MQCMD_INQUIRE_APPL_STATUS      220 

 MQRCCF_PS_REQUIRED_MQUC        3383 
 MQRCCF_OBJECT_ALREADY_EXISTS   4001 

 MQRCCF_APPL_STATUS_NOT_FOUND   4097 
 MQCFT_STATUS                   27 

MQBACF_LAST_USED               7035   
MQBACF_REQUEST_ID              7036 
MQBACF_PROPERTIES_DATA         7037 
MQBACF_CONN_TAG                7038 
MQBACF_LAST_USED               7038 

MQIACF_LAST_USED               1422   
MQIACF_IGNORE_STATE            1423 
MQIACF_MOVABLE_APPL_COUNT      1424 
MQIACF_APPL_INFO_ATTRS         1425 
MQIACF_APPL_MOVABLE            1426 
MQIACF_REMOTE_QMGR_ACTIVE      1427 
MQIACF_APPL_INFO_TYPE          1428 
MQIACF_APPL_INFO_APPL          1429 
MQIACF_APPL_INFO_QMGR          1430 
MQIACF_APPL_INFO_LOCAL         1431 
MQIACF_APPL_IMMOVABLE_COUNT    1432 
MQIACF_BALANCED                1433 
MQIACF_BALSTATE                1434 
MQIACF_APPL_IMMOVABLE_REASON   1435 
MQIACF_DS_ENCRYPTED            1436 
MQIACF_CUR_Q_FILE_SIZE         1437 
MQIACF_CUR_MAX_FILE_SIZE       1438 
MQIACF_LAST_USED               1438 

MQIACH_LAST_USED               1645         
MQIACH_SPL_PROTECTION          1646 
MQIACH_LAST_USED               1646 

MQCACF_LAST_USED               3208         
MQCACF_APPL_IMMOVABLE_DATE     3209 
MQCACF_APPL_IMMOVABLE_TIME     3210 
MQCACF_LAST_USED               3210 

MQGACF_LAST_USED               8017         
MQGACF_APPL_STATUS             8018 
MQGACF_CHANGED_APPLS           8019 
MQGACF_ALL_APPLS               8020 
MQGACF_APPL_BALANCE            8021 
MQGACF_LAST_USED               8021 
                                            
MQIS_NO                        0 
MQIS_YES                       1 

/* Movable Options */ 
MQAPPL_IMMOVABLE               0 
MQAPPL_MOVABLE                 1 
                                    
/* Active Options */ 
MQACTIVE_NO                    0 
MQACTIVE_YES                   1 
                                    
/* Balance Options */ 
MQBALANCED_NO                  0 
MQBALANCED_YES                 1 
MQBALANCED_NOT_APPLICABLE      2 
MQBALANCED_UNKNOWN             3 
                                    
/* Balance State */ 
MQBALSTATE_NOT_APPLICABLE      0 
MQBALSTATE_LOW                 1 
MQBALSTATE_OK                  2 
MQBALSTATE_HIGH                3 
MQBALSTATE_UNKNOWN             4 

  /* Immovable Reasons */ 
MQIMMREASON_NONE               0 
MQIMMREASON_NOT_CLIENT         1 
MQIMMREASON_NOT_RECONNECTABLE  2 
MQIMMREASON_MOVING             3 
MQIMMREASON_APPLNAME_CHANGED   4

From MQ 9.1.1 to MQ 9.2.2

CMQC

MQ_NHA_INSTANCE_NAME_LENGTH    48 

MQCMDL_CURRENT_LEVEL           915    
MQCMDL_LEVEL_920               920 
MQCMDL_LEVEL_921               921 
MQCMDL_LEVEL_922               922 
MQCMDL_CURRENT_LEVEL           922

CMQCF

MQCACF_APPL_IMMOVABLE_TIME     3210 
MQCACF_LAST_USED               3210       
MQCACF_NHA_INSTANCE_NAME       3211 
MQCACF_LAST_USED               3211

MQRQ_STANDBY_ACTIVATED         32 
MQRQ_REPLICA_ACTIVATED         33

From MQ 9.2.2. to 9.2.4

CMQC

MQBNO structure and  MQBNO_ constants

MQCNO_CURRENT_VERSION          7
MQCNO_VERSION_8                8 
MQCNO_CURRENT_VERSION          8

MQCNO_CURRENT_LENGTH           240    
MQCNO_LENGTH_8                 252 
MQCNO_CURRENT_LENGTH           252 

MQ_TEMPORARY_Q_PREFIX_LENGTH   32 
MQRC_BNO_ERROR                 2602 
MQRC_OUTBOUND_SNI_NOT_VALID    2603 

/* Streaming Queue Quality of Service Values */ 
MQST_BEST_EFFORT               0 
MQST_MUST_DUP                  1 

MQCMDL_CURRENT_LEVEL           922       
MQCMDL_LEVEL_923               923 
MQCMDL_LEVEL_924               924 
MQCMDL_CURRENT_LEVEL           924 

MQCA_LAST_USED                 2137
MQCA_LAST_USED                 2138 

MQIA_LAST_USED                 274      
MQIA_LAST_USED                 275 

MQIA_STREAM_QUEUE_QOS          275 

struct tagMQCNO { 
   ...     
   PMQBNO     BalanceParmsPtr;      /* Balance Parameter Pointer */ 
   MQLONG     BalanceParmsOffset;   /* Balance Parameter Offset */ 
   MQBYTE4    Reserved3;            /* Reserved */ 
   /* Ver:8 */ 
 };

CMQCFC

MQRCCF_STREAMQ_DEST_NOT_SUPP   3384 
MQRCCF_STREAMQ_DEST_CONFLICT   3385 
MQRCCF_STREAMQ_NOT_SUPPORTED   3386 
MQRCCF_STREAMQ_CONFLICT        3387 
 
MQBACF_LAST_USED               7038    
MQBACF_MQBNO_STRUCT            7039 
MQBACF_LAST_USED               7039 
  
MQIACF_SYSP_SMF_STAT_TIME_MINS 1199 
MQIACF_SYSP_TRACE_CLASS        1200 
  
MQIACF_LAST_USED               1438
MQIACF_BALANCING_TYPE          1439 
MQIACF_BALANCING_OPTIONS       1440 
MQIACF_BALANCING_TIMEOUT       1441 
MQIACF_SYSP_SMF_STAT_TIME_SECS 1442 
MQIACF_SYSP_SMF_ACCT_TIME_MINS 1443 
MQIACF_SYSP_SMF_ACCT_TIME_SECS 1444 
MQIACF_LAST_USED               1444 

MQIACH_AUTH_INFO_TYPES         1622 

MQCACH_LAST_USED               3571      
MQCACH_TEMPORARY_MODEL_Q       3572 
MQCACH_TEMPORARY_Q_PREFIX      3573 
MQCACH_LAST_USED               3573 
                                         
MQIMMREASON_APPLNAME_CHANGED   4 
MQIMMREASON_IN_TRANSACTION     5 
MQIMMREASON_AWAITS_REPLY       6

“I do not understand these get counts on z/OS”

I had a question from someone who was trying to use the MQ SMF statistics to understand their queue manager usage. This came from a post from Lynn Elkins who was talking about “Data ManagerGets”. I started writing a short answer… but the answer got longer, so I thought I would post on this topic.

The z/OS queue manager has different components, for example the logging component which manages recovery data, the buffer manager which manages pages in the buffer pool, the message manager which handles the API side of MQ requests, and the data manager which manages who messages are mapped to pages in a page set.

There is an SMF statistic QMSTGET which is the number of application MQGET requests, and QISTMGET, the number of requests to data manager to start getting a messages.

I’ll give some examples and explain why there could be a big difference in the numbers.

Shared queue

Simple MQGET of shared queue

An application issues an MQGET, there is a message on the queue but this is a shared queue, so data manager is not involved

QMSTGET is now +1, QISTMGET is unchanged

Local queue

Simple MQGET of local queue

An application issues an MQGET, there is a message on the queue the messsage is returned.

QMSTGET is now +1, QISTMGET is +1GET.

Get with wait when there was no message on the queue

The application issues an MQGET, so increment QMSTGET. The queue is empty, so do not change QISTMGET. Control is returned to the adapter and the application waits. A message arrives, the application is woken up, and reissues the MQGET. The message is returned.

Increment QMSTGET again, increment QISTMGET.
QMSTGET is now +2, QISTMGET is +1

The getting application polls the queue

Rather than use MQGET with wait, the application issues MQGET with no wait. There is a code

Loop forever:
MQGET; if message available then leave loop
else wait for 1 second
end

After 1 hour the statistics could be

QMSTGET is now + 3600, QISTMGET is unchanged.

Multiple applications waiting on a queue

  • 100 applications issue MQGET with wait on a queue.
    • QMSTGET is now +100, and all applications are waiting in the adapter.
  • A message arrives one application wakes up and gets the message
    • QMSTMGET is now +101, QISTGET is now +1
  • The woken application reissues the MQGET
    • QMSTGET is now +102, QISTGET is now +1, and all applications are waiting in the adapter.

An application gets a message by message-id or correl-id

If the queue has not been indexed there is internal logic which searches along the queue for a message with matching message-id or correl-did. For example if there were 1000 messages on the queue, and the message was not found:

QMSTGET is now + 1, QISTMGET is now +1000.

An application gets a message by filter or message property

For example get the next message where the property color=”red”. This is like the previous case. The queue has to be searched looking for the specified property in the message header.

For example if there were 1000 messages on the queue, and the message was not found:

QMSTGET is now + 1, QISTMGET is now +1000.

Should you compare QMSTGET and QISTMGET?

If you gave me the two values, it would be very hard to draw any conclusions from it. Where it might be useful is comparing the difference over time. It delta or ratio changes, then it is likely that something has changed in your system, and could be a hint to go sniffing around.

Of course the delta or difference is likely to change with time of day, and day of week, so make sure you compare a similar time period.

Where’s my publish?

Like many things I do, it all started with an easy problem. I had an application which subscribed to a topic, it published on a topic string, then got the published message. Another application subscribed on the same topic string but did not get the message. I’ll leave clues for the experts to spot as I explain the progress I made.

The first complication was the subscribing and publishing application was in Python on z/OS (which runs in ASCII), and the other application was written in C using the default EBCDIC. I had to spend time working out how to convert from ASCII to EBCDIC. (When using CHARV you can specify a string and its code page and have MQ do it for you automatically – easy once you know which code page to use.) For other strings I had to convert manually using the A2E function from C run time.

There are some commands for displaying status of pub sub, it took a little while to understand them and give me the data I wanted.

For example

My application did a subscription with topic string “/currency/rate/EUR/GBP”.
The command gave

DIS TPSTATUS(‘/currency/rate/EUR/GBP’)
CSQM297I … CSQMDRTC NO TPSTATUS FOUND MATCHING REQUEST CRITERIA

Once I had opened the topic using the topic string, I got

CSQM201I … CSQMDRTC DIS TPSTATUS DETAILS
TPSTATUS(/currency/rate/EUR/GBP)
TYPE(TOPIC)
DEFPRESP(SYNC)
PUB(ENABLED)
SUB(ENABLED)
ADMIN()
RETAINED(NO)
PUBCOUNT(0)
SUBCOUNT(1)
PUBSCOPE(ALL)
SUBSCOPE(ALL)
…

I set up the subscription using Python

topic_string = ‘/currency/rate/EUR/GBP’
sub_desc = pymqi.SD() # create the basic subscription
sub_desc[‘Options’] = pymqi.CMQC.MQSO_CREATE + pymqi.CMQC.MQSO_RESUME +
pymqi.CMQC.MQSO_DURABLE + pymqi.CMQC.MQSO_MANAGED
sub_desc.set_vs(‘SubName’, “MYSUB” )
sub_desc.set_vs(‘ObjectString’, topic_string)

The code to get the subscribed message was

get_opts = pymqi.GMO(
MsgHandle=hMsg.msg_handle,
Options=pymqi.CMQC.MQGMO_NO_SYNCPOINT +
pymqi.CMQC.MQGMO_CONVERT +
pymqi.CMQC.MQGMO_PROPERTIES_IN_HANDLE +
pymqi.CMQC.MQGMO_FAIL_IF_QUIESCING +
pymqi.CMQC.MQGMO_WAIT)
get_opts[‘WaitInterval’] = 1000
data = sub.get(None, pymqi.md(), get_opts)
print(‘Received data: [%s]’ % data)

The code to publish to a topic was

msg = ‘1.3961’
topic = pymqi.Topic(qmgr, topic_string=topic_string)
topic.open(open_opts=pymqi.CMQC.MQOO_OUTPUT)
topic.close()

The Python program was subscribe, publish to topic, get the message.

When I ran the code, no messages were published. I tried changing the topic_string to ‘/currency/rate/EUR/USD’, that had no effect. Eventually I tracked down a code page problem in my code; I fixed that – but the program still did not publish a message.

The more knowledgable people reading this may have spotted one problem.

The first time the program ran, the subscription code created a SUB(MYSUB) with the given topic string (‘/currency/rate/EUR/USD’). When I changed the topic string to ‘/currency/rate/EUR/GBP’, and published a message, the subscription was using the original topic string with USD!, but publishing to GBP. Any changes to the SUBscription are ignored unless MQSO_ALTER is specified.

I deleted the SUB and reran my application. Displaying the subscription (MYSUB) gave

CSQM201I %CSQ9 CSQMDRTC DIS SUB DETAILS 934
SUB(MYSUB)
SUBID(C3E2D8D4C3E2D8F94040404040404040DAFEED99504CE540)
DURABLE(YES)
SUBTYPE(API)
TOPICSTR(/currency/rate/EUR/GBP)
SUBLEVEL(1)
…

I can display the topic string and see what is subscribed to it

dis tpstatus(‘/currency/rate/EUR/GBP’) type(sub)

This gave two subscriptions:

TPSTATUS(/currency/rate/EUR/GBP)
TYPE(SUB)
SUBID(C3E2D8D4C3E2D8F94040404040404040DAFE062AED26E2C0)
DURABLE(YES)
SUBTYPE(ADMIN)
SUBUSER(IBMUSER)
…
NUMMSGS(12)

and

TPSTATUS(/currency/rate/EUR/GBP)
TYPE(SUB)
SUBID(C3E2D8D4C3E2D8F94040404040404040DAFC73649B3C7200)
DURABLE(YES)
SUBTYPE(ADMIN)
SUBUSER(IBMUSER)
…
NUMMSGS(0)

So one subscription has seen 12 messages, the other subscription has seen 0 messages. This does not look like a wrong topic string, as they are reported as part of the display tpstatus.

I could not find a way of displaying all the information about a subscription in one command. I had to use DIS TPSTATUS.. TYPE(SUB), then use cut and paste to issue the DIS SUB SUBID(….). (It would be much easier for the poor end users if there was an ALL option to display all this data in one go.)

For example

SUB(COLIN)
SUBID(C3E2D8D4C3E2D8F94040404040404040DAFC73649B3C7200)
DURABLE(YES)
SUBTYPE(ADMIN)
DISTYPE(RESOLVED)
TOPICSTR(/currency/rate/EUR/GBP)
DEST(CP0000)
DESTQMGR()
DESTCORL(C3E2D8D4C3E2D8F94040404040404040DAFC73649B3C7200)
TOPICOBJ()
PUBACCT(00000000000000000000000000000000000000000000000000000000…)
USERDATA(CZZZ)
SUBUSER(IBMUSER)
SUBLEVEL(1)
…

So we can see from this subscription the topic string being subscribed on, and the queue name being used.

This subscription had no messages published to it; see the DISPLAY TPSTATUS TYPE(SUB) field NUMMSGS. Those of you with a PhD in PubSub may have spotted the problem; it is very well hidden. It is all to do with PubLevel and SubLevel. (The documentation is very confusing, it implies a subscription can have more than one sublevel).

I think of it as logic within publish which says for each subscription

If SubLevel <= PubLevel then publish the message
else skip the publish
  • The subscription from the python code had SubLevel = 0. (This was latent bug – it should have been set to 1!
  • The SubLevel from my DEFINE SUB command was the default 1.
  • The PubLevel from the Python code was 0 – this was the problem.

So the publish to my manual DEFINE SUB was skipped, and the publish to the Python code worked because PubLevel = 0 matched SubLevel = 0

The message is being published – but it is not still not there.

When I restarted my queue manager and ran the test, the Python application got no message found, which was a surprise to me. When I reran it, with debug code enabled – the get worked, and carried on working all day. Next day the same problem occurred. Very strange – is the debug code affecting the messages ?

I then remembered the default syncpoint behaviour (one of those face palm moments). On midrange the default is to put out out sync-point, so the messages are immediately available. On z/OS the default is to put within sync-point. The messages are only visible after a commit, or the implicit commit at MQDISC. On z/OS the messages were in syncpoint, and there was no commit. When I ran my program the second time, it got the message from the first run, and so on!

I put the commit in – and it all worked.

Check list of what to check

Check you are subscribing to the correct topic string.

Plan carefully how you define your topic(strings).

If you want to have an application define a subscription, consider some defensive programming.

  • MQSO_CREATE will create a subscription based on the specified data. If it already exists report an error.
  • MQSO_CREATE + MQSO_RESUME will create a subscription based on the specified data. If it already exists it will not report an error. A second usage will not change the subscription.
  • MQSO_CREATE + MQSO_ALTER will create a subscription based on the specified data. A second usage will update the subscription. This could be considered dangerous if you have multiple applications running concurrently changing the topic string or other parameters.
  • MQSO_RESUME does not create a subscription, not updates it. it uses the information in the subscription object.

It would be safer for an Administrator to define the MQSUB, and the applications to refer to it. Do not allow application users to be able to define a subscription. They should use the pre-defined MQSUB and add the topic string to give more granular data. For example I could use a null object and have a topic string of /TOP/SECRET/DATA/SPIESIN/ENGLAND. If you haven’t set up security properly then I could access the data.

Check the PubLevel and the SubLevel.

If you are not using this functionality set PubLevel to 9 and SubLevel to 1 (the defaults).

You can DISPLAY SUB(….) SUBLEVEL to see the specified value.

The PubLevel is specified in the MQPMO.PubLevel. There is no display command for this.

Check the sync-point options.

You should always, always, always specify MQPMO_NO_SYNCPOINT, or MQPMO_SYNCPOINT and the commit. Do not let it default in case someone ports your code to or from z/OS. You can waste days hunting down this problem.

What do you mean, I can’t set the maximum queue depth?

Ive been involved with MQ for the whole of its life (I worked on the very first release), and I’ve just discovered something which is really basic!

I was testing out some code, and wanted to set the max depth of the queue to a low value for the duration of a test.

I thought this was easy; just use MQSET, specify parameter MQIA_MAX_Q_DEPTH, and the value I wanted (9).

When it reported

Reason 2067: FAILED: MQRC_SELECTOR_ERROR

I added in extra code to printout the values and yes, this was the value returned. If this was a bug I was sure it would have been spotted many years ago. I checked the documentation (remember, if all else fails, read the documentation). Look, I can set get inhibit, put inhibit etc ahh – max depth was not listed.

So when you get Reason 2067: FAILED: MQRC_SELECTOR_ERROR please check the documentation.