Tag: PCF

You can now easily display MQ PCF data using Python

For many years I have been frustrated with displaying data provided by MQ midrange in PCF format, as there was nothing to help you use it. Yes, you could display it, but it was a bit like saying … here is a bucket of bits, now build your own program from it to process the data – good luck.

I had started to write some C code to process it, but then I retired from IBM, and since then have found that Python is a brilliant systems management language.

I have put up on Github, some Python code which does the following.

You can create PCF command, such as INQUIRE QUEUES. It can then parse the output into “English” and you can then do things with it. By “English” I mean

Q_TYPE: LOCAL rather than the value 1
DEF_BIND: BIND_ON_OPEN rather than the value 16384

There are examples of what you can do with it.

get_pcf.py. You specify the connection details, and the queue, and it returns the data in json format which you can pipe into another command.

queues.py issues the INQUIRE QUEUE command. This outputs the data in json format. queues2.py then writes it into a file, one file per queue, one line per attribute. This is great

  • you do not have to worry about trying to parse the output from runmqsc
  • you can use standard tools on the file
  • you can do more…

There is a diff.py sample you give it a list of files and it tells you the difference in the queue definitions (while ignoring attributes like change date), for example

CP0000.yml CP0001.yml : Q_NAME CP0000 / CP0001
CP0000.yml CP0001.yml : Q_DESC Main queue / None
CP0000.yml CP0001.yml : MAX_Q_DEPTH 2000 / 5000
CP0000.yml CP0001.yml : Q_DEPTH_HIGH_EVENT ENABLED / DISABLED

There is standards.py which allows you to check attributes in the .yml file meet your corporate standards!

There is events.py and events2.py so you can now process the events produced by the define, delete and alter commands, and see who made the change what the change was, and when the change was made.

I am working on making the stats and accounting usable, so you can create a .csv file with useful data in it, or pass it into Kibana and other tools. So watch this space.

I would welcome any comments of feedback. Ive had one already. When using Eclipse and Python it supports text completion – so if you typed in MQ.INQ… it should give you a list of options to pick from.

These tools build on top of the excellent pymqi package which provide the MQAPI for Python programs. You use pymqi to put and get messages, then use the mqtools package to process the data.

How to get hold of it…

The README has instructions on how to download it. If there is enough interest I’ll package it up so PIP INSTALL can find it.

What do change events look like?

I was using my Python programs for processing MQ PCF data, and was testing them out with different sorts of MQ data. I found the documentation for change events in the IBM knowledge centre is a bit sparse, so here is some information on it.

I did

  • define QL(DELETEME)
  • alter ql(DELETEME) descr(‘change comment’)
  • delete QL(DELETEME)

For define and delete there is one event message created. For alter there are two messages created, one contains before data the other has the after data.

The MD looks like

"MQMD":{
     "StrucId":"MD  ",
     "Version":1,
     "Report":"NONE",
     "MsgType":"DATAGRAM",
     "Expiry":-1,
     "Feedback":"NONE",
     "Encoding":546,
     "CodedCharSetId":1208,
     "Format":"MQEVENT ",
     "Priority":0,
     "Persistence":"NOT_PERSISTENT",
     "MsgId":"0x414d5120514d4120202020202020202020…",
     "CorrelId":"0x414d5120514d4120202020202020202020…",
     "BackoutCount":0,
     "ReplyToQ":"  ",
     "ReplyToQMgr":"QMA",
     "UserIdentifier":"            ",
     "AccountingToken":"0x0000…",
     "ApplIdentityData":"  ",
     "PutApplType":"QMGR",
     "PutApplName":"QMA   ",
     "PutDate":"20190217",
     "PutTime":"11141373",
     "ApplOriginData":"    ",
     "GroupId":"0x000000...",
     "MsgSeqNumber":1,
     "Offset":0,
     "MsgFlags":0,
     "OriginalLength":-1
   },

When a change event occurs the MD for the “after” message is effectively the same but note the MsgID is different, and the CorrelId is the same, so you will need to clear the MsgId and keep the CorrelID when getting the second message.

For the create command the PCF header was

"Header":{
     "Type":"EVENT",
     "StrucLength":36,
     "Version":2,
     "Command":"CONFIG_EVENT",
     "MsgSeqNumber":1,
     "Control":"LAST",
     "CompCode":0,
     "Reason":2367,  "CONFIG_CREATE_OBJECT"
     "ParameterCount":58,
     },

For the delete command the PCF header was

"Header":{
     "Type":"EVENT",
     "StrucLength":36,
     "Version":2,
     "Command":"CONFIG_EVENT",
     "MsgSeqNumber":1,
     "Control":"LAST",
     "CompCode":0,
     "Reason":2369, "CONFIG_DELETE_OBJECT
     "ParameterCount":58

}

For the alter/change command the PCF headers were

"Header":{
     "Type":"EVENT",
     "StrucLength":36,
     "Version":2,
     "Command":"CONFIG_EVENT",
     "MsgSeqNumber":1,
     "Control":"NOT_LAST",
     "CompCode":0,
     "Reason":2368,
     "ParameterCount":58    
   },

and

"Header":{
     "Type":"EVENT",
     "StrucLength":36,
     "Version":2,
     "Command":"CONFIG_EVENT",
     "MsgSeqNumber":2,
     "Control":"LAST",
     "CompCode":0,
     "Reason":2368, "CONFIG_CHANGE_OBJECT"
     "ParameterCount":58
   },

The differences are the MsgSeqNumber value and Control being NOT_LAST or LAST.

The data is common to all requests. You need to compare the fields in both the records for the change event, to see which are different. With the data in Python dict – this was a trivial exercise.

"Data":{

    "EVENT_USER_ID":"colinpaice",

    "EVENT_ORIGIN":"CONSOLE",

    "EVENT_Q_MGR":"QMA",

    "OBJECT_TYPE":"Q",

    "Q_NAME":"DELETEME",

    "Q_DESC":"",

    …

    "Q_TYPE":"LOCAL"

  }

after the change

"Data":{
     "EVENT_USER_ID":"colinpaice",
     "EVENT_ORIGIN":"CONSOLE",
     "EVENT_Q_MGR":"QMA",
     "OBJECT_TYPE":"Q",
     "Q_NAME":"DELETEME",
     "Q_DESC":"change comment",
     …
     "ALTERATION_DATE":"2019-02-17",
     "ALTERATION_TIME":"11.14.33",
     …
     "Q_TYPE":"LOCAL"
   }

When I printed out the changed data, the first time ALTERATION_DATA and ALTERATION_TIME were both displayed. The second time, only ALTERATION_TIME was displayed. I thought this was a bug in my program. After checking my program, it was obvious… If I change the queue 10 minutes later – the ALTERATION_DATE does not change – so remember to report both of these, and not just the change values.