Understanding display authrec match(…) on midrange.

I found the documentation for the display authrec match(…) command hard to understand. There are ambiguous backward references (the profile … which profile?), too many ‘and’s, and I think some ‘or’s are missing. Below is how I interpret it.

  • Match exact: Select the record where the specified profile name specified is an exact match for a setmqauth record
  • Match profile: Display the setmqaut records which would be used to compute permissions, for the specified profile, and (specified userid or specified group).
  • Match membership:
    • For the specified userid do Match: profile() for the userid
    • For each group the specified userid is in, do Match: profile() for that group

I see the processing is in two stages

  1. Stage 1 extract the autrecs for the specified profile name
  2. Stage 2 filter the list using the specified userid or group.

Stage 1: extract the authrecs matching the specified profile name.

For match exact

Select the record where the specified profile name specified is an exact match for a setmqauth record. Profile(‘CP.**.99’) will match only ‘CP.**.99’.

For match profile and membership

If the specified profile has a generic then treat this as match(exact).

If the specified profile has no generics then extract all records which would apply when checking this profile.

For example for queue CP.AA.BB.99 might return

  • profile CP.AA.BB.99 (entity colinpaice) – this userid created the queue
  • profile CP.AA.BB.99 (entity mqm) – this entry is create when the queue is created
  • profile CP.**.99 (entity testuser) – this was done via setmqaut – not selected because less specific generic profile
  • profile CP.*.BB.99 (entity testuser) – this was done via setmqaut.

Stage 2. Filter the records depending on the specified userid or group.

Take the set of records from stage 1 and filter them. You can specify the principal (userid) or group. Note: If your qm.ini has SecurityPolicy=group then even if you have specified you setmqauth with a userid, it will use a group instead. This may mean that displaying a userid may give no results.

Match exact

Compare the specified entity and entity type with those in the the records. If they match display the record.

Match profile

Compare the specified entity, and entity type with those in the the records. If they match display the record.

Match membership

Compare the specified entity, and entity type with those in the the records. If they match display the record.

If the entity type in the record is group, and the specified userid is a member of the group then display it.

Examples

Match(profile) principal()

dis authrec profile(CP.AA.99) objtype(queue) match(profile)   principal('testuser')
AMQ8459I: Not found.    

There is no profile defined for the userid ‘testuser’

Match(profile) group()

dis authrec profile(CP.AA.99) objtype(queue) match(profile)   group('testuser')
   PROFILE(CP.*.99)                        ENTITY(testuser)  

There was a setmqauth -m -n “CP.*.99” -t queue -p testuser +get . Because of the qm.ini setting, and userid authorisations were converted to group authorisations. On some Unix systems, when a userid is created, it creates a group with the same name, and connects the userid to the group.

dis authrec profile(CP.AA.99) objtype(queue) match(profile)   group('test') 
AMQ8864I: Display authority record details.
    PROFILE(**) ENTITY(test) AUTHLIST(DSP,INQ) 

All members of group test, have Display and Inquire permissions on any queue.

Match(membership) principal()

dis authrec profile(CP.AA.99) objtype(queue) match(membership) principal('testuser')
AMQ8864I: Display authority record details.
   PROFILE(CP.*.99)                        ENTITY(testuser)
                         
AMQ8864I: Display authority record details.
   PROFILE(**)                             ENTITY(test)  

The display match(membership) combines all of the above. Any specific records, plus records for any group testuser is in.

Match(membership) group()

dis authrec profile(CP.AA.99) objtype(queue) match(membership) group('testuser')
AMQ8864I: Display authority record details.
   PROFILE(CP.*.99)                        ENTITY(testuser)

Because the group is specified, then this acts the same as match(profile) group(‘testuser’).

Match(profile) no group nor principal

dis authrec profile(CP.AA.99) objtype(queue) match(profile)   
AMQ8864I: Display authority record details.
   PROFILE(CP.AA.99)                       ENTITY(colinpaice)
 
AMQ8864I: Display authority record details.
   PROFILE(CP.AA.99)                       ENTITY(mqm)
 
AMQ8864I: Display authority record details.
   PROFILE(CP.*.99)                        ENTITY(testuser)
                
AMQ8864I: Display authority record details.
   PROFILE(CP.**.99)                       ENTITY(testuser)
                 
AMQ8864I: Display authority record details.
   PROFILE(**)                             ENTITY(test)

No userids or groups were specified, so all relevant autrecs for the profile CP.AA.99 are displayed.

Observation: There is a profile for PROFILE(CP.**.99) ENTITY(testuser) which does not show up when dis authrec profile(CP.AA.99) objtype(queue) match(membership) principal(‘testuser’) is used.

This is because with generic profiles, only the most specific generic profile is used, see Profile Priorities.

How easy is it to display security information on midrange.

If you thought it was easy, I would disagree with you.

While investigating a security problem (why my code worked, when it should have got a not authorised return code) I struggled to answer questions like

There are links to my solutions for some of the problems.

Morag contacted me and said:

I wrote a little blog post about using DISPLAY ENTAUTH – hope it helps. See here:

Morag

What commands are there?

There are four MQ commands providing similar functions

Definition of terms

Profile name

This can be the name of a queue, or a string which is used for comparisons and can include single * or double **. For example

  • PAYROLL for a particular queue,
  • PAYROLL.*.INPUT, which would apply to objects PAYROLL.A.INPUT, PAYROLL.B.INPUT, but not PAYROLL.A.B.INPUT, nor PAYROLL.INPUT.
  • PAYROLL.**.INPUT which would apply to PAYROLL.A.INPUT, PAYROLL.B.INPUT, PAYROLL.A.B.INPUT, but not PAYROLL.INPUT.
  • ** used as a fall back if there are no other definitions.

Example profiles

  • setmqaut -m QMA -n PAYROLL -t queue -g testuser +put
  • setmqaut -m QMA -n ‘PAYROLL.*.INPUT’ -t queue -g testuser +put
  • setmqaut -m QMA -n ‘PAYROLL.**.INPUT’ -t queue -g testuser +put
  • setmqaut -m QMA -n ‘**’ -t queue -g MQAdmin +chg +dlt

On Unix you need the single quotes around any profile which includes *, to prevent the shell replacing it with a list of file names.

You need to be careful about quoting strings with mixed case data. dspmqaut and setmqaut do not need quoting. For runmqsc display|set authrec the mixed case fields need quoting.

A profile is created

  • by the setmqaut command,
  • the runmqsc set authrec command,
  • when the object is created. In which case, the queue manager creates an entry for the mqm group, and for the principal group of the user that created the object. Userid ibmsys1 created a queue, and there were entries for group mqm, and group zpdt. These definitions have full access. (The userid that created a queue can put and get messages from the queue – and you cannot separate administration of the queue, and usage of the queue.)

Userid and group

Unix has the concept of User Private Groups (UPG). A userid can have a group with the same name as the userid, which makes the security just a touch more confusing.

You can use the id command to list which groups a userid is in

id ibmsys1
uid=1001(ibmsys1) gid=1006(zpdt) groups=1006(zpdt),27(sudo),1007(MQAdmin)

Userid ibmsys1 with numeric id 1001 is a member of 3 groups: zpdt, sudo and MQAdmin.

You can specify the authority to be given to a userid – but it looks like the userid is not used, but the default group is used! See Explicit giving a userid access – does not give the userid access. This could just be a Linux feature.

Displaying information.

dspmqaut -m QMA -n ‘PAYROLL.**.INPUT’ -t queue -g testuser

This lists the information for the group testuser and the profile PAYROLL.**.INPUT’. It reports “put” access.

What access does the group ibmsys1 have?

dspmqaut -m QMA -n ‘PAYROLL’ -t queue -g ibmsys1

Entity ibmsys1 has the following authorizations for object PAYROLL:

 

 

Shows the group has no access defined.

What access does the userid (principal) ibmsys1 have?

dspmqaut -m QMA -n ‘PAYROLL’ -t queue -p ibmsys1

Entity ibmsys1 has the following authorizations for object PAYROLL:
dlt
chg

This is because userid ibmsys1 has MQAdmin as one of its group.

Adding -e option to the dmpmqaut command (Display all profiles used to calculate the cumulative authority that the entity has to the object specified in -n Profile) shows which profiles will be used.

dmpmqaut -m QMA -n ‘PAYROLL’ -t queue -p ibmsys1 -e
profile: **
object type: queue
entity: MQAdmin
entity type: group
authority: dlt chg

We can see that the profile used to determine the access permissions came from the “**” profile statement for group(entity) MQAdmin.

Explicit giving a userid access – does not give the userid access.

On my Linux Ubuntu machine, using the following command to explicitly give a userid access; gives it to the userid’s principal group.

setmqaut -m QMA -n PAYROLL -t queue -p ibmsys1 +put

dmpmqaut -m QMA -n PAYROLL -t queue -p ibmsys1 -e
profile: PAYROLL
object type: queue
entity: zpdt
entity type: group
authority: put

profile: **
object type: queue
entity: MQAdmin
entity type: group
authority: dlt chg

This shows the group zpdt got the access – not the userid.

For this userid we can see that there was a specific profile “PAYROLL” for the zpdt group giving put permission, and the generic profile for the MQAdmin group giving inquire and display permissions.

The display authrec command

If you use dis authrec profile(PAYROLL) it will display all possible profiles including the profiles for queues, process etc. Ensure all mixed case values are surrounded by single quotes.

dis authrec profile(PAYROLL) objtype(queue)

gave

AMQ8864I: Display authority record details.
  PROFILE(PAYROLL) ENTITY(colinpaice)
  ENTTYPE(GROUP) OBJTYPE(QUEUE)
  AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

AMQ8864I: Display authority record details.
  PROFILE(PAYROLL) ENTITY(mqm)
  ENTTYPE(GROUP) OBJTYPE(QUEUE)
  AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

AMQ8864I: Display authority record details.
  PROFILE(PAYROLL) ENTITY(testuser)
  ENTTYPE(GROUP) OBJTYPE(QUEUE)
 AUTHLIST(PUT)

AMQ8864I: Display authority record details.
  PROFILE(**) ENTITY(test) ENTTYPE(GROUP) OBJTYPE(QUEUE) AUTHLIST(DSP,INQ) 

AMQ8864I: Display authority   record details. 
  PROFILE(**) ENTITY(MQAdmin)
  ENTTYPE(GROUP) OBJTYPE(QUEUE)
  AUTHLIST(CHG,DLT)

but

dis authrec profile(PAYROLL) objtype(queue) group(MQAdmin)

gives not found however, using single quotes,

dis authrec profile(PAYROLL) objtype(queue) group(‘MQAdmin’)

gives

AMQ8864I: Display authority record details.
  PROFILE(**) ENTITY(MQAdmin)
  ENTTYPE(GROUP) OBJTYPE(QUEUE)
  AUTHLIST(CHG,DLT)

List all objects

You can use

runmqsc dis authrec type(queue)

to display all queue records with (up to three lines per entry) see above for an example.

You can use the dmpmqaut -l option Dump only the profile name and type. Use this option to generate a terse list of all defined profile names and types.

dmpmqaut -m QMA -l -t queue

Which gives output like

profile: PAYROLL, object type: queue
profile: PAYROLL.*.INPUT, object type: queue
profile: PAYROLL.**.INPUT, object type: queue

Unfortunately this is output to stderr, so you cannot immediately pipe this into grep or other commands.

You can use

dmpmqaut -m QMA -l -t queue 3>&1- 1>&2 2>&3|grep PAYROLL

The dmpmqaut command says you can use generics and wild cards in the name described Using OAM generic profiles on AIX®, Linux®, and Windows systems. I could not get this to work, for example using PAYROLL.**

The runmqsc DIS AUTHREC command says profile can have The name of the object or generic profile for which to display the authority records.

I think the documentation means “the string used in the profile definition, not a generic search argument.”


For example I wanted to use a qualifier ‘*’, but the command

dmpmqaut -m QMA -n ‘CP.*’

gives

No matching authority records.

runmqsc dis entauth – display the authorizations an entity has to a specified object

This very useful command was pointed out to me by Morag. It has been in MQ since MQ 7.5.

It allows you to ask what permissions a userid or group has to a specific resource. For example

A typical display

dis entauth principal(‘testuser’) objtype(queue) objname(CP0000)
AMQ8866I: Display entity authority details.
OBJNAME(CP0000) ENTITY(testuser)
ENTTYPE(PRINCIPAL) OBJTYPE(QUEUE)
AUTHLIST(BROWSE,CRT,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

Queue does not exist

dis entauth principal(‘testuser’) objtype(queue) objname(CP000ZZZZZZ)
AMQ8147E: IBM MQ object CP000ZZZZZZ not found.

Silly me – I missed the quotes off from around the userid


dis entauth principal(testuser) objtype(queue) objname(CP0000)
3 : dis entauth principal(testuser) objtype(queue) objname(CP0000)
AMQ8871E: Entity, principal or group not known.

Dump mq configuration

You can use the dmpmqcfg command to dump out the authrec records.

dmpmqcfg -m QMA -x authrec -o 1line

gives output like (where there is one line for each SET… statement).

SET AUTHREC PROFILE(‘CP.*.00’) GROUP(‘colinpaice’) OBJTYPE(QUEUE) AUTHADD(GET,PUT)
SET AUTHREC PROFILE(‘PAYROLL.**.INPUT’) GROUP(‘testuser’) OBJTYPE(QUEUE) AUTHADD(PUT)
SET AUTHREC PROFILE(‘@class’) GROUP(‘colinpaice’) OBJTYPE(QUEUE) AUTHADD(CRT)

The -o 1line option creates one line of output for each record. You can use platform tools to process the data, for example use grep to filter lines, and parse the parts between ‘ ‘. See an example python script at the bottom.

The documentation says -n [*|ObjectName] Filter the definitions produced by object or profile name, the object/profile name can contain a single asterisk. But it didnt work for me.

My basic queries

Is this user authorised to use this queue?

You can use

dspmqaut -m QMA -n PAYROLL.A -t queue -p ibmsys1

If the queue does not exist you will get AMQ7085E: Object PAYROLL.A, type queue not found.

Or you can use the runmqsc command

dis entauth principal(‘testuser’) objtype(queue) objname(CP0000)

What security profiles are involved with this user and that queue.

Or which profiles gave a user access to the queue?

Display the userids id and groups. If userid is in group mqm, it will get all access regardless of the setmqaut settings.

Display all the profiles involved with a userid -e … or group -g …

dmpmqaut -m QMA -n PAYROLL.A -t queue -e

Display all the permission a user has with a userid -p … or group -g …

dspmqaut -m QMA -n ‘CP.ZZ.00’ -t queue -p testuser
Entity testuser has the following authorizations for object CP.ZZ.00:
inq
crt
dsp

Display the profiles which determine the permissions for a user

dmpmqaut -m QMA -n ‘CP.ZZ.00’ -t queue -e -p testuser
profile: **
object type: queue
entity: test
entity type: group
authority: inq dsp

profile: @class
object type: queue
entity: test
entity type: group
authority: crt

The inq and dsp permissions came from the “**” profile for the test group.

Who is authorised to this queue?

This is a tricky one. I could find two hard ways of

dmpmqaut -m QMA -n ‘CP.ZZ.00’ -t queue 2> file

dmpmqcfg -m QMA -x authrec -o 1line > file

Then post-processing the output to extract group name and permissions, then merging the records.

For example to see the members of a group.

getent group |grep mqm

gives
mqm:x:1003:colinpaice,ibmsys1

I want to audit the access to the queues beginning with CP.

Another tricky one.

  • dmpmqcfg -m QMA -n ‘CP*’ -t queue -o 1line -x object > queuelist
  • This gives a list of all queues starting with CP. Process this list to select each queue in turn
  • dmpmqaut -m QMA -n $queuename -t queue -e > accesslist
  • This accesslist file has the profile name, the group and the permissions.
  • Take each group name, and extract the members of the group.
  • For each member of the list use
  • dspmqaut -m QMA -n $queue -t queue -p $userid
  • Take this list extract the permissions, and produce a pretty report.

If there is a better way I would love to hear it.

A partial solution is to take the output of

dmpmqaut -m QMA -l -t queue 2>list

dmpmqcfg -m QMA -x authrec -o 1line > file

Then use tools like grep to extract the records of interest, then use other tools to issue dmpmqaut -m QMA -n ‘….’ -t queue, capture the output and process it.

But this means you have to interpret generic profiles like ‘CP.**.INPUT’ yourself.

Sample code for post processing dmpmqcfg output

Below is a small Python program that takes the output from dmpmqcfg and processes it to replace groups with users in the groups, and gives one line per userid,permission,profile, source_group

#
# Copyright (c) 2020 Stromness Software Solutions.
# 
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Eclipse Public License v1.0
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
# 
# * Contributors:
# *   Colin Paice - Initial Contribution
# 
from sys import stdin
import grp

for line in stdin:
  sline= line.strip()
  s = sline.split(' ')
  if s[0] != "SET":
     continue
  p = s[2][9:-2] # get the profile
  g = s[3][7:-2] # get the group 
  try:
    ginfo = grp.getgrnam(g)
    ids = ginfo.gr_mem 
    if len(ids) == 0:  # group can be empty
       ids = [g]  
  except KeyError:
    ids = [g] # group not found - must be userid
  t = s[4][8:-1] #type

  a = s[5][8:-1]  # get the authorities
  aa = a.split(",")
# Output userid authority profile type group_it_came_from
  for aaa in aa:
    for i in ids:
       print(i,aaa,p,t.g) 

Invoke it as

cat data | python3 ~/python/sec.py |sort -k1,3|uniq |grep -v mqm 

grep -v mqm removes all accesses due to group mqm membership

Example output

colinpaice SET SYSTEM.SELECTION.VALIDATION.QUEUE QUEUE 
colinpaice SET TEMP QUEUE colinpaice
colinpaice SET XQMB QUEUE colinpaice
colinpaice SUB SYSTEM.ADMIN.TOPIC TOPIC colinpaice
testuser PUT COLIN_* QUEUE test
testuser PUT CP0000 QUEUE testuser

How to administer AMS policies, and use the set policy command.

I had been using the setmqspl command (on z/OS and midrange) to manage my AMS policies. This command has the drawback that if you want to change a policy, for example add a new recipipient, you had to specify the whole command. Jon Rumsey pointed out the mid range MQSC commands “set policy” and “display policy” which allow you to add, delete, or replace; recipients and signers.

Examples of midrange runmqsc set policy command

Exporting parameters

If you want to keep a copy of the AMS definitions you can use display policy command, but this gives output like RECIP(CN=BBB,C=GB), without quotes. The set policy command needs the value within single quotes. The dmpmqcfg command does not support AMS policies.

To be able to capture the output so you can reuse it, you need to use the dspmqspl -export command. This gives output like

setmqspl -m QMA -p ABC -s SHA512 -e AES256 -r “CN=BBB,C=GB” -c 0 -t 0

This gives the parameters if a format that can be used directly.

Add or remove recipients or signers

Using runmqsc define a policy using the default action(replace)

set policy(ABC) signalg(SHA512) recip(‘CN=AAA,C=GB’)  ENCALG(AES256) 

You can add a new recipient

set policy(ABC) signalg(SHA512) recip(‘CN=BBB,C=GB’) ENCALG(AES256) action(ADD)

You can now display it

DIS policy (ABC)

AMQ9086I: Display IBM MQ Advanced Message Security policy details.
POLICY(ABC) SIGNALG(SHA512)
ENCALG(AES256)
RECIP(CN=BBB,C=GB)
RECIP(CN=AAA,C=GB)
KEYREUSE(DISABLED)
ENFORCE

You can delete a recipient

set policy(ABC) SIGNALG(SHA256) ENCALG(AES128) RECIP(‘CN=AAA,C=GB’) action(remove)

and display it

DIS policy(Abc)
AMQ9086I: Display IBM MQ Advanced Message Security policy details.
POLICY(ABC) SIGNALG(SHA512)
ENCALG(AES256) RECIP(CN=BBB,C=GB)


KEYREUSE(DISABLED)
ENFORCE

You have to specify SIGNALG and/or ENCALG each time, but for action(REMOVE|ADD) it can have any valid value (except NONE). The value is only used when ACTION(REPLACE) is used, or ACTION() is omitted. The following will add the recipient, and not change the signalg or encalg values.

set policy(ABC) recip(‘CN=CCC,C=GB’) action(ADD) signalg(MD5) encalg(RC2)

You can specify multiple RECIP

set policy(ABC) signalg(SHA512) recip(‘CN=BBB,C=GB’) recip(‘CN=DDD,C=GB’) ENCALG(AES256) action(ADD)

or multiple signers

set policy(ABC) signalg(SHA512) signer(‘CN=BBB,C=GB’) signer(‘CN=DDD,C=GB’) ENCALG(AES256) action(ADD)

or multiple signers and recipients.

Changing other parameters

If want to change an algorithm, the tolerate|enforce that every message must be protected, or the key reuse, then you must use the action(replace), and specify all the parameters, so it might be easier to use setmqspl -m … -policy … -export, and output it to a file, then modify the file.

Administering AMS on z/OS

On z/OS (and mid-range) you have dspmqspl and setmqspl commands. With the setmqspl command, you replace the entire statement.

It is good practice to have a PDSE with all of your definitions in, one member per policy, or perhaps all policies in one member – depending on how many policies you have. If you have a problem with your queue manager, you have a copy of the definitions.

Another good practice is to take a copy of a definition before you make the change (and keep it unchanged), so you can roll back to it if you need to undo a change.

You can use the export command, to output all policies, or a selected policy. You can have this going into a sequential data set or a PDSE member. You might want to have two copies,

  1. The before image – from before the change
  2. The copy you update.

Of course you could always use the previous copy, but you cannot tell if someone has updated the definitions outside of your change control system, so taking a copy of the existing definitions is a good idea. You could always compare the previous copy, with the copy you just created to check there were no unauthorised changes.

You may want to make the same change to multiple queue managers, so having updates in a PDSE member is a good way of doing it. Just change the queue manager name and rerun the job.

On z/OS, remember to use the refresh command on the AMS address space for it to pick up any changes.

Other AMS blog posts

How do I process messages on the dead letter queue (DLQ)?

I was setting up security on my system, and using AMS to protect messages. I kept getting messages on the Dead Letter Queues. As messages on the DLQ have been around from before MQ V1 was shipped (they hit this problem in development), I was expecting that to process them would be easy. There are some good bits and some not so good bits with the IBM supplied solution. I was reminded of a “call and response narration” game we enjoyed in the pub from when I was a student which went ..

They are building a house in the street – (audience) Boo!
A public house – (audience)Hooray!
They don’t sell beer – (audience) Boo!
They give it away – (audience) Hooray!

For a supplied Dead Letter Queue handler it goes…

MQ provides a Dead Letter Handler program (runmqdlq) – Hooray!
On z/OS (CSQUDLQH) and midrange (runmqdlq). – Hooray.
It is rule based and can handle many scenarios – Hoorary!
But not some of the difficult ones – Boo!
The provide a set of sample programs on mid range (amqsdlq) – Hooray!
But they are not well documented, didn’t build straight off, and not available on z/OS – Boo.
It can process many similar messages in one go- Hooray,
But not process just one message – Boo.

Why are messages put on the DLQ?

If a local application tries to put a message to a queue, and the queue is full then the application gets a return code, and takes an action. The message is not lost – it wasn’t created, and the DLQ was not used. If a message comes in from another queue manager, and the channel tries putting the message and gets queue full, it cannot just throw the message away. It puts it onto the DLQ.

Messages could be put on a DLQ for many reasons.

  • A message came in from a remote queue manager and was put to a local queue, but the queue was at max depth, so was put to the DLQ. This may be due to a short lived problem. The DLQ handler can process the DLQ queue, and every 60 seconds try moving the message from the DLQ back to the original. You can configure the rules so if it tries 5 times and fails, then it moves the message to a different queue.
  • A message came in from a remote queue queue manager, but the channel userid was not authorised to put to the queue. In this case retrying every 60 seconds is unlikely to solve the problem. The administrator needs to take an action, such as grant access and retry the put, or remove the message.
  • When AMS is used, if an ID tries to get the message and there are problems, such as the ID of the signer of the message is not authorised, the message is put to the SYSTEM.PROTECTION.ERROR.QUEUE queue. To resolve this, the AMS configuration needs to be changed, or the message moved to a quarantine queue. Once the configuration has been changed, put the message back on the queue for retry.

The runmqdlq handler provided with MQ

This is a bit of a strange beast. It is rule based so you can configure rules to select messages with certain properies and take actions, such as retry, or move to a different queue.

The program on midrange is runmqdlq, and on z/OS CSQUDLQH.

The syntax for runmqdlq is

runmqdlq [-u userid] MYDEAD.QUEUE QMA <qrule.rul

you have to pipe the file into stdin, until an empty line is processed. I would have preferred a -f filename option.

To end runmqdlq, set the input queue to get(DISABLED) because Ctrl-C does not work.

It processes message silently, unless there are any problems, for example I got

Dead-letter queue handler unable to put message: Rule 6 Reason 2035.

I had several problem messages on the DLQ, but I could not specify one message and get runmqdlq to process it, so I had to write a program to move one message to a different queue, then I could use runmqdlq. There is lots of good stuff in runmqdlq, but doesn’t quite do the job.

Understanding the rules.

The rules are the same for z/OS as mid-range.

Messages are read from the specified DLQ queue, and processed with a set of rules. The rules are described here. You can select on properties in the MQMD or the DLQ header. For example

DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(5)

DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(FWD) FWD(MYQUEUEOVERFLOW) HEADER(YES)

DEST(INQ*) PERSIST(MQPER_NON_PERSISTENT ACTION(DISCARD)

DEST(INQ*) PERSIST(MQPER_PERSISTENT ACTION(LEAVE)

Runmqdlq wakes up on new messages, and scans the queue periodically (the default RETRYINT is 60 seconds). It keeps track of messages on the queue, for example how many times it has retried an operation. For each message it scans the rules until it finds the first matching rule, then takes the action.

For for the rules above

DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(5)

DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(FWD) FWD(MYQUEUEOVERFLOW) HEADER(YES)

If a messages destination was MYQUEUE, and the reason code was MQRC_Q_FULL, it retries the put to the queue, at most 5 times. After 5 attempts, the next time the first rule is skipped, the second rule is used, and the message is forwarded to the queue MYQUEUEOVERFLOW keeping the DLQ header.

DEST(INQ*) PERSIST(MQPER_NON_PERSISTENT ACTION(DISCARD)

For message destination INQ* and non persistent messages, then just discard them.

DEST(INQ*) PERSIST(MQPER_PERSISTENT ACTION(LEAVE)

For message destination INQ* and persistent messages, then just leave them on the queue, for some other processing.

If runmqdlq is restarted, then all processing is reset, as all state information is kept in memory.

You should have a strategy for processing the DLQ.

For example, see Planning for MQ Dead Letter Queue handling, because you do not want thousands of non persistent inquiry messages filling up the DLQ, and preventing important persistent messages from being put onto the DLQ.

You may want to provide an audit trail of messages on the DLQ, so when someone phones up and says “MQ has lost my message”, you can look in the DLQ error logs, and say, “no… it is still in MQ, on the PENDING_SECURITY_ACTION queue, waiting for the security people to give the userid permission to process the message”.

Writing your own DLQ handler

While the MQ provided program is pretty good, there are times when you need a bit more, for example

  • Writing an audit message for each message processed, and what action was taken.
  • Printing out information about the message, such as queue name, putter, reason code etc
  • Moving one message, based on message ID or Correlid to another queue.

A one pass application is not difficult to create, it is a typical server application. A multi-pass application is much harder as you need to remember which messages have been processed.

  • I do not know if it is better to get with convert or not, especially if you are using AMS.
  • Print message information. You can use printMD from the amqsbcg0.c sample to print the MD.
  • You can create a similar function for printing the DLQ header. You may have to handle conversion yourself, for example big-indian/little endian numbers
  • You can print a hex string such as msgid using

for (ii = 0 ; ii < sizeof(msgid) ; ii++)
printf(“%02hhX”,msgid[ii])

  • If you specify a msgid as a parameter, you can read a hex string into a byte array using the following. The arrray had to be unsigned char to for it to work,otherwise you get negative numbers

unsignchar msgid[24];
int i;
for (i = 0; i < sizeof(msgid); i++)
{
sscanf(pIn + (i * 2), “%2hhx”, &msgid[i]);
}

Remove the DLQ header if needed.

mqoo_server =… MQOO_SAVE_ALL_CONTEXT ;

MQGET(hConn,
serverHandle,
&mqmd,
&mqgmo,
lBuffer,
pBuffer,
&messageLength,
&mqcc,
&mqrc);

// move the format and CCSID from the DLQ back to the mqmd
memcpy(&mqmd.Format,&pMQDLH -> Format,sizeof(mqmd.Format));
memcpy(&mqmd.CodedCharSetId,&pMQDLH -> CodedCharSetId,sizeof(mqmd.CodedCharSetId));

mqpmo.Options += MQPMO_PASS_ALL_CONTEXT;
mqpmo.Context = serverHandle;
long lDLQH = sizeof(MQDLH);

MQPUT1( hConn,
&replyOD ,
&mqmd ,
&mqpmo,
messageLength -lDLQH, // reduce the data by the size of the DLQ
pBuffer+lDLQH,// point past the DLQ
&mqcc,
&mqrc );

You can teach an old MQ program(mer) new tricks!

I wrote a program which could be used with local bindings on Linux, or as a client. Doing what I have done for 25 years, and following the IBM documentation I had a makefile with a create for each type.

gcc -m64 -o mer me.o -L/opt/mqm/lib64 -Wl,-rpath=/opt/mqm/lib64 -Wl,-rpath=/usr/lib64 -lmqm
gcc -m64 -o merc me.o -L/opt/mqm/lib64 -Wl,-rpath=/opt/mqm/lib64 -Wl,-rpath=/usr/lib64 -lmqic

Where -lmqm was for local bindings, and -lmqic was for client bindings.

For about the last 10 years, you have only needed one executable, not two!

Thanks to Morag Hughson of MQGem who pointed this out and said You can make a client connection using something linked with mqm.lib. Just set MQ_CONNECT_TYPE to CLIENT. See here.

I only need one program mer, and do not need the client version merc. I used

export MQ_CONNECT_TYPE=CLIENT
export MQCCDTURL=/home/colinpaice/c/ccdt.json
./mer CSQ9 CP0000

and it worked! (First time)

This support has been there since MQ 7.1, so as long as you have compiled your programs with MQ 7.1 or later you can use this support.

I’ll drop an email to Hursley because the documentation for generating a program says, for example

C client application, 64-bit, non-threaded

gcc -m64 -o amqsputC_64 amqsput0.c -I MQ_INSTALLATION_PATH/inc -L MQ_INSTALLATION_PATH/lib64 -Wl,-rpath=MQ_INSTALLATION_PATH/lib64 -Wl,-rpath=/usr/lib64 -lmqic

C server application, 64-bit, non-threaded

gcc -m64 -o amqsput_64 amqsput0.c -I MQ_INSTALLATION_PATH/inc -L MQ_INSTALLATION_PATH/lib64 -Wl,-rpath=MQ_INSTALLATION_PATH/lib64 -Wl,-rpath=/usr/lib64 -lmqm

It would be good if they told you about this great facility, and not only have it hidden away.

You could just build it once, and set the environment variable.

Using it

The documentation for MQ_CONNECT_TYPE says this is for MQCONNX.

If your application uses MQCONNX, then it will try local, then try as a client (using MQCCDTURL environment variable), and you do not even need to specify MQ_CONNECT_TYPE. You can force it to use local or client by speciying MQ_CONNECT_TYPE.

My application was using the old style of MQCONN. For this to work I had to specify MQ_CONNECT_TYPE=CLIENT (and the MQCCDTURL).

You also might consider upgrading your application so you use MQCONNX instead of the MQCONN. All you need is

  • MQCNO cno = {MQCNO_DEFAULT}; /* Connect Options*/
  • cno.Options = … ;
  • change MQCONN to MQCONNX and add the &cno.

plus testing it(for several weeks) of course.

Convert MQCONN to MQCONNX and you get connection to the local machine or to a client automatically – you do not need the MQ_CONNECT_TYPE.

See, you can get an old application to do new tricks.

Planning for MQ Dead Letter Queue handling.

With MQ, if a message cannot be successfully delivered, it can be put on a Dead Letter Queue for later processing.

You can have multiple queues

  • The system dead letter queue, where the MQ puts messages it cannot processed,
  • Application dead letter queues, and application can put messages to a queue,
  • The AMS dead letter queue for messages which had errors during get or put, for example a certificate mismatch.

Messages can be put to these queues for a variety of reasons.

  • Transient problems
    • If a channel is putting a message to a queue, and the queue is full, then the channel can put the message to the Dead Letter Queue. The DLQ handler can then try to put the message to the original queue, and retry a number of times after an interval. If the queue full condition was transient, then the DLQ handler is likely to succeed. If an application stops processing a queue, you can get quickly get thousands of messages on the DLQ queue.
    • The queue is put disabled. A queue can be set to put disabled, for example to stop messages from going onto a queue during queue maintenance. Once the maintenance has been done the queue can have put enabled.
  • Administration
    • The putting channel is not authorised to put to the queue, so the message gets put to the DLQ. An administrator needs to check to see if the putter is allowed to put the message. If so, fix the security and put the message back on original queue. If not remove the message, and educate the developer.
    • An AMS protected message has a problem, for example an unauthorised user has signed a message, or the id getting the message does not have a certificate to decrypt a message. You need to resolve any local certificate problems, or send the original message back to the requester saying it is in error.
  • Application
    • The message is too large for the queue. The administrator needs to educate the developer and/or make the queue max message size larger.

You may have a policy that non persistent messages for a particular queue which end up on the dead letter queue should be purged. Persistent message for another queue should have special treatment.

You may want administrators to be able to look at the meta data about a message, destination queue, MSGID, the list of recipients who can decrypt a message; but not to look at the message content.

Setting up your environment to cover these areas need considerable planning.

Implementing a solution

You want to try to keep the main DLQ close to empty, for example if your DLQ fills up with non persistent inquiries, then putting an important persistent message to the DLQ may fail.

You can use the runmqdlq program on midrange or CSQUDLQH on z/OS, to specify rules for automatic processing of messages on the DLQ.

You can select on attributes like original destination queue name, the reason why the message was on the DLQ, userid in the MQMD; and specify an action

  • Retry the put to the original queue
  • Move to another queue
  • Purge it
  • Leave it

When a message is processed on the DLQ, the rules are applied, and the action of the first matching rule is applied. For example

DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(5)
DESTQ(MYQUEUE) REASON(MQRC_Q_FULL) ACTION(FWD) FWD(MYQUEUEOVERFLOW) HEADER(YES)

This says that if a messages destination was MYQUEUE, and the reason code was MQRC_Q_FULL, it retries the put to the queue, at most 5 times. After 5 attempts, the first rule is skipped, the second rule is used, and the message is forwarded to the queue MYQUEUEOVERFLOW keeping the DLQ header.

DEST(INQ*) PERSIST(MQPER_NON_PERSISTENT ACTION(DISCARD)

For message destination INQ* and non persistent messages, then just discard them.

DEST(INQ*) PERSIST(MQPER_PERSISTENT ACTION(LEAVE)

For message destination INQ* and persistent messages, then just leave them on the queue, for some other processing.

If runmqdlq or CSQUDLQH is restarted, then all processing is reset.

Generic rules

For transient type problems you may want to consider

  • Non persistent messages for a set of queues get purged, dont even try to put them back on the queue.
  • Persistent messages for INVOICE* queues get moved to INVOICE_DLQ queue, where you have another DLQ monitor running on the queue.

For administrator type problems

  • You could pass non persistent messages to an admin_DLQ_NP queue, and have a program which reads the meta data, and prints it to a file, then deletes the original message
  • You could pass persistent messages to an admin_DLQ_P queue
    • have a program which reads the meta data, and prints it to a file, and leaves the message on the queue.
    • Using the meta information resolve the problem.
    • Have another program which takes the msgid and correlid as input parameters, then puts the message on the original queue. (If there is only one message, you could use the default DLQ handler to do this.)

For AMS problems

  • This is complicated by having to use a different queue. If the DLQ handler tries to put to the AMS protected queue, it will be “protected” (enciphered) again. You need to use put the message to an alias queue, with the original queue as the target. On midrange Java and C clients can disable AMS processing, either by using an environment variable, or through the MQCLIENT.ini file. See here.
  • This is also complicated by possibly needing access to information in the payload, such as the list of recipients, and decrypting the message to get the DN of the signer.
    • Once you have resolved the problem, have another program which takes the msgid and correlid as input parameters, and puts the message on the alias queue (if there is only one message you could use the default DLQ handler to do this).

How do I check it I have got it right?

It is worth putting a process in place to monitor the depth of the dead letter queue, and if it does not become empty a few a minutes, display the contents of the queue, and add rules to handle the residual messages.

I do not think that IBM provides a list of return codes of messages that it puts onto the DLQ, I think you’ll have to go through the list (over 500!), and put a rule in place for each one. If an application invents its own return codes, you may need rules for these as well.

My quick look at the list includes

Common problems

2053 (0805) (RC2053): MQRC_Q_FULL
2056 (0808) (RC2056): MQRC_Q_SPACE_NOT_AVAILABLE
2071 (0817) (RC2071): MQRC_STORAGE_NOT_AVAILABLE

Other problems
2030 (07EE) (RC2030): MQRC_MSG_TOO_BIG_FOR_Q
2031 (07EF) (RC2031): MQRC_MSG_TOO_BIG_FOR_Q_MGR
2033 (07F1) (RC2033): MQRC_NO_MSG_AVAILABLE
2051 (0803) (RC2051): MQRC_PUT_INHIBITED
2052 (0804) (RC2052): MQRC_Q_DELETED
2053 (0805) (RC2053): MQRC_Q_FULL
2056 (0808) (RC2056): MQRC_Q_SPACE_NOT_AVAILABLE
2071 (0817) (RC2071): MQRC_STORAGE_NOT_AVAILABLE
2102 (0836) (RC2102): MQRC_RESOURCE_PROBLEM
2120 (0848) (RC2120): MQRC_CONVERTED_MSG_TOO_BIG
2141 (085D) (RC2141): MQRC_DLH_ERROR
2142 (085E) (RC2142): MQRC_HEADER_ERROR
2148 (0864) (RC2148): MQRC_IIH_ERROR
2149 (0865) (RC2149): MQRC_PCF_ERROR
2150 (0866) (RC2150): MQRC_DBCS_ERROR
[z/OS]2342 (0926) (RC2342): MQRC_DB2_NOT_AVAILABLE
[z/OS]2345 (0929) (RC2345): MQRC_CF_NOT_AVAILABLE
[z/OS]2348 (092C) (RC2348): MQRC_CF_STRUC_AUTH_FAILED
[z/OS]2349 (092D) (RC2349): MQRC_CF_STRUC_ERROR
2362 (093A) (RC2362): MQRC_BACKOUT_THRESHOLD_REACHED

You may want common ones, such as queue full and not authorised, on a per queue basis, and all the less common ones, such as all of the z/OS ones all putting to one “admin” queue.

Building amqsdlq sample Dead Letter Handler.

MQ provides a sample Dead Letter Queue Handler in /opt/mqm/samp/dlq/ (on Ubuntu). It looks and behaves just like runmqdlq. I think trying to extend it, for example to filter on MSGID or CORRELID would be difficult, and it would be easier to write a small program just to process one message with input queue, and pass filter parameters.

Build it

There are no instructions in the IBM documentation on how to build this. I used the following

  • create a directory to contain the code, for example mkdir ~/dlq
  • go to this directory cd ~/dlq
  • copy the code from MQ cp -r /opt/mqm/samp/dlq/* .
  • make the directory rw chmod +w ~/dlq
  • install a yacc compiler sudo apt install byacc
  • issue the make command make all -f amqodqx.mk

If you get

gcc -m64 -I. -I/opt/mqm/inc -c -o amqodqka.o amqodqka.c
Assembler messages:
Fatal error: can’t create amqodqka.o: Permission denied

Your directory is not read/write. Use the chmod command.

If you get

yacc amqodqma.y
make: yacc: Command not found

You need to install a yacc compiler. I used sudo apt install byacc .

I dont undestand why I got this message, because the build for the yacc files is commented out. It may be there is a default make for yacc.

The YACC ( compiler generator) produces a c file. If you are not changing the keywords, you could rename the .y and .l files, so they are not used, and so use the shipped c files. In this case you will not need the yacc compiler.

Run it

  • export ODQ_MSG=amqsdlq.msg This is needed for the message catalog
  • ./amqsdlq SYSTEM.PROTECTION.ERROR.QUEUE QMA < /home/colinpaice/mqams/dlq.rul
    • for a while it failed with a syntax error in line 1 of the file, I rebuilt it, then suddenly it worked.
    • Ctrl-c ended it
  • ./amqsdlq ? gave me a segmentation core dump

I changed the make file to build a client

cc -m64 -o amqsdlqc… -lmqic_r

You should ensure you use a TLS channel if you do run this as a client.

I have a message on the AMS DLQ – what can I do about it?

If AMS has problems with a protected message, AMS can put the message on the SYSTEM.PROTECTION.ERROR.QUEUE queue. This blog post discusses what you can do about it. I consider this a hard problem – not in the same league as trying to simulate the beginnings of the Universe – more like climbing Ben Nevis mountain in Scotland, when you are only used to walking down to the shops.

What are the problems?

There are several problems you need to consider

  1. Why is the message on the queue? Is it a problem with the putting application, or with the getting environment?
  2. Which user had the problem. For example it may not be obvious which application instance had a problem, if applications come in through one channel, and many users have the same MCA userid.
  3. What you need to do about it to get the message reprocessed, and prevent future problems.

Why is the message on the queue

The messages could be on the queue because

  • The certificate was signed, but the DN of the signer is not in the setmqspl list of authorised signers (-a). This is an example of an MQ configuration problem
  • The user getting a message was not able to verify the signers certificate sent in the message, for example it is missing the CA of the signer, or missing the signers self signed certificate. This is an example of a user’s configuration problem.
  • The message was encrypted, but the user getting the message did not have access to a private certificate to allow the message to be decrypted. The user’s DN needs to be added to the recipients when the message is put and enciphered (or the user needs to be stopped from getting messages from this queue). This is an example of the putting of the message message is missing configuration information.

What other information is there to help me?

If you know which id had the problem, there should be information in the error logs. For example a problems within a Java JMS client program may write to mqjms.log.0. A local application may write to the queue manager’s error log, for example /var/mqm/qmgrs/QMA/error/*01*

In the mqjms.log.0 I got

5 April 2021 at 14:53:23 BST[main] com.ibm.mq.ese.prot.MessageProtectionBCImpl
The receiver of this encrypted message is not on the message recipient list ‘CN=ja2,O=aaaa,C=GB’
The certificate of a user that is receiving a message is not on the message RecipientsInfo list.

Verify that the user is on a recipients list in a security policy definition.

5 April 2021 at 14:53:23 BST[main] com.ibm.mq.ese.intercept.JmqiGetInterceptorImpl
The IBM MQ Advanced Message Security Java interceptor failed to unprotect the received message.
An error occurred when the IBM MQ Advanced Message Security Java interceptor was unprotecting the received message.

See subsequent messages in the exception for more details about the cause of the error

5 April 2021 at 14:53:23 BST[main] com.ibm.mq.ese.service.EseMQServiceImpl
The IBM MQ Advanced Message Security interceptor has put a defective message on error handling queue ‘SYSTEM.PROTECTION.ERROR.QUEUE ‘.

(On z/OS the messages are less helpful.)

On the SYSTEM.PROTECTION.ERROR.QUEUE queue there was a message with a Dead Letter Header (DLH) with a reason 2063 0x0000080f MQRC_SECURITY_ERROR.

From the MQMD you can see the time the message was put to the queue, the putting application, and the user identification. This userIdentified may have been set for example by the channel MCAUSER, or CHLAUTH, and so you do not always know where the original request came from.

If you are testing then you will know what caused the error.

If you are in a production like environment, you know the application, and as you will have configured all user keystores the same you may not need to know which specific user caused the problem. If there is a problem with a missing certificate, then you fix the problem, redeploy the keystore to all your users (as part of your automated process) and try again.

How do you tell what the problem is.

Your systems administrator needs a process for extracting meta data about messages, while keeping the application payload protected. You could build a process around displaying the recipients and signer from How do I find the recipients and signer of an AMS message? The systems administrator needs to know

  • the original queue name with the problem
  • the time, date, and user that put the message
  • the msgid and correlid of the message – so if you put it back on the queue, you know which message to process.
  • from the message, the type of protection: Integrity (it was signed), Encryption (it was encrypted), Privacy (it was encrypted with a signed payload)
  • any id information from the message, such as recipient DN’s, and the signer DN. See this post.
  • you may have to have some special processing to decrypt the Privacy payload, just to extract the signer information.

If this process can be automated, then any application content can be kept secure.

With this information and your “up to date application work book” (do you have one of these?) , you should be able to identify the problem.

Once you have fixed the root cause of the problem….

The fix may be to change the setmqspl to add an authorised signer, or to add a certificate to the recipients keystore, you need to get the message reprocessed.

  • You get the specific message from the message id and correlid.
  • You need to remove the DLQ header from the message.
  • You need some special set up for the queue. If you try to put to the original queue, it will get the AMS protection again, for example re-encrypted or resigned. You need a queue alias so you put to the queue alias bypassing any AMS processing.

There are lots of things you need to consider, which is why I consider this a hard problem.

This application would be a good example where message handle is used to “move” the message.

MQCRTMH(hConn,&cmho,&hMsg,&CompCode,&Reason);

gmo.MsgHandle = hMsg;

MQGET(hConn,….);

pmo.Action = MQACTP_FORWARD;

pmo.OriginalMsgHandle =hmsg

MQPUT(…)

I found Learn to code the MQ Message Property MQI calls from MQGEM software useful in understanding message handle and message properties, and how to delete the DLQ header.

Other AMS blog posts

How do I find the recipients and signer of an AMS message?

When you have a problem trying to decrypt/get a protected message, the message can end up on the AMS dead letter queue. This blog post explains how to look into the message and find the list of recipient, and the signer of the data.

This started off as a short blog post, then as I added more background and more detailed instructions it just grew!

The sections are

Background on the format of the protected message.

The protected message conforms to the PKCS#7 protocol as defined here.

The specification defines six formats, of which three are:

  • signed data
  • envelopedData where data is wrapped in an opaque envelope – it has been encrypted
  • signedAndEnveloped.

These may look like the AMS classes: Integerity, Confidential, and Privacy; close – but no cigar. AMS does the signing before encryption, and so only signed data, and envelopedData are used. To see who signed a privacy message, you have to decrypt it.

I wrote a program to browse the messages on a queue, and write the protected data out to a Linux file. You can use openssl tools to explore file.

There is a PDMQ header at the front of the data. This has a variable length

  • at offset 0 is “PDMQ” in ASCII.
  • like other variable length MQ structures, at offset 8 is an MQLONG StrucLength. Don’t forget you may need to convert from big-endian to little-endian on some platforms.
  • The data after the header is the data is ASN.1 encoded, starting with x’30’.

30 second overview of ASN.1 encoding

ASN.1 encoding takes data like “typeOfRequest,{[id1,key1],[id2,key2]} and converts it into

  • a SEQUENCE (the items have an order) of typeOfRequest, keys
  • where keys are a SET (because it does not matter if the order is id1, id2, or id2, id1)
  • ID is an integer…

Terms like countryName C=GB map into a String of type 2.5.4.6 with length 2 and value GB.

When this is decoded it take 2.5.4.6 and says this is CountryName. You can look up the id here. There is a document with popular ones in it.

You can use openssl to format a file of ASN.1 encoded data and interpret the values. For example

openssl asn1parse -inform DER -in colin.pkj -i -offset 104 > asn1

says

  • openssl asn1parse this command
  • -inform DER in format is decimal encoded (not base 64)
  • -in colin.pkj this is the input file
  • -i says indent the output to prettify it
  • -offset 104 skip over the PDMQ header
  • > asn1 write the output to the file asn1

When I changed my program to skip over the PMDQ header and write out just the data, I didn’t need the -offset parameter.

I’ve also used this to display the contents of a certificate in .der format.


What does signed data look like?

The spec says signed data looks like

ContentInfo ::= SEQUENCE {
     contentType ContentType,
     content
}
where contentType is SignedData... 
SignedData ::= SEQUENCE {
     version Version,
     digestAlgorithms DigestAlgorithmIdentifiers,
     contentInfo ContentInfo,
     certificates
     signerInfos SignerInfos
 }

The start of the output of the openssl asn1parse command for an integrity message contained

    0:d=0  hl=4 l=1723 cons: SEQUENCE          
    4:d=1  hl=2 l=   9 prim:  OBJECT            :pkcs7-signedData
   15:d=1  hl=4 l=1708 cons:  cont [ 0 ]        
   19:d=2  hl=4 l=1704 cons:   SEQUENCE          
   23:d=3  hl=2 l=   1 prim:    INTEGER           :01
   26:d=3  hl=2 l=  15 cons:    SET               
   28:d=4  hl=2 l=  13 cons:     SEQUENCE          
   30:d=5  hl=2 l=   9 prim:      OBJECT            :sha256
   41:d=5  hl=2 l=   0 prim:      NULL              
   43:d=3  hl=2 l=  20 cons:    SEQUENCE          
   45:d=4  hl=2 l=   9 prim:     OBJECT            :pkcs7-data
   56:d=4  hl=2 l=   7 cons:     cont [ 0 ]        
   58:d=5  hl=2 l=   5 prim:      OCTET STRING      :AAAAA

The columns are

  • offset from the start
  • d= nesting depth, so a sequence within a set, within a sequence would be d=3
  • dl= length of length field
  • l= length of the field
  • cons|prim. I think of these as
    • cons is construction … SEQ, SET etc.
    • prim is primary data
  • SEQUENCE, OBJECT, INTEGER type of value

Looking at the output

  • at offset 4 we have ContentType which is :pkcs7-signedData
  • at offset 23 we have the SignedData version :01
  • at offset 30 we have a SET of digestAlgorithms, there is one in the SEQUENCE and it has the value :sha256
  • at offset 45 we have contentInfo :pkcs7-data
  • at offset 58 we have the message data, the very boring AAAAA.

After the data we have signer info

SignerInfo ::= SEQUENCE {
     version Version,
     issuerAndSerialNumber IssuerAndSerialNumber,
     digestAlgorithm DigestAlgorithmIdentifier,
     authenticatedAttributes
       [0] IMPLICIT Attributes OPTIONAL,
     digestEncryptionAlgorithm
       DigestEncryptionAlgorithmIdentifier,
     encryptedDigest EncryptedDigest,
     unauthenticatedAttributes
       [1] IMPLICIT Attributes OPTIONAL }

Extracting the interesting fields out of the data (to make it easier to read)

  • Certificate serial number: 01B0
  • Issuer:
    • countryName:GB,
    • organizationName:SSS,
    • organizationalUnitName: CA,
    • commonName:SSCARSA1024
  • Not before time: 210328172922Z
  • Not after time: 231223172922Z
  • Subject:
    • countryName:GB,
    • organizationName:aaaa,
    • commonName: ja2
  • digestAlgorithm: rsaEncryption
  • X509v3 Key Usage:….
  • X509v3 Subject Alternative Name…
  • digestEncryptedAlgorithm: sha256WithRSAEncryption

What does encrypted data look like?

The spec says

EnvelopedData ::= SEQUENCE {
     version Version,
     recipientInfos RecipientInfos,
     encryptedContentInfo EncryptedContentInfo }

   RecipientInfos ::= SET OF RecipientInfo

   EncryptedContentInfo ::= SEQUENCE {
     contentType ContentType,
     contentEncryptionAlgorithm
       ContentEncryptionAlgorithmIdentifier,
     encryptedContent
       [0] IMPLICIT EncryptedContent OPTIONAL }

Instead of listing the DN’s of the recipients, it listed CA certificate + serial number (which equates to the same thing). As a self signed has signed itself, there is an entry for the self signed.


4:d=1 hl=2 l= 9 prim: OBJECT :pkcs7-envelopedData

23:d=3 hl=2 l= 1 prim: INTEGER :00

42:d=8 hl=2 l= 9 cons: SEQUENCE
self signed 
C=GB,O=AAA,CN=colinPaice,  serial=606439D5
44:d=9 hl=2 l= 3 prim:     OBJECT          :countryName
49:d=9 hl=2 l= 2 prim:     PRINTABLESTRING :GB

57:d=9 hl=2 l= 3 prim:    OBJECT            :organizationName
62:d=9 hl=2 l= 3 prim:    PRINTABLESTRING   :AAA  

71:d=9 hl=2 l= 3 prim:    OBJECT            :commonName
76:d=9 hl=2 l= 11 prim:   PRINTABLESTRING   :colin paice
89:d=6 hl=2 l= 4 prim:    INTEGER           :606439D5

97:d=6 hl=2 l= 9 prim:    OBJECT            :rsaEncryption

the following is the encrypted key for this user
110:d=5 hl=3 l= 128 prim: OCTET STRING [HEX DUMP]:861C...F2
241:d=4 hl=4 l= 348 cons: SEQUENCE
245:d=5 hl=2 l= 1 prim: INTEGER :00
248:d=5 hl=2 l= 68 cons: SEQUENCE

This is the issuer C=GB,O=SSS,OU=CA,CN=SSCARSA1024, serial 1B0
256:d=9 hl=2 l= 3 prim:    OBJECT          :countryName
261:d=9 hl=2 l= 2 prim:    PRINTABLESTRING :GB
269:d=9 hl=2 l= 3 prim:    OBJECT          :organizationName
274:d=9 hl=2 l= 3 prim:    UTF8STRING      :SSS
283:d=9 hl=2 l= 3 prim:    OBJECT          :organizationalUnitName
288:d=9 hl=2 l= 2 prim:    UTF8STRING      :CA
296:d=9 hl=2 l= 3 prim:    OBJECT          :commonName
301:d=9 hl=2 l= 11 prim:   UTF8STRING      :SSCARSA1024
314:d=6 hl=2 l= 2 prim:   INTEGER          :01B0
318:d=5 hl=2 l=13 cons: SEQUENCE          
320:d=6 hl=2 l= 9 prim:  OBJECT            :rsaEncryption
the following is the encrypted key for this user
333:d=5 hl=4 l=256 prim:OCTET STRING      [HEX DUMP]:35..7BF

This is for Issuer O=TEMP, OU=TEST,CN=TEMP4Certification Authority Serial = 5D
593:d=4 hl=4 l= 354 cons: SEQUENCE          
597:d=5 hl=2 l=   1 prim: INTEGER           :00
608:d=9 hl=2 l=   3 prim:   OBJECT            :organizationName
613:d=9 hl=2 l=   4 prim:   PRINTABLESTRING   :TEMP
623:d=9 hl=2 l=   3 prim:   OBJECT            :organizationalUnitName
628:d=9 hl=2 l=   4 prim:   PRINTABLESTRING   :TEST
638:d=9 hl=2 l=   3 prim:   OBJECT            :commonName
643:d=9 hl=2 l=  28 prim:   PRINTABLESTRING   :TEMP4Certification Authority
673:d=6 hl=2 l=   1 prim:  INTEGER           :5D
678:d=6 hl=2 l=   9 prim:  OBJECT            :rsaEncryption
the following is the encrypted key for this user
691:d=5 hl=4 l= 256 prim:OCTET STRING      [HEX DUMP]:6A...68
951:d=3 hl=2 l=  76 cons: SEQUENCE          
953:d=4 hl=2 l=   9 prim:  OBJECT            :pkcs7-data
964:d=4 hl=2 l=  29 cons:  SEQUENCE          
966:d=5 hl=2 l=   9 prim:   OBJECT            :aes-128-cbc
977:d=5 hl=2 l=  16 prim:   OCTET STRING      [HEX DUMP]:9D...543A

There were three recipients configured for the queue (setmqspl -r option)

  • CN=ja2, O=aaaa, C=GB. See offset 256 to 333, giving the Issuer, the certificate serial number, encryption type, and the hex dump of the encrypted key.
    • Serial Number: 01 B0
    • Issued by: CN=SSCARSA1024, OU=CA, O=SSS, C=GB
  • CN=colin paice, O=AAA, C=GB See offset 44 to 110, giving the Issuer (this is self signed), the certificate serial number, encryption type, and the hex dump of thea encrypted key.
    • Serial Number: 60 64 39 D5
    • Issued by: CN=colin paice, O=AAA, C=GB
  • CN=ADCDB,O=SSS. See offset 597 to 691, giving the Issuer, the certificate serial number, encryption type, and the hex dump of the encrypted key.
    • Serial Number: 5D
    • Issued by: CN=TEMP4Certification Authority, OU=TEST, O=TEMP

It was not easy to take the issuer information and map this to the Subject. I had to go through the whole of my keystore to find the information.

How to map Issuer+ serial to subject

For .jks keystores you can use

keytool -list -keystore ja2.jks -storepass zpassword -v |grep ‘Serial\|Issuer:\|Owner:’ | less

and search for the serial Issuer CN, and the serial number which follows it. For example

Owner: CN=ja2, O=aaaa, C=GB
Issuer: CN=SSCARSA1024, OU=CA, O=SSS, C=GB
Serial number: 1b0

For .CMS I used a bash shell script to extract the contents, then display the details of each label.

ks="/home/colinpaice/mq/zzserver.kdb"
pw="password"

z=runmqakm -cert -list -db $ks -pw $pw | awk '{print $2}'
for x in $z;do
     runmqakm -cert -details -db $ks -pw $pw -label $x |grep 'Serial\|Issuer\|Subject'
done

What encryption algorithm was used?

At offset 966 is

966:d=5 hl=2 l= 9 prim: OBJECT :aes-128-cbc

The aes128 matches the -e AES128 option on the queue.

What does privacy look like?

Just like encrypted. it has

955:d=4 hl=2 l= 9 prim: OBJECT :pkcs7-signedData

instead of

953:d=4 hl=2 l=9 prim: OBJECT:pkcs7-data

The signing data is encrypted, so cannot be viewed.

Decrypting the data

If you have a file of the data, you can use openssl cms to decrypt it. The example below uses the private key for ja2 in ja2.key.pem. ( I knew it was ja2’s key file from the information above).

openssl cms -decrypt -in colin.pkj -inform der -inkey ~/ssl/ssl2/ja2.key.pem -out colin.der

Now that it is decrypted, display the signed package

openssl asn1parse -inform DER -in colin.der -i

and I can see information about the signer: the issuer, the signer and the signing time; and the payload.

Using Java to look at the data

You can use Java to process the message, using code from bouncycastle (used by MQ).

Read the message as a byte array (data) and use

import org.bouncycastle.asn1.ASN1InputStream;
import org.bouncycastle.asn1.ASN1Primitive;
import org.bouncycastle.asn1.util.ASN1Dump;   
....  
ASN1InputStream ais = new ASN1InputStream(data);
ASN1Primitive obj = ais.readObject();
System.out.println(ASN1Dump.dumpAsString(obj, true));
       

This produces output like the ans1parse above but without the conversion of 2.5.4.6 to CountryName etc. You can either guess the meaning, or look it up here.


ObjectIdentifier(1.2.840.113549.1.7.2)
 Tagged [0]
  Sequence
   Integer(1)
   Set
    Sequence
     ObjectIdentifier(2.16.840.1.101.3.4.2.1)
     NULL
   Sequence
    ObjectIdentifier(1.2.840.113549.1.7.1)
    Tagged [0]
     DER Octet String[3] 
      424242 BBB
    Tagged [0]
   Sequence
    Sequence
     Tagged [0]
      Integer(2)
      Integer(432)
      Sequence
       ObjectIdentifier(1.2.840.113549.1.1.11)
       NULL
      Sequence
       Set
        Sequence
         ObjectIdentifier(2.5.4.6)
         PrintableString(GB) 
       Set
        Sequence
         ObjectIdentifier(2.5.4.10)
          UTF8String(SSS) 
       Set
       Sequence
        ObjectIdentifier(2.5.4.11)
        UTF8String(CA) 
       Set
        Sequence
         ObjectIdentifier(2.5.4.3)
         UTF8String(SSCARSA1024) 
       Sequence
        UTCTime(210328172922GMT+00:00) 
        UTCTime(231223172922GMT+00:00) 

Other AMS blog posts

Adding more users to AMS

Having got a basic AMS set up working (on z/OS), it takes a bit of planning (and getting your head round it) to add more users and get the certificates in the right place when you have more than one queue manager. For example for someone to get an encrypted message the putting userid on the remote queue manager needs the getter’s public certificate. If there are 1000 potential putters, you start to see the complexity of the problem.

As I have tried to implement AMS across my baby enterprise I’ve realised you need a process for maintaining the key stores. When using multiple queue managers you have to remember which queue manger needs which options on the setmqspl command.

Basic key store management

z/OS key rings

On z/OS, each local userid has its own private key on its own keyring, with all of the shared public keys on the xxxxAMSM userid’s keyring (userid/drq.ams.keyring). Update the xxxxAMSM’x keyring, and every one picks up the change – great – easy.

You can have multiple queue managers in a single z/OS image or a sysplex, each with the xxxxAMSM address space. If they use the same started task userid, they will share the same keyring. (So to have queue managers use a different keyring, they will need a different userid.)

If any of the users are outside of the RACF key ring, you may need to export public certificates and send them to other environments: different sysplexes, or different midrange machines. The certificate will need to be imported on these other machines.

Midrange key stores

With midrange, a userid needs a key store with the private and public keys. With multiple machines you need a process to update and distribute key stores. For example:

  • Create a key store in a central site. Do all maintenance to this key store, adding and removing certificates as required.
  • Securely distribute this to all machines that are using AMS.
  • The key store which has been distributed may have the public key for the user. You need to remove it, from the downloaded keystore and add the private key from the local machine.
    • runmqakm -cert -delete -db /home/colinpaice/.mqs/shared.kdb -type kdb -pw passw0rd -label COLIN
    • runmqakm -cert -export -db $private.kdb -pw passw0rd -label COLIN -target COLIN.key.p12 -target-type pkcs12 -target_pw zpassword
    • runmqakm -cert -import -target /home/colinpaice/.mqs/shared.kdb -target_type kdb -pw zpassword -file COLIN.key.p12 -type p12
    • when you import the private key you can use -label COLIN -new_label COLIN2 to give it a new label.
    • You have to make the key the default using runmqakm -cert -setdefault (even though this is deprecated), because without this, the first certificate in the keystore is used..
  • Rename the shared key stores; current to old, new to current.
  • Restart the client.
  • Check it all works, delete the old key store.

If you add a new user, you may need to send the user’s public key to the cental site, to get it distributed to the mid range machines, and send it to the z/OS systems to add to the xxxxAMSM key rings.

In my case I just used one client, and updated the client’s key store directly.

Adding a new user for putting messages to a queue

Signed messages

When the data is signed, the recipient needs:

  • the Certificate Authority of the putter (the signer) – which the recipient may already have
  • or, when the putter has used a self signed certificate, the recipient needs the public part of the putter’s self signed certificate.

When using a CA signed certificate you may not need to export anything if the receiving end already has the CA certificate in the key store or keyring.

To export and import the Certificate authority.

  • To export a CA certificate from z/OS, certificate see here, the certificate owner will typically be CERTAUTH
  • To export a putter’s mid range CA certificate, see here.
  • To import a CA certificate on z/OS, the certificate owner could be a userid, site, or CERTAUTH (check with your security administrator). You might not have authority to use SITE or CERTAUTH.
  • To import a CA certificate on mid range, see here.

When using a self signed certificate (not a good idea for production)

  • export the putter’s public certificate
  • import it into the midrange key stores
  • import it to any other z/OS systems, and add it to the xxxxAMSM keyring.

You can use

  • To export a putter’s z/OS certificate see here.
  • To export a putter’s mid range certificate, see here.
  • To import a certificate on z/OS for a getter, see here. Once the certificate has been imported into the xxxxAMSM keyring it is available to all uses of the queue manager, and does not need to be imported again.
  • To import a certificate on mid range for a getter, see here.

Specify the list of authorised DNs

At the getting end, if the setmqspl -a option is used to specify the list of authorised signers, then the DN from the putter needs to be added to the list of authorised signers (-a…) . You should use dspmqspl -m mq -p q -export to get the current definition for the queue and use setmqspl to add the new -a data to that.

On z/OS, when you have used setmqspl to update the AMS configuration, you need to use the F xxxxAMSM,REFRESH ALL command.

Putting encrypted messages

The putter needs a private key in the keystore or key ring – though this is not used when encrypting messages.

At the putting end, the queue needs to be configured with the list of recipient DNs setmqspl … -r … .

The public certificates for these DNs need to be in the putter’s xxxxAMSM keyring, or in the mid range user’s key store.

  • To export the recipients’s z/OS certificate see here. You could store this in a central file so you can just reuse it rather than having to export it every time.
  • To export the recipients’s mid range certificate, see here. You could store this in a central file so you can just reuse it rather than having to export it every time.
  • To import the certificate on z/OS for the putter, see here. You should only need to do this once per keyring and user.
  • To import the certificate on mid range for the putter, see here. This is where you need a process for a centralised key store containing all of the public keys your organisation needs, and distributing this to all of the users. You may have a cms key store (for C programs) and a JKS key store for Java programs.

Adding a new user for getting messages from a queue

Getting signed messages

If other userids are getting signed messages from this queue there may be no additional configuration on the getting queue manager.

If you are using setmqspl … -a … see Specify the list of authorised DNs above. It may already be set if other userids are configured to get messages from this queue.

When using a CA signer certificate you may not need to export anything if the receiving end already has the CA certificate in the key store or keyring.

If the getting end does not have the CA see To export and import the Certificate authority above.

If the putter is using self signed, see When using a self signed certificate (not a good idea for production) above.

Getting encrypted messages

The getter’s private key is used to decrypt the message. Each putter needs to have access to the public key of the getter.

On the system doing the MQPUT,

  • the recipient’s DN needs to be configured on the putting queue manager using setmqspl with the -r (recipient) option for the queue
  • each user doing the MQPUT will need access to the recipient’s public certificate in its key store or the xxxxAMSM keyring.
Specify the list of recipient DNs

On the putting queue manager you need to add the recipient to the list of the recipients for message. Use the command dspmqspl -m QMA -p AMSQ -export to display the existing configuration for the AMSQ queue, and add the DN of the new user to the list using the setmqspl command.

You need to do this on all squeue managers which can be putting to this queue manager

On z/OS, when you have used setmqspl to update the AMS configuration, you need to use the F xxxxAMSM,REFRESH ALL command.

Get the recipients public certificate in the putters’ keyrings and key stores.
  • To add a z/OS user’s public certificate to the xxxxAMSM’s keyring in the same RACF database, see here.
  • To export the z/OS certificate from a recipient, see here.
  • To export the mid range certificate from a recipient, see here.
  • To import a certificate for putters on z/OS, see here.
  • To import a certificate for putters on mid range, see here.

Other AMS blog posts