Using LDAP with MQ multiplatform and nested groups.

This blog post follows on from Using LDAP with multi platform.

Nested groups can be used to simplify administration

It is good practice to grant authority to access resources using groups, rather than giving access to individual userids. For example if an application uses 10 queues, and a new person joins the team you can either connect the new id to one group, or give the id access to the 10 queues.

Using nested groups takes this further. Imagine the payroll queues are managed by the GRPAYROLL access group, the HR queues managed by the GRHR access group, and the Finance queue managed by the GRFINANCE access group. The MQ system programmers can manage any queue. You could give GRMQADMIN access to the Payroll queues, HR queues, and Finance queues. Or you say the GRMQADMIN is the super set and includes a reference to GRPAYROLL, GRHR, and GRFINANCE. If you define a HR queue, you just give GRHR access to it, and the MQ Administration team get access to it “for free”.

You can take this further, and have a group GRHR_GR1, and GRHR_GR2 which give access to a subset of HR queues. The GRHR group could include both of these.

The model of this is a tree where a group high up in the tree incorporates the groups lower down the tree.

There are two ways of defining groups in LDAP.

1. Define a group record, and list the members of the group. This is called a static group

dn:cn=GR1,OU=groups,o=myorg
member:cn=user1,ou=users,o=myorg
member:cn=user2,ou=users,o=myorg

To retrieve the groups for a user, you issue a query for all groups which have the given member data.

2. Specify the group name as part of the user’s record, this is known as a dynamic group.

dn=cn=user1,ou=users,o=myorg
group=cn=group1,ou=groups,o=myorg
group=cn=group2,ou=groups,o=myorg

For this you retrieve the “group” attributes from the record. Note: The group attribute does not exist in LDAP, you have to use a different one ( I used the st attribute).

Nested group support.

There is an MQ AUTHINFO parameter NESTGRP. The documentation says

  • NESTGRP Group nesting.
    • YES The group list is searched recursively to enumerate all the groups to which a user belongs.

Static groups and nested group support.

My MQ ADUTHINFO had


AUTHORMD(SEARCHGRP) +
BASEDNG(‘ou=groups,o=your Company’) +
CLASSGRP(‘groupOfNames’) +
GRPFIELD(sn) +
FINDGRP(‘member’) +
NESTGRP(yes) +

On my system the userid ibmuser is a “member” of two groups mqstatic and mqstatic2

dn: cn=mqstatic,ou=groups,o=your Company
objectclass: groupOfNames
cn: mqstatic
ou:groups.
member: cn=ibmuser,o=your Company
member: cn=adcdb,o=your Company

and

dn: cn=mqstatic2,ou=groups,o=your Company
objectclass: groupOfNames
cn: mqstatic2
ou:groups.
member: cn=ibmuser,o=your Company

There is another group, which refers to the mqstatic group.

dn: cn=mega,ou=groups,o=your Company
objectclass: groupOfNames
cn: mega
ou:groups.
member: cn=mqstatic,ou=groups,o=your Company
member: cn=gg,ou=groups,o=your Company

As part of the userid to group mapping, MQ issues the query What groups have member: cn=ibmuser,o=your Company. The response is

  • dn: cn=mqstatic2,ou=groups,o=your Company
  • dn: cn=mqstatic,ou=groups,o=your Company

When NESTGRP(YES) is specified MQ then queries all groups that have cn=mqstatic2,ou=groups,o=your Company, and does another query for dn: cn=mqstatic,ou=groups,o=your Company

and gets the response dn: cn=mega,ou=groups,o=your Company.

The cn=ibmuser is associated with the three groups, and those three groups are used for access checking.

For a data model I see it as

dn=cn=GRHR_GR1
isusedby: CN=GRHR

and

dn:cn=GRHR
isusedby: CN=MQADMIN

and

dn: cn=mqadmin
member:cn=user1

The “isusedby” attribute name, is in reality “member”, which I feel is very confusing, as member implies membership, and there is no membership involved. This is an LDAP naming problem – not MQ’s.

This model feels upside down (or inside out) (or both). It takes several cups of tea with biscuits to draw up the groups and the connections definitions you need. Put the kettle on, and try defining the groups for the MQADMIN, GRFINANCE, and GRFINANCEGR1, GRFINANCEGR2 groups; three people in MQADMIN, and two people in finance; then add a definition for GRFINANCEGRN.

Using dynamic groups and nested group support

This feels more the right way up – but it is upside down compared to static groups.

My authinfo had

AUTHORMD(SEARCHUSR) +
FINDGRP(‘st’) +
BASEDNU(‘o=Your Company’) +
CLASSGRP(‘person’) +

With my userid I had

dn: cn=user1, o=Your Company
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: ibm-nativeAuthentication
objectclass: inetOrgPerson
departmentNumber: mqadmin
cn: ibmuser
sn: ibmuser
ou: test
st: cn=group,ou=groups,o=your Company
st: cn=mqadmin,ou=groups,o=your Company
ibm-nativeId: ibmuser

Where st: defined the groups.

For the entry for cn=group, I defined a nested group called cn=deepergroup.

dn: cn=group,ou=groups,o=your Company
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: cpgroup
ou: groups
st : cn=deepergroup,o=your Company
sn : groupgroup

You need an entry for cn=deepergroup… in LDAP

dn: cn=deepergroupou=groups,o=your Company
objectclass: person
cn: mega
sn:mega

# no st attribute, so there is no more nesting

I define the cn=mqadmin group to LDAP with no “sn” entry, because it has no nesting.

MQ issues a query to LDAP asking for the group definitions ( st in my case) for the given user. For each group that gets returned, query that group for any more group (st in my case) definitions, etc.

Together these give three groups for the cn=ibmuser:

  • cn=mqadmin,ou=groups,o=your Company
  • cn=deepergroup,ou=groups,o=your Company
  • cn=group,ou=groups,o=your Company

Dynamic group data model

The data model for dynamic group feels more natural, where the “is in” and “includes” are the attribute you chose, “st” in my case.

  • dn=cn=user1
    • is in: cn=group1…
  • dn=cn=group1…
    • includes: cn=groupdeep

Enabling nestgrp(yes) with dynamic groups may throw errors

If you start with

dn=cn=user1,…
group=cn=group1,..
group=cn=group2,…

With nestgrp(no) this defines the userid is in two groups.

If you enable nestgrp(yes) then when the user id is used, MQ checks there are entries in LDAP for group=cn=group1,.. and group=cn=group2,… If the definitions do not exist, you get the MQ message:

AMQ5532E: Error authorizing entity in LDAP

EXPLANATION:
The LDAP authorization service has failed in the ldap_first_entry call while trying to find user or group ‘NULL’. Returned count is 0. Additional context is ‘cn=group1,… ‘.

To solve the problem I defined the two groups to LDAP.

For group1 I defined

dn:cn=group1,…
group:cn=deepergroup…

and I defined the deepergroup group (to keep mq happy) with no group entries (to indicate end of the nesting).

dn:cn=deepergroup,…

Using LDAP with MQ multi platform.

MQ multiplatform can use LDAP as a userid and group repository, so you can logon to any machine where MQ is running, and use your corporate userid and password.

I’ve logged on to MQ on my Linux machine, and used my z/OS userid and password. It was pretty easy to set up (I had prior experience of using LDAP) but although it didn’t quite behave as I thought it would – I thought this was pretty clever.

Once you have installed LDAP, started it, and created your directory structure, (user, groups) and access permissions, you can start to use it. I’ve documented some of the initial settup here. It covers some of the concepts referred to below.

I used LDAP (Tivoli Directory Server) on z/OS as my LDAP server.

Contents

I’ve also written using LDAP with MQ and nested groups (MQ NESTGRP).

Using LDAP from MQ Multiplatform

The IBM documentation for this is so-so. It gives examples, but the examples didn’t work for me, but they were enough go point me in the right direction.

Start here.

I created a LDAP.MQSC file with

DEFINE AUTHINFO(MYLDAP) +
AUTHTYPE(IDPWLDAP) +
CONNAME(‘10.1.1.2(389)’) +
AUTHORMD(SEARCHGRP) +
BASEDNG(‘o=Your Company’) +
BASEDNU(‘o=Your Company’) +
LDAPUSER(‘cn=adcda, o=Your Company’) +
LDAPPWD(‘adcdapw1’) +
SECCOMM(NO) +
CLASSUSR(‘ibm-nativeAuthentication’) +
CLASSGRP(‘groupOfNames’) +
GRPFIELD(sn) +
SHORTUSR(sn) +
REPLACE

ALTER QMGR CONNAUTH(MYLDAP)

REFRESH SECURITY TYPE (CONNAUTH)
* ALTER QMGR CONNAUTH(SYSTEM.DEFAULT.AUTHINFO.IDPWOS)

Where the key fields for connecting to LDAP are

  • conname – the IP address of the LDAP server.
  • ldapuser and ldappwd – userid and password to access LDAP.
  • seccomm – use TLS/SSL to contact the LDAP server. I used “no” while setting this up.

the key fields for identifying users are

  • basednu – the subtree to be used for userids, for example all users are one level under ou=user,o=myorg.
  • classusr – is the objectclass attribute to identify the userid. The default is inetOrgPerson.
  • shortusr – the dn identifiers are too long. MQ needs IDs with 12 characters or less. This attribute says which attribute to use.

the key fields for identifying which groups an id belongs to

  • authormd – how to search for the authorisation.
  • basedng – the subtree to be used for groups, for example ou=group,o=myorg.
  • classgrp – the objectType which objects must have to be recognised as a group
  • grpfield – the simple name of the group
  • findgrp – what to filter for. For example ‘member’

Being a careful person, I started an interactive runmqsc session in one terminal, and used runmqsc … < LDAP.MQSC in another window. This way if there were problem I could use the interactive session to reset the QMGR CONNAUTH (as in the comment above). I know that the userid that started the queue manager, does not need a password; so if you issued strmqm qma you can use runmqsc qma without a userid and password. It gets harder if your id is not the id the queue manager is running under.

LDAP definition of my logon userid

The userid I wanted to use with MQ was defined

dn: cn=ibmuser, o=Your Company
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: ibm-nativeAuthentication
objectclass: inetOrgPerson
cn: ibmuser
sn: ibmuser
ou: test
st: cn=group,ou=groups,o=your Company
st: cn=mqadmin,ou=groups,o=your Company
ibm-nativeId: ibmuser

The authinfo data has

BASEDNU(‘o=Your Company‘) +
CLASSUSR(‘ibm-nativeAuthentication‘) +
SHORTUSR(sn) +

When I try to logon with userid ibmuser, MQ issues an LDAP query for the record with

  • sn=ibmuser
  • with an object class = ‘ibm-nativeAuthentication (which provides the RACF support for userid and password)
  • in the subtree o=Your Company

Check the LDAP configuration when things go wrong

It took me a few hours to determine why I could logon with one id, but not another id. Some LDAP entries worked, and some did not. It turned out to be an Access Control List (ACL) set up problem, where the LDAPUSER userid was not authorised to see some of the records. With the above AUTHINFO object, the query that MQ uses to check authorisation is like

ldapsearch -h 127.0.0.1 -D “cn=adcda, o=Your Company” -w ? -b “o=Your Company” “&(objectClass=ibm-nativeAuthentication)(sn=zadcdc)”

Where the parameters match up with the authinfo object above, and zadcdc is the userid trying to logon.

If you get no data back, get an authorised person (cn=ibmuser…) to issue the command for the user problem userid zadcdc:

ldapsearch -h 127.0.0.1 -D “cn=ibmuser, o=Your Company” -w ? -b “o=Your Company” “&(objectClass=*)(sn=zadcdc)”aclentry aclsource

The aclentry will give you the userids or groups who are authorised to use the entry, and the access they have.

The aclsource tells you which node in the tree the ACL was inherited from (for example aclsource=o=Your Company says it came from the root node). I had set up an ACL for my zadcdc which did not include my LDAPUSER.

Setting up MQ connect authorities

You can issue the command

setmqaut -m qml -t qmgr -p ibmuser +connect

to give ibmuser connect authority.

You can use LDAP groups for example

setmqaut -m qml -t qmgr -g “cn=mqadmin,ou=groups,o=your Company” +connect.

How do you set up a group in LDAP?

This is where it gets interesting. You can define a static group with its list of members, or create a dynamic group which is more flexible and “modern” (where modern is within the last 30 years).

Using a static group (with a list of members defined in it)

You can define a static group in LDAP using

dn: cn=mqstatic,ou=groups,o=your Company
objectclass: groupOfNames
ou:groups.
member: cn=ibmuser,o=your Company
member: cn=adcdb,o=your Company

It has two members.

When the userid authenticates, the queue manager asks LDAP for the groups that the userid is in; using the AUTHINFO definitions CLASSGRP(‘groupOfNames’) + GRPFIELD(…) FINDGRP(‘…’) a query is done for the groups which have the userid id. For example with

BASEDNG(‘ou=groups,o=your Company’) +
CLASSGRP(‘groupOfNames’) +
FINDGRP(‘member’) +

and cn=ibmuser o=your Company. The query is

(&(objectClass=groupOfNames)(member=cn=ibmuser, o=Your Company) in subtree (ou=groups,o=your Company)

The ldap search asking to return the member attribute

ldapsearch … -b “ou=groups,o=your Company” “(&(objectClass=groupOfNames) (member=cn=ibmuser, o=Your Company)) ” cn

gave two group names –

cn=mqstatic2,ou=groups,o=your Company
cn=mqstatic,ou=groups,o=your Company

This gives the information as to which groups a user is in. MQ then saves the userid and group information, and does not need to go to LDAP the next time the userid needs access checking.

Knowing which groups a userid is in, MQ can then decide on the access by comparing with the setmqaut definitions.

Using a dynamic group – or using a group attributer in the user record.

Instead of a group having a list of members, you can add information to the user’s record.

For example

dn: cn=adcda, o=Your Company
changetype: add
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: ibm-nativeAuthentication
sn: adcda
group: cn=group,o=your Company
cn: mq
ibm-nativeId: adcda

Unfortunately there is no “group” attribute defined in the LDAP schema, so I had to find another attribute to use. I used st for state. I used st: cn=group,o=your Company instead of group: cn=group,o=your company.

In my MQ AUTHINFO definition I had

BASEDNU(‘o=Your Company’) +
CLASSUSR(‘ibm-nativeAuthentication’) +
SHORTUSR(sn) +
AUTHORMD(SEARCHUSR) +
FINDGRP(st,‘) +
BASEDNG(‘ou=zzzzz,o=your Company’) +

MQ did an LDAP search using this information

ldapsearch… -b “o=Your Company” “&(objectClass=(ibm-nativeAuthentication)
(sn=ibmuser))” st

Which is just a display of the userid information – and return the fields with the attribute you specified(st). It returned

cn=ibmuser, o=Your Company
st=cn=group,ou=groups,o=your Company
st=cn=mqadmin,ou=groups,o=your Company

This tells MQ to use the groups

cn=group,ou=groups,o=your Company and cn=mqadmin,ou=groups,o=your Company.

You can use the setmqaut command to give the group access

setmqaut -m qml -t qmgr -g “cn=mqadmin,ou=groups,o=your Company” +connect

Once this was done, the cn=ibmuser could connect to MQ using groups.

Can I use LDAP to hold my setauth information?

Only the group and userid information are held in LDAP, all the other information is held in the queue manager. You cannot use LDAP to have your setmqaut configuration in LDAP, and shared by multiple queue managers. You still have to use setmqaut to set up each queue manager access.

Giving userids access to MQ objects

You can use the setmqaut command or the set authrec runmqsc command to give principals or groups access to resources.

For example

setmqaut -m qml -n CP0000 -t queue -g “cn=mqstatic,ou=groups,o=your Company” +inq

I’ve changed the definitions in LDAP – when will they get picked up?

Changes get picked up when

  • the queue manager is restarted
  • when the resfresh security, refresh security type(authserv) or the refresh security type(connauth) command is issued.

What they don’t tell you about using a REST interface.

After I stumbled on a change to my Python program which gave 10 times the throughput to a Web Server, I realised that I knew only a little about using REST. It is the difference between the knowledge to get a Proof Of Concept working, and the knowledge to run properly in production; it is the difference between one request a minute to 100 requests a second.

This blog post compares REST and traditional client server and suggests ways of using REST in production. The same arguments also apply to long running classical client server applications.

A REST request is a stateless, self contained request which you send to the back-end server, and get one response back. It is also known as a one shot request. Traditional client server applications can send several requests to the back-end as part of a unit of work.

In the table below I compare an extreme REST transaction, and an extreme traditional Client Server

AttributeRESTClient Server
ConnectionCreate a new connection for every request.Connect once, stay connected all day, reuse the session, disconnect at end of day.
Workload BalancingThe request can select from any available server, and so on average, requests will be spread across all connections. If a new server is added, then it will get used.The application connects to a server and stays connected. If the session ends and restarts, it may select a different server.
If a new server is added, it may not be used.
AuthenticationEach request needs authentication. If the userid is invalidated, the request will fail. Note that servers cache userid information, so it may take minutes before the request is
re-authenticated.
Authentication is done as part of the connection. If the userid is invalidated during the day, the application will carry on working until it restarts.
IdentificationBoth userid+password, and client certificate can be used to give the userid.Both userid+password, and client certificate can be used to give the userid. If you want to change which identity is used, you should disconnect and reconnect.
CostIt is very expensive to create a new connection. It is even more expensive when using TLS, because of the generation of the secret key. As a result it is very very expensive to use REST requests.The expensive create connection is done once, at start of day. Successive request do not have this overhead, so are much cheaper
Renew TLS session keyBecause there is only one transfer per connection you do not need to renew the encryption key.Using the same session key for a whole day is weak, as it makes it easier to break it. Renewing the session key after an amount of data has been processed, or after a time period is good practice.
RequestSome requests are suitable for packaging in one request, for example where just one server is involved.This can support more complex requests, for example DB2 on system A, and MQ on system B.
Number of connectionsThe connection is active only when it is used.The connection is active even though it has not been used for a long time. This can waste resources, and prevent other connections from being made to the server.
StatisticsYou get an SMF record for every request. Creating an SMF record costs CPU.You get one SMF record for collection of work, so reducing the overall costs. The worst case is one SMF record for the whole day.

What are good practices for using REST (and Client Server) in production?

Do not have a new connection for every request. Create a session which can be reused for perhaps 50 requests or ten minutes, depending on workload. This has the advantages :

  • You reduce the costs of creating the new connection for every request, by reusing the session.
  • You get workload balancing. With the connection ending and being recreated periodically, you will get the connections spread across all available connections. You should randomise the time a connection is active for, so you do not get a lot of time-out activity occurring at the same time
  • You get the re-authentication regularly.
  • The TLS key is renewed periodically.
  • You avoid the long running connections doing nothing.
  • For a REST request you may get fewer SMF records, for a Client-Server you get more SMF requests, and so more granular data.

How can I do this?

With Java you can open a connection, and have the client control how long it is open for.

With Python and the requests package, you can use

s = requests.Session()
res = s.get(geturl,headers=my_header,verify=v,cookies=jar,cert=cpcert)

res = s.get(geturl,headers=my_header,verify=v,cookies=jar,cert=cpcert)
etc

With Curl you can reuse the session.

Do I need to worry if my throughput is low?

No, If you are likely to have only one request to a server, and so cannot benefit from having multiple requests per connection you might just as well stay with a “one shot” and not use any of the tuning suggestions.

Thoughts on planning for security on midrange.

As I was working on some other blog posts relating to security profiles on midrange MQ, I thought about the best way to set up MQ security. Below are some things you need to consider, outside of the well documented topics.

Give people the authority they need to do their job – but no more

One of the rules of security is to give people the authority they need to do their job – but no more. People may need access to one set of resources (eg queues) but do not need access to other sets of resources (other queues). This means you need more than one security profile – one for each. Having one big security group covering both sets of resources is a no-no.

Use groups to give permissions rather than to userids

It is better to give groups permissions, than to give individual’s userid permissions. To add a new member to the team you add their userid to the group(s) – rather than to all the queues. To remove someone’s access – you delete them from the group(s). These changes are easy to do and easy to reverse. If you did it at the userid level, there could be many records you would needs to process.

If you want to give a userid permission to do something, then other people are likely to want the same permission. Give permission to one group, and all userids in that group will get the access.

Removing authority

If you want to remove someone’s authority to do something

  • “Just remove” them them from the mqm group.
  • Use dmpmqaut .. -p userid -e to see which groups have the permission. Either
    • “Just remove” the permission from the group. This means other people in the group will also lose permission
    • or “just remove” the userid from the group, which may mean the userid may lose access to other facilities.

“Just remove” could be a lot of work, as you want to remove access to one queue, but retain their access to other queues. You may need to set up other groups to cover this.

Review your setup

You may want to review your groups, permissions, and userids within groups for your MQ estate and develop a scheme where you have a queue, and groups to define the permissions to the queue. If you have queues PAY_IN…. and PAY_OUT… you will need two AUTHREC records, one for each.

Generic profiles

If you have multiple generic profiles which apply to a queue, only the most specific profile will be used. Profile AA.BB.*.DD would be used in preference to AA.*.CC.DD

If a generic profile exists with the same name, but different groups, then all applicable instances of the most specific profile will be used. Profile AA.BB.*.DD group test, and profile AA.BB.*.DD group test2, would both be used if the userid is in both groups.

Separation of responsibilities

You may not want people who can administer queues, from processing messages on queues. Userids in the mqm group, or in the default group of the userid that created the queue, have all permissions for the queue. You may want to have a tool for deploying administrative changes, and use MQAdmin groups to control who can administer the object.

It took me a while to partially understand setmqaut and dspmqaut – and your security may be a mess.

On Linux, I was looking into MQ context security , where you can prevent a user from putting a fake information into the MQMD, and to my surprise it kept on working! It took me a while to work out what was going on.

Some of the things I learned include

  • If a userid is in the mqm group, it overrides any setmqaut statements. You may want to have a group called MQADMIN, and have your administrators in that group rather than the mqm group; so they can create or alter queues , but not put messages to sensitive queues.
  • Your overall access permission depends on the settings for the policy for the userid and any groups it is in. If one of your groups has permission, you have permission. If no group has permission, your userid gets no permission.
  • If you add or remove userids from groups, you have to use the runmqsc command refresh security to pick up the changes.
  • If you are using a userid in the mqm group to test your application – you may be testing with a different set of permissions to your target user.
  • It is better to define security profiles using groups rather than userids. If someone joins the team, you just put them in the group. If security is done using userids, you have to replicate all of the definitions of an existing user. Similarly if someone leaves the team.
  • The difference between group and principal (userid) is fuzzy. You define authority on a userid – but it appears as a group. Thanks to Joshm who pointed out that on Unix the qm.ini SecurityPolicy=user|group|UserExternal|default controls whether security is done with userids or groups. I need to explore this, but it explains some of my confusion
  • Some of the commands, for example dmpmqaut do not work as documented, but you can work round it.
  • setmqauth and dspmqauth parameters are read asis – with display authrec mixed case fields need to be surrounded with single quotes.
  • If you remove permission, then check, as you may get permission from more than one group.

Controlling the ability to change the context of a message.

This facility allows an application to specify the MQMD fields PutApplType, PutApplName, PutDate and ApplOriginData, rather than have them filled in by the queue manager. An MCA channel would do this when putting a message, to keep the original information. See origin context.

The key security parmeter is “setall” which allows these fields to be set.

Being a good tester, I displayed the permissions before changing them.

dspmqaut -m QMA -n CP0000 -t queue -p colinpaice

This says display the attributes for Queue CP0000 for user colinpaice.

I also defined authorities for another userid

setmqaut -m QMA -n CP0000 -t queue -p testuser +allmqi

I could not see how to use dspmqaut to display the authorities for all users of a queue, however you can use the runqmqsc DIS AUTHREC command, dmpmqaut and dmpmqcfg -m … -x authrec to dump out the permissions for users of an object.

The documentation is a little unclear – it talks about entity without actually saying what an entity was. I assumed the entity was the queue name, it actually means userid or group – but this is not strictly true.

runmqsc dis authrec displayed

dis authrec profile(CP0000)
     3 : dis authrec profile(CP0000)

AMQ8864I: Display authority record details.
   PROFILE(CP0000)                         ENTITY(mqm)
   ENTTYPE(GROUP)                          OBJTYPE(QUEUE) 
   AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,
   PASSALL,PASSID,SET,SETALL,SETID)

AMQ8864I: Display authority record details.
   PROFILE(CP0000)                         ENTITY(testuser)
   ENTTYPE(GROUP)                          OBJTYPE(QUEUE)
   AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

AMQ8864I: Display authority record details.
   PROFILE(**)                             ENTITY(test)
   ENTTYPE(GROUP)                          OBJTYPE(TOPIC)
   AUTHLIST(DSP)
...                        

This needs a bit of explaining.

Display information about a group’s access to a profile

AMQ8864I: Display authority record details.
   PROFILE(CP0000)                         ENTITY(mqm)
   ENTTYPE(GROUP)                          OBJTYPE(QUEUE)
   AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID) 

For Object type of Queue, there is a profile for object CP0000. The profile applies to group mqm. It has lots of access. That was pretty self explanatory.

Display information about a userid’s access to a profile

AMQ8864I: Display authority record details.
   PROFILE(CP0000)                         ENTITY(testuser)
   ENTTYPE(GROUP)                          OBJTYPE(QUEUE)
   AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

This is just like the first section, except I had defined it for principal (userid) testuser – not a group.

Using DIS AUTHREC gave

dis authrec profile(CP0000) objtype(queue) group('testuser')
AMQ8864I: Display authority record details.
PROFILE(CP0000) ENTITY(testuser)
ENTTYPE(GROUP) OBJTYPE(QUEUE)
AUTHLIST(BROWSE,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

dis authrec profile(CP0000) objtype(queue) principal('testuser')
4 : dis authrec profile(CP0000) objtype(queue) principal('testuser')
AMQ8459I: Not found.

However if I display information about the userid, it displays

uid=1002(testuser) gid=1004(testuser) groups=1004(testuser),1005(test)

so there is a group for the userid as well as the userid itself.

All in all – I found this a little confusing.

Other profiles.

AMQ8864I: Display authority record details.
   PROFILE(**)                             ENTITY(test)
   ENTTYPE(GROUP)                          OBJTYPE(TOPIC)
   AUTHLIST(DSP)

There is a generic profile ** for object type topic. If there was a topic CP0000 then userid/group test would have DSP access.

Displaying what you want

You can limit the object type displayed

dis authrec profile(CP0000) objtype(queue)

this just displayed the queue records for profile CP0000 (and so did not display the topic record above), so you can be selective.

You can use

dis authrec profile(CP0000) objtype(queue) group(‘testuser’)

or principal(‘userid’) to display records for just a group or userid.

However

dis authrec profile(‘CP*’) objtype(Queue)

gives

AMQ8459I: Not found.

Because there is no profile with name CP*. It does not display all profiles beginning with CP.

Using setmqaut

I wanted to test out context security, and did not want my application to be able to set certain fields in the MQMD.

If the userid had permissions, put and setall, my application was able to set the application name.

I turned off the permission with

setmqaut -m QMA -n CP0000 -p testuser -t queue -setall

and got rc=2035 MQRC_NOT_AUTHORIZED as expected.

Note: The userid also needs the “global level” switch

setmqaut -m QMA –t qmgr -p testuser +setall

enabled at the queue manager level. The application needs both +setall permissions to be able to set these context fields in the MQMD at put time.

This is not the end of the story…

With my normal administrator userid colinpaice, I used

setmqaut -m QMA -n CP0000 -t queue -p colinpaice -setall

to remove the setall permission; and this had no effect! I was still able to set the certain fields in the MQMD.

Hidden away in the documentation is a line

If a principal is a member of more than one user group, the principal effectively has the combined authorities of all those user groups

After I issued the setmqauth command, the dspmqauth command

dspmqaut -m QMA -n CP0000 -t queue -p colinpaice

gave

get
put
setall

which shows the userid still has setall access.

From the dis authrec command, it gave

2 : dis authrec profile(CP0000) objtype(QUEUE) 
AMQ8864I: Display authority record details.
PROFILE(CP0000) ENTITY(mqm)
ENTTYPE(GROUP) OBJTYPE(QUEUE)
AUTHLIST(BROWSE,CHG,CLR,DLT,DSP,GET,INQ,PUT,PASSALL,PASSID,SET,SETALL,SETID)

so the group mqm has SETALL access to the queue.

It gets more confusing. I used the command

setmqaut -m QMA -n CP0000 -t queue -p colinpaice -setall

to turn off setall permissions

dspmquth with -p colinpaice showed setall

dspmquth with -g colinpaice did not have setall!

This has me stumped for a few hours!

Being in the mqm group trumps the setmqaut definitions.

If a userid is in the mqm group, then it looks like there are no other checks. An implications of this is if your development team are in the mqm group, and they checking “error paths”, the MQ call will work, when it would fail for non group mqm users.

I used

sudo gpasswd –d colinpaice mqm

to remove my userid from the group. I had to use the runmqsc command refresh security command to get the queue manager to pick up the change.

Once I had done this then the setmqaut commands with +setall and -setall worked as expected. I used

sudo gpasswd –add colinpaice mqm

and the runmqsc refresh security command, for the setmqaut commands to be ignored.

The command

dspmqaut -m QMA -n CP0000 -t queue -p colinpaice

was accurate, in that when the userid was in the group, the command shows what access the userid has.

When the userid was not in the group, I remove access

setmqaut -m QMA -n CP0000 -t queue -p colinpaice -remove

the display command gave

Entity colinpaice has the following authorizations for object CP0000:
crt

When the userid was in group mqm the dspmqaut command gave

dspmqaut -m QMA -n CP0000 -t queue -p colinpaice
Entity colinpaice has the following authorizations for object CP0000:
get
browse
put
inq
set
crt
dlt
chg
dsp
passid
passall
setid
setall
clr

What certificates do I need for AMS with a client going to z/OS

The MQ documentation usually shows a local application doing a put and get, to show “how simple” it is.
Most people have a more complex environment where they have a mixture of z/OS and midrange, and clients into both.

This blog posts shows how I got clients to work with AMS on z/OS. My starting point was a working z/OS queue manager with working Java client able to put and get messages from a queue. I then enabled AMS on the z/OS queue manager.

Getting the z/OS end working

The userids on z/OS used a CA signed certificate (which is more typical in production than a self signed certificate).

Set up the xxxxAMSM keystore

My AMSM adress space has a started task userid of START1.

The xxxxAMSM address space needs a keystore, and the CA for the certificates on z/OS

#RACDCERT ID(START1) DELRING(drq.ams.keyring)
RACDCERT ID(START1) ADDRING(drq.ams.keyring)
RACDCERT ID(START1) CONNECT(CERTAUTH LABEL(‘TEMP-CA’) –
RING(drq.ams.keyring))
RACDCERT LISTRING(drq.ams.keyring) ID(START1)

Set up the keyring and certificate for the tso userid

Userid COLIN needs a keyring and an RSA certificate to identify COLIN. It is signed by “TEMP-CA”.

#RACDCERT ID(COLIN) DELRING(drq.ams.keyring)

RACDCERT ID(COLIN) ADDRING(drq.ams.keyring)

#RACDCERT ID(COLIN) DELETE(LABEL(‘AMS’))

#It defaults to certificate type RSA
RACDCERT ID(COLIN) GENCERT –
SUBJECTSDN(CN(‘COLIN’) O(‘SSS’)) –
SIZE(2048) –
SIGNWITH (CERTAUTH LABEL(‘TEMP-CA’)) –
WITHLABEL(‘AMS’)

RACDCERT id(COLIN) ALTER(LABEL(‘AMS’))TRUST

# add the certificate to user’s keyring
RACDCERT ID(COLIN) CONNECT(RING(drq.ams.keyring ) –
ID(COLIN) –
DEFAULT –
LABEL(‘AMS’))

# Connect the user’s public certificate to the xxxxAMSM keyring
#Usage(SITE) only exposes the public certificate, the private certificate
# is not visible in the keyring
RACDCERT ID(START1) CONNECT(ID(COLIN) LABEL(‘AMS’) –
RING(drq.ams.keyring) USAGE(SITE))

RACDCERT LISTRING(drq.ams.keyring) ID(COLIN)

RACDCERT ID(COLIN) LIST

# export it so we can send public key to Linux machine.

RACDCERT ID(COLIN) EXPORT(LABEL(‘AMS’))-
DSN(‘COLIN.AMS.PEM’) –
FORMAT(CERTB64) –
PASSWORD(‘password’)

# give the userid access to the keyring

permit IRR.DIGTCERT.LISTRING class(facility) access(read) id(COLIN)

setropts raclist(facility) refresh

FTP the COLIN.AMS.PEM certificate to Linux as ASCII.

Make the queue AMS protected

You need to use setmqspl to define the level of protection

// EXPORT SYMLIST=* 
// SET QM=CSQ9
//CSQ40CFG EXEC PGM=CSQ0UTIL, 
//         PARM='ENVAR("_CEE_ENVFILE_S=DD:ENVARS") /' 
//STEPLIB  DD DSN=COLIN.MQ921.SCSQANLE,DISP=SHR 
//         DD DSN=COLIN.MQ921.SCSQAUTH,DISP=SHR 
//ENVARS   DD DSN=COLIN.MQ921.SCSQPROC(CSQ40ENV),DISP=SHR 
//EXPORT   DD SYSOUT=* 
//SYSPRINT DD SYSOUT=* 
//SYSIN    DD *,,SYMBOLS=(JCLONLY)  
 setmqspl -m &QM.
   -p AMSQ 
   -remove 
 setmqspl -m &QM.
   -p AMSQ 
   -s SHA256 
   -e AES256 
   -a CN=COLIN,O=SSS 
   -a "CN=testuser, O=aaaa, C=GB" 
   -r "CN=testuser, O=aaaa, C=GB" 
   -r CN=COLIN,O=SSS 
dspmqspl -m &QM. 
dspmqspl -m &QM. -export 

Tell the xxxxAMSM address space to pick up the changes

You can use refresh all | refresh policy to get the the xxxxAMSM address space to use the changed policy information.

F xxxxAMSM,REFRESH ALL

or

F xxxxAMSM,REFRESH POLICY

Set up Linux

Create the keystore.conf

By default the configuration file used by AMS is in $HOME/.mqs/keystore.conf . I used different files, so I could test different scenarios with just one userid. I could set up a files called Alice.conf and Bob.conf so my ID can have different roles.

The location of the keystore.conf is available using

  • export MQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf for C program and
  • -DMQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf for java programs

Create a file for the configuration (/home/colinpaice/mqamsclient/ks.conf)

JKS.keystore = /home/colinpaice/ssl/ssl2/trust.jks
JKS.certificate =testuser

JKS.encrypted = no
JKS.keystore_pass = zpassword
JKS.key_pass = zpassword

# in V9 JKS.provider is ignored
# JKS.provider = IBMJCE

The options are documented here.

The key_pass is to access the private key within the keystore, see -keypass here. Because you are storing the clear text password in the file, it is recommended to use runamscred to encrypt the passwords.

Create the Linux keystore and certificate

I have scripts which generate my certificates, so I modified an existing one. I created a certificate with CN=testuser,O=aaaa,C=GB.

name=”testuser”
rm $name.pem $name.key.pem $key.csr $key.p12

# create an RSA private key
openssl genpkey -out $name.key.pem -algorithm RSA -pkeyopt rsa_keygen_bits:2048

# Create a certificate requests – so we can get it signed
openssl req -config eccert.config -passin password -sha384 -new -key $name.key.pem -out $name.csr -outform PEM -subj “/C=GB/O=aaaa/CN=”$name -passin file:password.file -passout file:password.file

#use this CA
ca=”carsa1024″

# sign it
openssl ca -config openssl-ca-user.cnf -policy signing_policy $ext -md sha256 -cert $ca.pem -keyfile $ca.key.pem -out $name.pem -in $name.csr $enddate -extensions clientServer

#Create the .p12 file so we can import it into the .jks
openssl pkcs12 -export -inkey $name.key.pem -in $name.pem -out $name.p12 -CAfile $ca.pem -chain -name $name -passout file:password.file -passin file:password.file

# remove the old one and import it
/opt/mqm/bin/runmqckm -cert -delete -db trust.jks -type jks -label $name -pw zpassword

/opt/mqm/bin/runmqckm -cert -import -target trust.jks -target_type jks -file $name.p12 -label $name -pw password -target_pw zpassword

#Add the z/OS user’s public key, so we can encrypt with the certificate and public key
/opt/mqm/bin/runmqckm -cert -add -db trust.jks -type jks -file zCOLIN.PEM -label zcolin -pw zpassword

# extract the Linux certificate and public key, along with its CA.
runmqckm -cert -extract -db trust.jks -pw zpassword -label $name -target $name.PEM -format ascii -type jks

exit

FTP the Linux public key to z/OS and add it to the keyring

The public key for testuser was extracted to testuser.PEM ( $name.PEM above). FTP this to z/OS (in ASCII). This became file COLIN.TESTUSER.PEM on z/OS.

To simplify the administration, you could setup an ID to “own” the certificates for each enterprise you work with. For example certificates from YOURORG are attached to ID(YOURORG).

I’ve used id ADCDA.

#Try to delete it if needed
#RACDCERT ID(ADCDA) DELETE (LABEL(‘LINUXTESTUSER’))


RACDCERT ADD(‘COLIN.TESTUSER.PEM’) –
ID(ADCDA) WITHLABEL(‘LINUXTESTUSER’) TRUST

# Connect it to the xxxxAMSM’s keyring
RACDCERT ID(START1) CONNECT(RING(drq.ams.keyring ) –
ID(ADCDA) LABEL(‘LINUXTESTUSER’))

Tell xxxxAMAM to pick up the updated keystore

You can use refresh all | refresh keystore to refresh the xxxxAMSM keyring information

f CSQ9AMSM,REFRESH ALL

or

f CSQ9AMSM,REFRESH keyring

Run the java client

. /opt/mqm/java/bin/setjmsenv64

#Java does not used export MQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf

ksc=”-DMQS_KEYSTORE_CONF=/home/colinpaice/mqamsclient/ks.conf”
lib=”-Djava.library.path=/opt/mqm/java/lib64″

#java $lib $ksc JmsBrowser -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

#java $lib $ksc JmsConsumer -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

java $lib $ksc JmsProducer -m CSQ9 -d AMSQ -h 10.1.1.2 -p 1414 -l SYSTEM.DEF.SVRCONN

Any error messages are written to file mqjms.log.0 .

Other AMS blog posts

Upgrading to MQ 9.2.1 on Ubuntu using apt was easy, but got messy.

Installing MQ 9.2.1 over an earlier release of MQ was almost easy!

I started here and downloaded the developer edition into ~/MQ921.

    • I had to cd ~/MQ921/MQSERVER to run the license.sh.
    • I could not edit a file in /etc/apt/sources.list.d because I was not authorised, using sudo gedit failed with:
      • Invalid MIT-MAGIC-COOKIE-1 keyUnable to init server: Could not connect: Connection refused.   Gtk-WARNING **: 10:34:55.390: cannot open display: :0
      • I could have used xhost +local:  to fix this problem.
    • I used the filename  ibmmq921-install.list instead of the suggested  ibmmq-install.list because I already had an ibmmq.list file in the directory.
    • I created  ibmmq921-install.list in my home directory and then used sudo mv ibmmq921-install.list /etc/apt/sources.list.d/ .
    • Initially I forgot to do sudo apt update – so the install did not find the updated file.
    • When I ran sudo apt update it displayed 10 packages can be upgraded. Run ‘apt list –upgradable’ to see them.
    • I ran sudo apt update and it displayed 
ibmmq-client/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-gskit/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-java/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-jre/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-man/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-runtime/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-samples/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-sdk/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-server/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]
ibmmq-web/unknown 9.2.1.0 amd64 [upgradable from: 9.1.5.0]

    • I ran sudo apt upgrade it ran for a short while and ended successfully.
    • The command dspmqver showed Version 9.2.1.0 – so success.
    • Starting the queue manager strmqm QMA gave IBM MQ queue manager ‘QMA’ started using V9.2.1.0

It got messy

After I had successfully installed MQ, the next time I ran sudo apt update it gave me lots of messages like 

Get:1 file:/home/colinpaice/MQ921/MQServer ./ InRelease
Ign:1 file:/home/colinpaice/MQ921/MQServer ./ InRelease
Get:2 file:/home/colinpaice/MQ921/MQServer ./ Release
Ign:2 file:/home/colinpaice/MQ921/MQServer ./ Release

By adding in the file, means apt checks it every time.  I could not see how any updates were going to appear in the files, so I removed the file from /etc/apt/sources.list.d/

I also deleted the files I downloaded (890 MB) and the unzipped directory (927MB).  (I needed to use sudo rm -r MQ921 because the files have write-protection.

Not very tidy.