I thought that a Client Channel Definition Table (CCDT) in json will be very good for enterprise customers, as they can now do change management, and parameterize key values.
As usual with any new stuff, I tried to get it to work, and tried the silly user errors that people often make, to see how it holds up.
This is a long blog post, so I’ve created some sections
The good stuff – one liners
Having a human (and machine) readable file has many advantages
- You can use change control on it
- You can add your own name:value fields such as “comment”:[”Changed by Colin”,”Date-March 2019”], these are ignored by MQ
- You can use commands like sed to take the .json file, change the contents and create a new json file – for example change the host name and port for production
- If you have the same channel defined in several queue managers you can have one definition and provide an array of hostnames and their ports.
- You can use runmqsc -n to display the mq contents of the .json file.
Ive written a short python script which checks the syntax of the
.json file, see below.
When deploying it for the first time, you should introduce errors
to the .json file, to ensure you are actually using the ccdt.json and
not alternative ways of defining channels.
My journey first steps – happiness
I ran on Ubuntu 18.04.
I used runmqsc to display my existing CLNTCONN channel definitions. I also used the provided sample with the complete list of ccdt channel attributes.
I used gedit to edit the file as this has highlighting support for json (other editors and eclipse also have this support).
Creating the file was a bit slow, instead of a simple list of entries like
“description”: “colins channel”,
"host": "localhost",
"port": 1416,
you have some values
nested within structures, for example
"connection": { "host": "localhost", "port": 1416 },
“description”: “colins channel”,
I had to keep referring to the documentation to look at the structure. I also just used the above sample, “filled in the blanks”, and remove the unused sections.
You have to convert
from terms QMNAME to queueManager (remembering to get the spelling,
and upper case/lower case correct).
I eventually
completed my ccdt.json , and updated my mqclient.ini to include the
definition.
CHANNELS:
ServerConnectionParms=COLIN/TCP/127.0.0.1(1414),127.0.0.1(1416)
MQReconnectTimeout=30
ReconDelay=(1000,200)(2000,200)(4000,1000)
ChannelDefinitionDirectory=.
ChannelDefinitionFile=ccdt.json
I tried running my programs – and they all worked – first time. Hurrah! I went for a beer to celebrate this unusual event.
Second steps, depression
A bit later, I changed the definitions, restarted my programs – and the changes made no difference. I got depressed and went for a cup of tea.
About an hour later,
I discovered that I still had
ServerConnectionParms=COLIN/TCP/127.0.0.1(1414),127.0.0.1(1416)
in my mqclient.ini file! I commented out this
ServerConnectionParms statement.
Then I found I had the environment variable MQCHLLIB set, soI used unset MQCHLLIB
The IBM Knowledge Centre says If the location is specified both in the client configuration file and by using environment variables, the environment variables take priority.
I tried my program again. This time I got messages
AMQ9695E: JSON file format error for ‘./ccdt.json’.
And my program gave
MQCONNX to *GROUP cc 2 rc 2058 MQRC_Q_MGR_NAME_ERROR
This was a major
step forward as it proved I was finally using the ccdt.json file.
This is why I recommend you introduce a few errors in your .json file
the first time you use it.
I searched for the message in the KC for AMQ9695E I got a hit on the page, but searching within the
page, it was not found; but AMQ9695 without the E was found! (As a
psychic programmer you are meant to know to drop the last letter off
the message number).
The explanation from the KC. was Parsing of JSON file <insert_3> failed. The file was expected to contain an attribute named <insert_4> but this was not found or was defined with an unexpected type. The parser returned an error of <insert_5> which may be useful in determining any invalid formatting.
This was not very helpful, what are insert_3, insert_4, insert_5,
and where where are insert_1, insert_2.
I went for another cup of tea. Half way through eating a
chocolate ginger biscuit I had inspiration, there might be
information in the error logs.
I used tail -n100 /var/mqm/errors/*01*|less to look in the MQ errors log. This had a full error description and explanation (Hurrah!).
EXPLANATION:
parsing of JSON file ‘./ccdt.json‘ failed. The file was expected to contain an attribute named ‘channel‘ but this was not found or was defined with an unexpected type. The parser returned an error of ‘Required ‘name’ attribute is missing’ which may be useful in determining any invalid formatting.
ACTION: Check that the contents of the file use the correct JSON schema.
I checked my file – it had “channel” with an array of two elements (definitions)(tick), I had “type”: “clientConnection”, which is valid( tick).
By now I was getting bored with it, so I wrote some python code to
take my ccdt.json and compare it with the IBM sample one. This told
me I had defined “Name”, instead of “name”.
Where the documentation said ‘Required ‘name’ attribute is missing’, it did not mean the generic name:value, it meant the specific field called “name” was missing. So once I understood the problem, the error message made sense!
I fixed that, and a couple of other typos.
Displaying what MQ thinks is in the ccdt.json –
Using runmqsc -n.
I had to use unset MQCHLLIB for this to work (as above). Then I ran runmqsc -n and this gave me.
5724-H72 (C) Copyright IBM Corp. 1994, 2019.
Starting local MQSC for ‘AMQCLCHL.TAB’.
(Ignore the ‘AMQCLCHL.TAB’ which is confusing and not true – it would be nicer if it were to say Starting local MQSC for ‘ccdt.json’)
dis chl(*)
1 : dis chl(*)
AMQ9696E: JSON attribute ‘[1] COLIN: sharingConversations’ has an invalid value or an unexpected type.
AMQ9555E: File format error.
The explanation said Ensure string values use double quotes and
ensure numeric and boolean values are unquoted.
What does the error message mean?
I had two entries in the file for channel with “name”: “COLIN”.
For JSON attribute ‘[1] COLIN: sharingConversations’ this means
- The data with key “name”: “COLIN”, with the attribute sharingConversations has a problem.
- [1] COLIN means the second entry for COLIN, (the counting is 0 based).
I checked my file and I found I had specified “sharingConversations”: “30” with quotes when it should just be 30 (no quotes).
I fixed these, and next time my application worked using these
definitions. It was time for another cup of tea and a second
chocolate ginger biscuit to celebrate.
If you have specified
“timestamps”: { “altered”: “2018-12-04T15:37:22.000Z” }
This will display in the ALTDATE field. ALTDATE(2018-12-04) ALTTIME(15.37.22). If you do not specify this field you will get ALTDATE(1970-01-01) ALTTIME(01.00.00).
Putting your own fields in the ccdt.json file
I added a comment
data into the file. For example
{
“comment”:
{“Createdby”: “Colin Paice”, “Ondate”: “March 2019”},
“channel”: […]
}
the queue manager
ignores this; so you can add your own data into the file.
Checking the syntax of the file.
There are tools
which can check the syntax of your .json file. I used some web
based tools to create a schema from the IBM sample. I then used a
validator to check the syntax of my ccdt.json. Overall, I thought it
was not worth the effort, as I could not run in as a command line,
and the output was not that useful.
I have created some
python which takes a ccdt.json, and makes sure all the fields are
also in the IBM sample.json file, and that the type of the values are
the same. For example with
-
“SharingConversations”:
30 it reports “SharingConversations” not found in …
sharingConversations …, so you can spot the spelling mistakes,
and
-
“sharingConversations”:
“30” it reports types do not match sharingConversations in….
You can install it
using
git clone https://github.com/colinpaicemq/MQccdt.json/
then
cd MQccdt.json/MQccdt
and use it
python3 ccdt.py –ccdt …path_to_ your_ccdt
or
python3 ccdt.py –ccdt …path_to_ your_ccdt -schema …path_to_ your_full_ccdt.json
One definition, multiple connections – No Initial
connection balancing
You can have one
definition with multiple host names.
{ “channel”:
[ {“name”: “COLIN”,
“clientConnection”:
{
“connection”:
[{“host”: “localhost”,”port”: 1414},
{“host”: “localhost”,”port”: 1416}
],
“queueManager”: “GROUP”
},
“type”: “clientConnection”,
With this, my program always connected to the last entry (port
1416) it is was active, if it was not active it chose port 1414. I
did not get connections balanced across the available channels.
Multiple definitions, one connection each – No
Initial connection balancing
I had a channel[{“name”:”COLIN”,… } {“name”:”COLIN”…}]and “connectionManagement”: { “clientWeight”:
90} on both.
It always connected to the second queue manager.
If I changed the second to have {“clientWeight”: 89} it always connected to the first queue manager.
So it looks like some of the parameters for doing the initial
connection balancing are not working.
Tailoring the definitions
I used the shell command
sed ‘s/localhost/remotehost/g’ ccdt.json|sed ‘s/1414/2424/g’
to change localhost to remote host, and port 1414 to 2424
ColinPaice 