Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Classify the services with WLM to give them the right priority

You can configure an API or service, for example https://10.1.3.10:9443/stockmanager/stock/items/999999, so it gets an appropriate service class in WLM, for example high priority.

You can use RMF to report on service classes to see the response time profile, and if the service class is meeting its performance goals.

See here for a good article.

This page gives the example

<wlmClassification>
  <httpClassification transactionClass="CLASS001" 
      host="127.0.0.1" 
      port="9080" 
      method="GET"
      resource="/testResource" />
</wlmClassification>

You can classify the traffic depending on the IP address and port of the server, as well as the resource name.

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Copy the PDS from the install system to where they will run.

These files can be sent using FTP or XMIT.

Copy the product ZFS to where it will be used.

You can use DFDSS to dump the ZFS and move it to the SYSPLEX where it will be used.

1. Mount the file system read/write for example /usr/lpp/IBM/zosconnect/v3r0beta
2. Perform Setting up the product extensions directory.  You need to do this on each LPAR, as it set up files in /var/zosconnect
3. Mount the file system READ
4. Change the BPXPRMxx parmlib member so the file sytsem is mounted READ it at IPL
5. You may want to create an alias /usr/zosc to the product executables.
   • ln -s /usr/lpp/IBM/zosconnect/v3r0beta/bin /usr/zosc

Create the angel started task if needed.

If you will be using an existing Angel you will need to set up RACF permissions.

Follow the instructions the z/OS Connect  documentation

• If you need a named Angel, specify the value in the NAME=value.  You can specify NAME when you start the task.
• You do not need to classify this in WLM as it uses little CPU once it has been started.

Create the file system for the server

1. See Creating the z/OS Connect EE shared directory. You will need to know the path for server’s home directory. The path /var/zosconnect can only be used on the LPAR.   The files cannot be shared.
2. If you need to allocate a file system.
   1. Allocate a ZFS. If the initial allocation of the ZFS is too small, then the ZFS will grow on demand, or using the zfsadm grow command to expand it.
   2. Mount the file system over the directory.
   3. Change the BPXPRMxx parmlib member so the file system is mounted RW at IPL.

Create the Z/OS Connect started task

See Creating a z/OS Connect Enterprise Edition server.

Using /usr/zosc alias defined above

export  WLP_USER_DIR=/var/zosconnect
/usr/zosc/zosconnect create default --template=zosconnect:default

If you need to start again, use the following to remove the server.

rm -r  /var/zosconnect/servers/default

1. Copy BAQSTRT to the SYS1.PROCLIB concatenation, to the name you are going to use, BAQSTRT, BAQSTRT1 etc
2. When you edit the member, use caps off to use lower case text in the member.
3.  If you will be using MQ, add the MQ libraries to STEPLIB

   //STEPLIB   DD DSN=CSQ913.SCSQANLE,DISP=SHR 
   //          DD DSN=CSQ913.SCSQAUTH,DISP=SHR

4. You may find it easier to remove the _CEE_RUNOPTS= and add

   //CEEOPTS DD DISP=SHR,DSN=...(CEEOPTS)

   to the JCL. This makes it easier to add CEE options, and specify storage heap and pool sizes.

5. In my CEEOPTS I use RPTSTG(ON), to report the storage pool usage. This provides useful information which may be useful when investigating performance problems.
6. Despite what the Liberty documentation says you do need

   • • JVM_OPTIONS=<Optional JVM parameters>

   • for example

     • JVM_OPTIONS=-Xoptionsfile=/var/zosconnect/servers/default/jvm.options

7. You may want a JVM specific to a server, as you may want to set some service specific options such as TLS trace.
8. You may want to specify a default server name in the JCL parms. You can specify a different one when z/OS Connect is started.
9. Specify WLP_USER_DIR for your server’s directory. You could make this as a parameter to the started task JCL if you want to provide isolation for the servers, and give each server its own file system.

Create the JVM options file

Go to the server’s directory.

cd /var/zosconnect/servers/default

use oedit jvm.options to create an EBCDIC file.

add

-Dcom.ibm.ws.zos.core.angelRequired=true 
#-Dcom.ibm.ws.zos.core.angelName=MYZANGL

The -Dcom.ibm.ws.zos.core.angelRequired=true option stops the server from starting if the angel process is not running. You get message:

CWWKB0116I: This server is not registered with an angel process even though it is configured to require registration with an angel process. This server is attempting to stop.

The option -Dcom.ibm.ws.zos.core.angelName=MYZANGL allows you to specify a non default angel process name. The # in front of the statement makes it into a comment.

add the following, if required, with your time zone.

-Duser.timezone=Europe/London

Override TCPIP

If you are using a non standard TCPIP, edit server.env and add

_BPXK_SETIBMOPT_TRANSPORT=TCPIP2

Tailor the server.xml file

It was easy to get my configuration options mixed up, and hard to compare two server configurations. I found it much easier to manage my configuration by putting the components into their own xml file. This also makes having shared components much easier to implement- as you just include the name of the shared file.

The configuration files have to be in ASCII, and you have to take care ensure the files are in the correct code page.

There are at least three ways of creating the .xml files (I use the third option through ISPF 3.17)

1. using omvs commands to copy and edit existing ASCII files
2. use omvs commands to create ASCII files
3. use ispf option 3.17

1 Using omvs commands to copy an existing ascii file,  then edit it it

For example

cp server.xml httpEndpoint.xml

edit the httpEndpoint.xml to keep the <server> ,httpEndpoint information, </server> and delete the rest.  The file should look like

<server description="new server"> 
    <httpEndpoint id="defaultHttpEndpoint" 
                  host="*" 
                  httpPort="9080" 
                  httpsPort="9443" /> 
</server>

then edit the server.xml and replace

<httpEndpoint id="defaultHttpEndpoint" 
                  host="*" 
                  httpPort="9080" 
                  httpsPort="9443" />

with

<include location="${server.config.dir}/httpEndpoint.xml"/> 

Note: if you use the edit command source ? It should say ASCII already set.

2 Use OMVS to created and edit files

By default, if you edit a file in OMVS it becomes an EBCDIC file.

If you create a file without copying it, you need to use

touch httpEndpoint.xml 
chtag -t -c ISO8859-1 httpEndpoint.xml     
oedit httpEndpoint.xml

Make the same changes as above.

3 Use ispf option 3.17

If you use ISPF 3.17 (z/OS UNIX Directory List) you have more flexibility with editing files.

Edit server.xml

use the line commands mm…mm and the primary command create to create a new member, give the new name when prompted, for example httpEndpoint.xml

Replace the lines you copied out with

	<include location="${server.config.dir}/httpEndpoint.xml"/>

and edit the file you just created to put <server> … </server> around the content.

Add more logging to the JCL’s STDOUT.

The default logging to the jobs STDOUT is only audit records. I found it much easier to include other levels of information so I did not have to keep looking in the message.log file. Note: the STDOUT content does not have time stamps so you do not know when the output was produced- so it is easier to use – but has less information!

I created logging.xml with the following content.

<server> 
 <logging 
 consoleLogLevel="INFO" 
 /> 

 <!-- this is a comment where I store trace information
   traceFormat="BASIC" traceSpecification="*=info:
       com.ibm.ws.security.*=fine:zos.native.03=fine" 
  end comment   -->  
</server>

I added

<include location="${server.config.dir}/logging.xml"/> 

to the server.xml file.

The consolLogLevel=”INFO” writes additional information to the STDOUT file, than the default AUDIT value which just outputs AUDIT records.

Define the http address and ports

Edit the httpEndpoint definitions

    <httpEndpoint id="defaultHttpEndpoint" 
                  accessLoggingRef="ALR" 
                  host="*" 
                  httpPort="9081" 
                  httpsPort="9443" /> 
    <httpAccessLogging 
       enabled="true" 
       id="ALR" 
       logFormat="%{c}a! %h %u %{t}W %q %r %s %b" 
   /> 

Check

• host matches the TCPIP value to use. With my VIPA I specify the VIPA address host=”10.1.3.10”
• httpPort – if you already have a Liberty instance running port 9081 will be in use by another server
• httpsPort – if you already have a Liberty instance running port 9443 will be in use by another server

You will need the port values when you use the server.

The httpAccessLogging statement pointed to by accessLoggingRef, will create a log file with every attempt to use the server.

Example output

10.1.1.1! 10.1.1.1 ADCDC {12/Sep/2020:16:07:17 +0000} - 
GET /zosConnect/apis/stockmanager HTTP/1.1 200 296

Where 10.1.1.1 is the source IP address and ADCDC is the userid used.

By default this file is http_access.log in the …/logs directory.

See here for the format of the logFormat statement.

After tailoring

I did more packaging the content into include .xml files.

After my “repackaging” my server.xml file looked like

<?xml version="1.0" encoding="UTF-8"?> 
<server description="new server"> 
    <include location="${server.config.dir}/applicationMonitor"/> 
    <include location="${server.config.dir}/logging.xml"/> 
    <include location="${server.config.dir}/httpEndpoint.xml"/> 
    <include location="${server.config.dir}/cors.xml"/> 
    <include location="${server.config.dir}/config.xml"/> 
    <include location="${server.config.dir}/zosconect.xml"/>                                                                              
</server>

You can use ispf option 3.17 to list the files in the servers’s directory.

Once you have configured the server you can use the admin centre to manage the files.  This will require

<remoteFileAccess>  
    <writeDir>${server.config.dir}</writedir>
</remoteFileAccess>

Finally ! You can start the server.

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Starting the server

Using the definitions above I started the server, using the default configuration directory, and used jobname=default.

s baqstart,parms=’default’,jobname=default

On my z/OS system, running on zPDT on Linux, it takes about 5 minutes to start.

During start up, check that the parameter have been accepted – with the correct case.

If you get configuration errors, fix them and use the

f ...zcon,refresh

command to get the server to pickup the changes. Note. Not all changes are picked up by the refresh command, and you have to stop and restart the server to pick up changes.

There is output in the STDOUT and possibly in STDERR, also in the /var/zosconnect/default/logs/messages.log

STDOUT will have messages like

[AUDIT] CWWKG0028A: Processing included configuration resource: 
/var/zosconnect/servers/default/httpEndpoint.xml

which tells you which configuration files you are using.

If you get any syntax errors you should fix them. The command f …,zcon,refresh  may cause it to reread the definitions.

The second +  start of server start  after  IPL may be faster than the first if Java has cached some of the executables.

Check for messages

Authorized service group … is available.

Which tells you that the Angel service is enabled.

Web application available (default_host): http://10.1.3.10:9080/

check this is what you expect. If the port is not valid check the httpEndoint.xml. You do not get an entry for the TLS port, because TLS is not enabled.

If you use a web browser to logon to the URL http:10.1.3.10:9081 (note:not https) you will just get a blank screen. If you look in logs/http_access.log you will see a code 302. In this case, 302 means you were trying to use an http session to a session that requires https.

If you use other URLs you may get a message saying you need to use https.

If you use z/OS explorer and use the http:…9081 above you will get “302, Found”. This means you were trying to use an http session to a session that requires https.

You have successfully created a z/OS connect server, and used it – even though you could not do much.

To stop the server use the z/OS P … command.

Most of the facilities in z/OS Connect require TLS, so you cannot do much until you have configured that.

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Enable logon security

To enable security checking you need  to use the zosconnect_zosConnectManager tag.

See here for information, and here for the tag  syntax.

<zosconnect_zosConnectManager 
     requireAuth="true"  
     requireSecure="true" 
     setUTF8ResponseEncoding="true"/>

add to the zosconnect.xml or server.xml file.

You cannot do much with this enabled as you need TLS

Add in the SAF repository to use z/OS security services.

Within the server directory create a file called saf.xml. (I always used ISPF 3.17, and use cc…cc create.. to create a new file.

<server> 
                                                                                
  <featureManager> 
     <feature>transportSecurity-1.0</feature>   
     <feature>zosSecurity-1.0</feature> 
     <feature>appSecurity-2.0</feature> 
  </featureManager> 
  <authentication id="Basic" cacheEnabled="false" /> 
  <safAuthorization id="saf-authorization"  racRouteLog="ASIS" />                                                                                 

  <safRegistry id="saf" realm="myrealm"/> 
  <safCredentials profilePrefix="ZZZZDFLT" 
     suppressAuthFailureMessages="false" 
     unauthenticatedUser="WSGUEST"/> 
</server>

Where you need to specify the profile prefix (for example ZZZBFLT), and define the security profile.

I do not think the realm is used for SAF processing.

If you are planning on using a new profilePrefix  (YYYYDFLT) you will need to create several profiles.

RDEFINE APPL YYYYDFLT  UACC(NONE) 
PERMIT YYYYDFLT  CLASS(APPL) ACCESS(READ) ID(group1,group2)
SETROPTS raclist(APPL) refresh

RDEFINE server BBG.SECPFX.YYYYDFLT uacc(NONE)
permit  BBG.SECPFX.YYYYDFLT class(server) access(read) id(start1)
setropts raclist(server) refresh

rdefine ejbrole YYYYDFLT.zos.connect.access.roles.zosConnectAccess 
   uacc(none)
permit YYYYDFLT.zos.connect.access.roles.zosConnectAccess  
   class(ejbrole) 
   id(Adcdc) access(READ)
permit YYYYDFLT.zos.connect.access.roles.zosConnectAccess  
    class(ejbrole) 
    id(group1,group2) access(READ)
setropts raclist(ejbrole) refresh

These profiles are only accessed once someone tries to use the server – not at startup, so try logging on using the https port.

Every one using the server needs read access to YYYYDFLT CLASS(APPL).

As an interim step towards getting your server up and secure, enable userid and password authentication.

<webAppSecurity overrideHttpAuthMethod="BASIC"/> 
<webAppSecurity allowFailOverToBasicAuth="true"/> 

overrideHttpAuthMethod=”BASIC” says used userid and password – and not client certificates.

allowFailOverToBasicAuth=”true” says if the client certificate logon fails – fail over to basic, userid and password.

Once you have configured the server, you can remove the overrideHttpAuthMethod tag, so client certificates are used.  Once these are working remove the allowFailOverToBasicAuth, to remove the userid and password option.  With these both removed, people will have to use client certificates to authenticate.

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Adding APIs and Services

For services you have to include a feature. For example with mq you need

 <feature>zosconnect:mqService-1.0</feature>

These are listed here.   You need the CICS service to be able to use CICS applications etc.

You also need to copy the files application (*.aar) files into the apis directory, and the service(*.sar) files into the services directory.

There are several ways of doing this.

You can do this with the uss cp command. The command below copies an MQ one – you can use others

Find what applications and services are available

find  /usr/lpp/IBM/zosconnect/v3r0beta/runtime/* -name *.aar
find  /usr/lpp/IBM/zosconnect/v3r0beta/runtime/* -name *.sar   

copy the files to the servers directory

cp /usr/lpp/IBM/zosconnect/v3r0beta/runtime/templates/servers
  /sampleMqStockManager/resources/zosconnect/apis/*
  /var/zosc*/ser*/default/re*/z*/ap*                                                                                   
cp /usr/lpp/IBM/zosconnect/v3r0beta/runtime/templates/servers
  /sampleMqStockManager/resources/zosconnect/ser*/* 
  /var/zosc*/ser*/default/re*/z*/se*                                                                                        

Use a REST request or use the zOS explorer.

Once you have done this either restart the server or use the operator command

f ...zcon,refresh

and the application and services should be available

You should get in the message.log on the STDOUT (if you have configured it to print information messages – see earlier posts).

BAQR7000I: z/OS Connect EE API archive file stockmanager installed successfully.
BAQR7043I: z/OS Connect EE service archive stockQuery installed successfully.

Using a web browser, or curl with the URL https://10.1.3.10:9443/zosConnect/services gave me

{"zosConnectServices":
  [
    {"ServiceName":"stockQuery",
     "ServiceDescription":"A stock query service based on IBM MQ.",
     "ServiceProvider":"IBM MQ for z/OS",
     "ServiceURL": "https://10.1.3.10:9443/zosConnect/services/stockQuery"
      }
  ]
}

URL https://10.1.3.10:9443/zosConnect/apis gave me

{"apis":    
  [
    {"name":"stockmanager","version":"1.0.0","description":"",
     "adminUrl":"https://10.1.3.10:9443/zosConnect/apis/stockmanager"
     }
   ]
}

You can invoke the service with https://10.1.3.10:9443/zosConnect/services/stockQuery and get details of the stockQuery service.

Once it all works…

I played with the curl interface to deploy a service.

• I copied the stockQuery.sar file down to linux in binary
• I could use jar -tvf stockQuery.sar to display the contents
• I used curl –insecure -v –header Content-Type:application/zip -i –cacert cacert.pem –cert adcdd.pem:password –key adcdd.key.pem -X post –data-binary @/home/zPDT/stock2.sar https://10.1.3.10:9443/zosConnect/services
  • Note:  If I used –data-binary “@/home/zPDT/stock2.sar” in quotes I got
    • HTTP/1.1 415 Unsupported Media Type
    • “errorMessage” : “BAQR0418W: An unsupported media type of application/x-www-form-urlencoded was specified.”}
  • because the string was sent up – not the file.  I used the –verbose option to see the number of bytes sent.
  • I also got these messages if the file was not found.  It sends the string instead!

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protect the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Protect the service and APIs

z/OS connect provides interceptors to allow the product function to be extended. These are like exits in other program products.

z/OS Connect provides (at least)  2 interceptors

1. For authorisation checks, to see if a userid is allowed to perform an operation.
2. Creating SMF records.

You can also write your own interceptors, for example for data validation, or for collecting statistics.

You can configure APIs and services to have a list of interceptors. One service can have authorisation and SMF records, another service can just have authorisation

You create a list like

<!-- Interceptor list configuration -->
<!-- this refers to the configuration elements following -->
<zosconnect_zosConnectInterceptors 
   id="interceptorList1" 
   interceptorRef="auditInterceptor,zosConnectAuthorizationInterceptor"
/>

<!-- Audit interceptor configuration -->
<zosconnect_auditInterceptor 
   id="auditInterceptor" 
   sequence="1"    
   apiProviderSmfVersion="2"
/>
<!-- Authorisation checking --> 
<zosconnect_authorizationInterceptor 
    id="zosConnectAuthorizationInterceptor"
/> 

To protect the server, and control the global roles, have you need to use the following where you provide lists of group names such as SYS1.

 <zosconnect_zosConnectManager 
     globalInterceptorsRef="interceptorList1" 
     globalAdminGroup="SYS1,SYSADMIN" 
     globalInvokeGroup="SYS1" 
     globalOperationsGroup="SYS1" 
     globalReaderGroup="SYS1" 
       /> 
<!-- "interceptorList1" above points to …  -->
 <zosconnect_zosConnectInterceptors 
     id="interceptorList1" 
     interceptorRef="IR1,..."/>

<!--  zosConnectAuthorizationInterceptor is defined    -->
 <zosconnect_authorizationInterceptor 
     id="IR1"/>

This shows the global security definitions. The globalInterceptorsRef=”interceptorList1″ points to the <zosconnect_zosConnectInterceptors .. which in turn points to the <zosconnect_authorizationInterceptor . There is a program or interceptor zosConnectAuthorizationInterceptor which does the actual checking of userid and roles.

With this set of definitions when I try to query the service using an unauthorised userid, I got

{"errorMessage":"BAQR0435W: The zosConnectAuthorization interceptor 
  encountered an error while processing a request. ",
"errorDetails":"BAQR0409W: User ADCDC is not authorized to 
  perform the request."}

I changed the definitions to globalReaderGroup=”TEST” , refreshed the configuration, and the request worked.

You can make API security more specific

 <zosconnect_zosConnectAPIs> 
   <zosConnectAPI name="stockmanager" 
     adminGroup="SYS1" 
     invokeGroup="TEST" 
     operationsGroup="TEST" 
    readerGroup="SYS1" 
    /> 
 </zosconnect_zosConnectAPIs>

and make the service security more specific.

<zosconnect_services> 
   <service name="stockQuery" 
     serviceDescription="stockQueryServiceDescriptionColin" 
     id="stockQueryService" 
     adminGroup="SYS1,TEST2" 
     invokeGroup="TES2" 
     operationsGroup="SYS1" 
    readerGroup="SYS1,TEST2" 
    /> 
</zosconnect_services> 

If you use the swagger to try it – and get the json data with

response body no content
response code 0
response header { “error”: no response from server}

This is what Swagger UI displays when a request fails due to a security issue such as an untrusted self-signed cert, invalid cert, or bad username:password.

Introduction

I’ll cover the instructions to install z/OS Connect, but the instructions are similar for other products. The steps are to create the minimum server configuration and gradually add more function to it.

The steps below guide you through

1. Overview
2. planning to help you decide what you need to create, and what options you have to choose
3. initial customisation and creating a server,  creating defaults and creating function specific configuration files,  for example a file for SAF
4. starting the server
5. enable logon security and add SAF definitions
6. add keystores for TLS, and client authentication
7. adding an API and service application
8. protecting the API and service applications
9. collecting monitoring data including SMF
10. use the MQ sample
11. using WLM to classify a service

With each step there are instructions on how to check the work has been successful.

Monitoring data

You can collect SMF data and/or Http audit data to get reports on the performance and usage of your system.

There are some queries – you can issue via a REST query – but you do not get much data back.

http_access.log

If you have configured <httpEndpoint   ..  accessLoggingRef=…>  you can collect audit information for traffic through that end point.  If you have more than one httpEndpoint, for example with a different port, you can collect different information, or log it to a different file.

The information you can log (see here for a full description) includes

• the client IP address
• the userid
• date time
• the service requested
• the response code
• bytes sent and received
• the response time ( in seconds, milliseconds, or microseconds)

You can include delimiters (for example quotes around a string, or !.. !)  in the output to simplify post processing.

If you have high throughout, this solution may not scale, and SMF may be a better solution.

Collecting SMF records

You can collect SMF 120 records from the Liberty base, and SMF 123 records from z/OS connect.

To collect SMF 120 records you need to add

<featureManager> 
    <feature>zosRequestLogging-1.0</feature> 
</featureManager>

to your configuration.

SMF 123 records are produced by another interceptor (exit). You need to define it, and add it to the list of global, API or service list of interceptors.

Configure the auditInterceptor

<zosconnect_auditInterceptor id="auditInterceptor" 
   sequence="1" 
   apiProviderSmfVersion="2"/>

and add it to the list of the intereptors

<zosconnect_zosConnectInterceptors 
    id="interceptorList1" 
    interceptorRef="zosConnectAuthorizationInterceptor,auditInterceptor"
/>

For both record types, the server started task needs access to the BPX.SMF class.

PERMIT BPX.SMF CLASS(FACILITY) ACCESS(READ) ID(USERID)
setropts raclist(facility) refresh

If the server does not have this permission it will get an FFDC with

Stack Dump = java.io.IOException: Failed to write SMF record, __smf_record errno/errno2 return code 139

Processing SMF 120. You can download SMF Browser for WebSphere Application Server for z/Osfrom

This is a java “formatter” here which provides just a dump of the records, and so is not very usable.

I wrote a formatter for this to summarise key information ( and ignore irrelevant stuff).   I’ll put this up on github when Ive got it documented.

Some of the interesting data is

• Request start and stop time for example 2020/09/26 16:35:42.977709, from which you can calculate request duration
• CPU for the request
• The userid
• The URI /zosConnect/services/stockQuery
• TCPIP Origin and port 10.1.1.1 (33546) into the server port (9443)
• Sysplex, LPAR, Server name, Server job number, level,

I took the data and accumulated it, so I could see which requests used all of the CPU, and report it by hour, and userid.

Processing SMF 123.

z/OS connect provides a sample C program, and JCL to compile it.   See here.

The SMF 123 records are written when the z/OS Connect server shuts down, or when the SMF buffer is full, so there is a risk that data from today, is not produced until tomorrow because there was no activity overnight.

I typically got about 20 services/APIs per SMF record.

Combing the records

I could not see how to correlate the SMF 123 and the SMF 120 records.   This would be useful to get the CPU used by each API or service.

Rest request

This page describes how to get REST statistics.  For example

https://10.1.3.10:9443/zosConnect/operations/getStatistics

This returned

{"zosConnectStatistics":
  [
   {"stockQuery":
     {
       "ServiceProvider":"IBM MQ for z\/OS",
       "InvokeRequestCount":21,
       "TimeOfRegistrationWithZosConnect":
       "2020-10-01 14:52:26:049 BST",
       "ServiceStatistics":{}
    }
   }
  ]
}

With nothing in the ServiceStatistics{}.

You can ask for a specific service https://10.1.3.10:9443/zosConnect/operations/getStatistics?service=stockQuery.  You get the same data back as above.

I could not find how to get information on APIs.

You can get real time statistics data see here.

I had

<zosconnect_zosConnectManager
globalInterceptorsRef=”interceptorList1″
globalAdminGroup=”TEST”
globalInvokeGroup=”SYS1″
globalOperationsGroup=”TEST”
globalReaderGroup=”TEST”
/>
<zosconnect_authorizationInterceptor id=”zosConnectAuthorizationInterceptor”/>
<zosconnect_auditInterceptor id=”zoscauditInterceptor” sequence=”1″ apiProviderSmfVersion=”2″/>
<auditInterceptor id=”auditInterceptor” sequence=”1″/>
<zosconnect_zosConnectInterceptors
id=“interceptorList1”
interceptorRef=”zosConnectAuthorizationInterceptor,auditInterceptor,zoscauditInterceptor “/>