Creating a spread sheet from Python to show time spent

I’ve been using xlsxwriter from Python to create data graphs of data, and it is great.

I had problems with adding more data to a graph, and found some aspects of this was not documented. This blog post is to help other people who are trying to do something similar.

I wanted to produce a graph like

The pink colour would normally be transparent. I have coloured it to make the explanation easier.

This shows the OMS team worked from 0900 to 1600 on Monday, and SDC worked from 10am to 12am and for a few minutes on the same day around 1700. I wanted the data to be colour coded, so OMS was brown and SDC was green.

Create the workbook, and the basic chart

spreadSheet = "SPName"
workbook = xlsxwriter.Workbook(spreadSheet+"xlsx")
workbook.set_calc_mode("auto")
summary = workbook.add_worksheet("data")
hhmm = workbook.add_format({'num_format': 'hh:mm'})
# now the basic chart
chart = self.workbook.add_chart({'type': 'bar','subtype':'stacked'})
chart.set_title ({'name': "end week:time spent"})
chart.set_y_axis({'reverse': True})
chart.set_legend({'none': True})
chart.set_x_axis({
'time_axis': True,
'num_format': 'hh:mm',
'min': 0,
'max': 1.0,
'major_unit': 1/12,
'minor_tick_mark': 'inside'
})

chart.set_size({'width': 1070, 'height': 300})
summary.insert_chart('A1', chart)

Data layout

It took a while to understand how the data should be laid out in the table

ABCDEFG
1OStart1ODur 1SStart 1SDur1SInt2SDur2
2OMS Tue9.07.0
3OMS Wed17.02.0
4SDC Wed10.02.07.00.1

Where the data

  • OSTart1 is the time based on hours for OMS
  • ODur1 is the duration for OMS, so on Wed, the time was from 1700 to 1900, and interval of 2.0 hours
  • SStart1 is the start time of the SDC Wed item
  • SDur1 is the duration of the work. The work was from 1000 to 1200
  • SInt2 is the interval from the end of the work to the start of the next work. It is not the start time. It is 1000 + interval of 2 hours + interval of 7 hours, or 1900
  • SDur2 is the duration from the start of the work. It ends at 1000+ 2 hours + 7 hours + 0.1 hours

Define the data to the chart

To get the data displayed properly I used add_series to define the data.

Categories (The labels OMS Tue, OMS Wed, SDC Wed): You have to specify the same categories for all of your data. For me, range A2:A5. Using add_series for the OMS data, and add_series for the SDC data did not display the SDC data labels. This was the key to my problems.

You define the data as columns. The first column is the time from midnight. I have coloured it pink to show you. Normally this would be fill = [{‘none’ : true } ] You use

fill = [{'color': "pink"}] # fill = [{'none': True}]
chart.add_series({
'name': "Series1,
'categories': ["Hours",1,0,4,0],
'values': ["Hours",1,1,4,1],
'fill': fill
})

This specifies categories row 1, column 0 to row 4, column 0, and the column of data row 1, column 1, to row 4, column 1. (Column 0 is column A etc.)

For the second column – the brown, you use

fill = [{'color': "brown"}]
chart.add_series({
'name': "Series2,
'categories': ["Hours",1,0,4,0],
'values': ["Hours",1,2,4,2],
'fill': fill
})

The categories stays the same, the superset of names.

The “values” specifies the column of data row 1, column 2, to row 4, column 2.

Because the data for SDC is missing – this is not displayed.

For the SDC data I used 4 add_series request. The first one

  • name:Series3
  • ‘categories’: [“Hours”,1,0,4,0], the same as for OMS
  • values: row 1,column 3 to row 4 column 3

I then repeated this for columns (and Series) 4,5,6

This gave me the output I wanted.

I used Python lists and used loops to generate the data, so overall the code was fairly compact. The overall result was

What is my ISPF screen size and other attributes?

I was trying to use a very large screen in ISPF, but I could only get 132 * 43.
You can display information about your screen using

  • ISPF option 0 Settings
  • top line, tab to environ
  • option 1 Environ settings…
  • tab down to Terminal Status (TERMSTAT)
  • option 2 Query terminal information

If gives information like

ISPF TERMINAL CHARACTERISTICS
14 BIT ADDRESSING = ON
16 BIT ADDRESSING = OFF
EXTENDED COLOR = ON
EXTENDED HIGHLITING = ON
DBCS = OFF
PRIMARY SCREEN SIZE - PARTITIONED MODE
SCREEN SIZE = 1,920 - x'00000780'
SCREEN DEPTH = 24 - x'00000018'
SCREEN WIDTH = 80 - x'00000050'

ALTERNATE SCREEN SIZE - PARTITIONED MODE
SCREEN SIZE = 3,564 - x'00000DEC'
SCREEN DEPTH = 27 - x'0000001B'
SCREEN WIDTH = 132 - x'00000084'

PARTITION SCREEN SIZE - PARTITIONED MODE
SCREEN SIZE = 0 - x'00000000'
SCREEN DEPTH = 0 - x'00000000'
SCREEN WIDTH = 0 - x'00000000'
PRIMARY SCREEN SIZE - NON-PARTITIONED MODE
SCREEN SIZE = 1,920 - x'00000780'
SCREEN DEPTH = 24 - x'00000018'
SCREEN WIDTH = 80 - x'00000050'
ALTERNATE SCREEN SIZE - NON-PARTITIONED MODE
SCREEN SIZE = 3,564 - x'00000DEC'
SCREEN DEPTH = 27 - x'0000001B'
SCREEN WIDTH = 132 - x'00000084'
ISPF TERMINAL BUFFER INFORMATION
TERMINAL BUFFER ADDR = x'000326A8'
TERMINAL BUFFER SIZE = 4,989 - x'0000137D'
PHYSICAL SCREEN SIZE = 3,564 - x'00000DEC'
PARTITION ARRAY ADDR = x'0001DAA0'
GTTERM INFORMATION
RETURN CODE = x'00000000'
PRIMARY ROW = 24 - x'00000018'
PRIMARY COL = 80 - x'00000050'
ALTERNATE ROW = 27 - x'0000001B'
ALTERNATE COL = 132 - x'00000084'
ATTRIBUTE BYTE = x'000000C9'
EBCDIC TERMINAL
EDS TERMINAL
GTSIZE INFORMATION
RETURN CODE = x'00000000'
ROWS = 27 - x'0000001B'
COLUMNS = 132 - x'00000084'

VTAM ACCESS METHOD

All the information says the screen is 27 rows and 132 columns.

Customising ISPF appearance

Screen size: using bigger screens.

  • Use x3270 -model 3279-5-E   to get 132*27 screen size.  It works for the console and ISPF terminal.
  • I could not get the -oversize 160*45 working. When I edited the .3270.pro file and added x3270.oversize: 160×45 it worked.
  • Logon and use ISPF =0  to set defaults.  Scroll down
    • To have command line at the top / Command line at bottom  remove the /
    •  Scroll down.  Screen format 3 1. Data 2. Std 3. Max 4. Part
    • Terminal Type 4 1. 3277 2. 3277A 3. 3278 4. 3278A though I think this field is ignored.

Other ISPF personalisation

  • Options
    _ Command line at bottom  remove the / to have the command line at the top
    / Tab to point-and-shoot fields so you can tap to column headers, press enter and sort by the column
  • Member list options
    / Scroll member list
    / Allow empty member list
    / Allow empty member list (nomatch)
    / Empty member list for edit only
  • pfshow off remove the PFKEYS at the bottom
  • ISPF  keys set PF12 to retrieve not cancel
  • ISPF set scroll to CSR not PAGE. You will need to do this in all applications
  • ISPF 3.4 use reflists list of the last 30 data sets used, or your own list
  • Setting the ISPF main panel.
    • Copy ADCD.Z24A.ISPPLIB(ISR@PRIM)  to USER.Z24A.ISPPLIB(MYMAIN).
    • Add extra content and comparisons at the bottom for example ISMF,’PGM(DGTFMD01) NEWAPPL(DGT)’ .
    • The following are already defined
      •  RACF,’PANEL(ICHP00)’
      • ISMF,’PGM(DGTFMD01) NEWAPPL(DGT)’
      • SMPE,’PGM(GIMSTART) PARM(&ZCMD) NOCHECK’
      • WLM,’CMD(%IWMARIN0)’
    • When you use the TSO Logon panel specify Command ===> ispf panel(MYMAIN)  

RACF setup, and changing the RACF dataset

I used ADCD supplied version of z/OS, and needed to change which data set I used, so it could be used on z/OS 3.1 or earlier (but not both at the same time).

What options are active?

You can use the operator command

#rvary

where # is the RACF subsystem character (defined in INITPARM(‘#’) in the IEFSSN definition for RACF).

Before z/OS 2.3

You had to define RACF data set parameters using a load module.

  • The databases are defined in a module ICHRDSNT.
  • If you use SDSF and issue the LOAD ICHRDSNT command, it will display the data.
  • The source is in ADCD.LIB.JCL(ICHRDSNT).
  • If you want to change it, you need to generate the module, and REIPL

Parmlib definitions

In z/OS 2.3 and later you can specify parameters in the parmlib concatenation.

In your IEASYS member you can specify RACF=(xx,yy,zz) and specify up to three parameters. These parameters take precedence over the ICHRDSNT table. I specified RACF=(00)

My equivalent definition in USER.Z25D.PARMLIB(IRRPRM00) is

DATABASE_OPTIONS 
DATASETNAMETABLE
ENTRY
PRIMARYDSN(SYS1.RACFDS)
BACKUPDSN(SYS1.RACFDS.BACKUP)

You can have one primary data set and up to one backup data set. If you want to partition your database by key, you can have more primary and backup data sets entries, and you need to specify the key range to data set mapping.

The syntax is defined here.

The datasets have to be cataloged. I have used a data set catataloged in a user catalog, they do not need to be SYS1.xxxx.

If you get it wrong and the member is invalid, at IPL RACF prompts

 *IRRY115I RACF IS NOT USING PARMLIB FOR INITIALIZATION.                     
*01 ICH502A SPECIFY NAME FOR PRIMARY RACF DATA SET SEQUENCE 001 OR 'NONE'
IRA600I SRM CHANNEL DATA NOW AVAILABLE FOR ALL SRM FUNCTIONS
IRA860I HIPERDISPATCH MODE IS NOW ACTIVE
R 01,SYS1.RACFDS
*02 ICH502A SPECIFY NAME FOR BACKUP RACF DATA SET SEQUENCE 001 OR 'NONE'
R 2,SYS1.RACFDB.BACKUP

Changing parmlib

If you change the IEASYS RACF=, or change an IRRPMRxx member, you have to REIPL to pick up the changes. Stopping and restarting RACF does not pick up the changes, because the RACF started task handles operator commands and other admin stuff, it does not process the data sets. See here.

Copying the RACF database

You can copy a RACF database and use that. For example

//IBMCRACO  JOB 1,MSGCLASS=H 
//STEP EXEC PGM=IRRUT200
//SYSRACF DD DSN=SYS1.RACFDS,DISP=SHR
//SYSUT1 DD UNIT=SYSDA,SPACE=(CYL,(20)),
// DCB=(LRECL=4096,RECFM=F),DSN=COLIN.RACFDS,DISP=(MOD,CATLG)
//SYSUT2 DD SYSOUT=A
//SYSPRINT DD SYSOUT=A
//SYSIN DD *
INDEX
MAP
END
/*

IRRUT200 is called a RACF database verification utility program (IRRUT200), and to make an exact copy of a RACF data set. It takes a lock on the RACF database for the duration of the copy to ensure the data is consistent.

You should check the size of the source dataset, and make the copy the same size.

Can I easily make an ISPF edit line command macro?

The key word in the title is easily. If you search for “Working with an edit line command table”, the documentation gives you instructions on how to create a table, then add your definition, create your macro, and then use it. When you were expecting to do this invisibly for the end user, it is a lot of work.

A command line rexx program can process line commands. I created a command line program so when I use it, and use S or SS..SS line commands, the macro can extract the tagged lines, and so do things with the tagged data.

The command line Rexx

/* REXX */ 
address isredit
'MACRO (a) NOPROCESS '
'process range S '
IF RC = 0
THEN DO
"(first) = LINENUM .ZFRANGE"
"(last) = LINENUM .ZLRANGE"
data = ""
/* build up a long string from the tagged lines */
do i = first to last
"(l) = LINE (i)"
data = data || strip(l) || " "
end
l = data
address ispexec "edit dataset('COLIN.$$TEMP$$') macro(l2) parm(l)"
END

Where the lines of interest are

  • ‘MACRO (a) NOPROCESS ‘
    • Usually when a command line macro is processed, ISPF processes any outstanding line commands (such as D) before processing any commands in the rexx exec. The NOPROCESS says – hold off doing this until instructed.
    • a is the data passed in
  • ‘process range S ‘
    • This locates any line commands beginning with S, for a single line, or SS…SS for multiple lins.
  • if rc = 0 then do
  • “(first) = LINENUM .ZFRANGE”
    • get the line number of the First in the RANGE
  • “(last) = LINENUM .ZLRANGE”
    • get the line number of the Last in the Range.
  • data = “”
  • do i = first to last /* process the data build up the string */
    • “(l) = LINE (i)”
    • data = data || strip(l) || ” “
  • end
  • ldata = data
  • address ispexec “edit dataset(‘COLIN.$$TEMP$$’) macro(l2) parm(ldata)”
    • Invoke edit on a file – invoke macro l2, and pass the data in the rexx variable ldata
  • END

If the “process range” gives a non zero return code, set an error message

else do 
zedsmsg = 'You need to specify line prefix s|ss ss'
zedlmsg = 'You need to specify line prefix s|ss ss long message'
address ispexec 'SETMSG MSG(ISRZ001)'
end

This sets a Short message (zedSmsg) which is display at the top right of the display, and a long message (zedLmsg) if the user presses PF1 on the short message.

Macro L2

This macro is processed when the edit dataset(..) macro(l2) command is issued

/* REXX */ 
address isredit
'MACRO (a) '

say "Passed data" a
...

Linux: Why are my cursor keys not working?

I created a new userid, and when I use the cursor keys in a command window it does not work as expected and I get commands like ]]?a

Solution

sudo chsh -s /bin/bash myid

If you use the command adduser you can specify the shell at create time.

Once you have done this you need to logoff and logon again (or just start a new window).

You should also have a .profile script for when you logon. My .profile has

# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login
# exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.

# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

# if running bash
if [ -n "$BASH_VERSION" ]; then
# include .bashrc if it exists
if [ -f "$HOME/.bashrc" ]; then
. "$HOME/.bashrc"
fi
fi

# set PATH so it includes user's private bin directories
PATH="$HOME/bin:$HOME/.local/bin:$PATH"


JVM_ARGS="-Dcom.sun.management.jmxremote.port=9081 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "
# export MQCHLLIB=/var/mqm/qmgrs/QMA/@ipcc
export MQSSLKEYR=/var/mqm/qmgrs/QMB/ssl/key
export SWT_GTK3=0
export SWT_WEBKIT2=0
export CLASSPATH="${JAVA_HOME}/lib/tools.jar:${CLASSPATH}"
export LESS="-I -R"

This . profile invokes .bashrc (if it exists)

My file has

####### added for zPDT #######200725095805
export PATH=/usr/z1090/bin:$PATH
export LD_LIBRARY_PATH=/usr/z1090/bin:$LD_LIBRARY_PATH
export MANPATH=/usr/z1090/man:$MANPATH
ulimit -c unlimited
ulimit -d unlimited
####### end of added for zPDT #######

export LESS="-I -R "
export HISTTIMEFORMAT="%T "
export HISTFILESIZE=5000

Get your granny to read it before you distribute it

I have spend a day trying to understand an electricity bill. There is online help called “how to understand your electricity bill”. It is a good example of how not to provide help, and people should learn from this.

When your write documents, get someone else outside of your area to review the document and make sure the documents are useful. My father was great, when I was eleven years old I had to write down how to make a pot of tea. My father would point out I had not checked/put any water in the kettle etc.

Some of the problems I found with the electricity bill help information were:

  • A PDF file showing the bill and help information, this had been created from a web page. I searched for a word, it was found but not displayed! In the PDF document it was white text on a white background – and so was invisible. If you copied the page to the clipboard and pasted it into a document, the text was visible.
  • In the pdf document, the instructions “move your mouse over the line to get more information” only applied to the web page, not the pdf page, but it was displayed in the pdf.
  • In the web page “how to understand your electricity bill” it gave an example bill, with a pop-up beside each line of interest. I thought this would be useful – but no. On the topic “Agreed Availability Charge”, the pop up said “Agreed Availability Charge”. Great – this adds no value, and does not explain, nor give me a link to what the Agreed Availability Charge actually is.
  • In the “Power information summary” section, it gives values like HH, MD, RE REAP, and only explains some of them. REAP is “reactive power”, what is this? I guess HH is Half Hourly readings. (Reactive power represents the electrical power that flows back and forth between the phase conductors and the neutral conductor of a three-phase network, but does not perform any mechanical work – this didn’t help me either.)
  • There is a helpful(?) video of someone who said to reduce the cost of your electricity, consider reducing the the availability cap/limit. Our value is 69, but does not give any units, nor how to compare it with you current usage.

The moral of this story is – get someone who is not an expert to try out your information.

RACF – I cannot delete a certificate!

I made a mistake creating a certificate, and could not delete it, because I got a message

IRRD109I The certificate cannot be added. Profile … is already defined.

In my JCL I had

RACDCERT GENCERT  -                                           
CERTAUTH -
SUBJECTSDN(CN('DocZosCADSA')-
O('COLIN') -
OU('CA')) -
NOTAFTER( DATE(2027-07-02 ))-
KEYUSAGE( CERTSIGN ) -
DSA
SIZE(1024) -
WITHLABEL('DocZosCADSA')

This created a certificate

I was missing the – after DSA, so the DSA, SIZE(1024) and importantly the WITHLABEL() was missing.

When I tried to recreate this certificate I got message

IRRD109I The certificate cannot be added. Profile 00.CN=DocZosCADSA.OU=CA.O=COLIN is already defined.

My problem was, what do I delete ?

The following command listed all of the certificate owned by certauth.

RACDCERT certauth LIST

I searched for CN=DocZosCADSA and it found. (Use the CN=… not the whole string)

Label: LABEL00000002 
...
Subject's Name:
>CN=DocZosCADSA.OU=CA.O=COLIN<
...

Note the label.

The command

RACDCERT CERTAUTH DELETE(LABEL('LABEL00000002'))

Deleted the certificate in error – problem solved.

Note: For a personal certificate the reported certificate was like

02AB.CN=SSCA256.OU=CA.O=SSS.C=GB

If you use the list command the certificate sequence number 02AB is on a different line to the remainder of the label.

Setting up a JES2 output NJE TCPIP node as a client using AT-TLS

This is part of some work I did to configure AT-TLS for a JES2 TCPIP node to another system.

I didn’t have a remote system to connect to, but I had a Python TLS server which the NJE node could connect to (and then end), which demonstrated the TLS connection.

The JES2 definition

The address of the remote end, running the Python TLS server was 10.1.0.2.

$ADDSOCKET(LAPTOP),IPADDR=10.1.0.2,LINE=3,NETSRV=1,NODE=50,PORT=2175,SECURE=NO 

Starting the NJE node

$SN,SOCKET=LAPTOP

The AT-TLS definitions

This definition acts as a client to a remote server, so AT-TLS needs to be configured as a AT-TLS client.

TTLSRule CPJES2OUT 
{
RemoteAddr 10.1.0.2
RemotePortRange 2175
Direction Output
TTLSGroupAction
{
TTLSEnabled On
}
TTLSEnvironmentAction
{
HandshakeRole Client
TTLSEnvironmentAdvancedParms
{
# clientAuthType needs to be required or Passthru
ClientAuthType PassThru
TLSv1 Off
TLSv1.1 Off
TLSv1.2 On
# TLSv1.3 On
}
TTLSKeyringParms AZFKeyringParms
{
Keyring start1/TN3270
}

TTLSConnectionAction
{
TTLSCipherParmsRef AZFCipherParms
TTLSConnectionAdvancedParms
{
# ServerCertificateLabel is for a server connection
# ServerCertificateLabel RSA2048
CertificateLabel RSA2048
# ApplicationControlled OFF
}
}
}

AZFCipherParms

I put common definitions into their own section, for example

TTLSCipherParms AZFCipherParms 
{
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# TLSv1.3
V3CipherSuites TLS_CHACHA20_POLY1305_SHA256
}

Using TLSv1.3

You need TTLSEnvironmentAdvancedParms to contain

TTLSEnvironmentAdvancedParms 
{
TLSv1.1 Off
TLSv1.2 On
TLSv1.3 On
}

and at least one TLSV1.3 cipher spec.

TTLSCipherParms AZFCipherParms 
{
# TLSv1.2
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
V3CipherSuites TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
V3CipherSuites TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
# TLSv1.3
V3CipherSuites TLS_CHACHA20_POLY1305_SHA256
# TLSv1.2
V3CipherSuites4Char TLS_CHACHA20_POLY1305_SHA256
V3CipherSuites4Char TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
V3CipherSuites4Char TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
V3CipherSuites4Char TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 C02
}

such as TLS_CHACHA20_POLY1305_SHA256

See Cipher suite definitions and search for 1301 (TLS_AES_128_GCM_SHA256) ,1302 (TLS_AES_256_GCM_SHA384) ,1303(TLS_CHACHA20_POLY1305_SHA256).
There is a column called TLSv1.3 (but it is hard to find). There are two tables, you need to use the second table to find what version of TLS the cipher specs provide.

Python server

The code below acted as a remote TLS server for the handshake.

import socket
import ssl
import struct
import pprint

HOST= ''
PORT = 2175

cafile="/home/colinpaice/ssl/ssl2/jun24/docca256.pem"
certfile="/home/colinpaice/ssl/ssl2/jun24/docec521june.pem"
keyfile="/home/colinpaice/ssl/ssl2/jun24/docec521june.key.pem"
certpassword = None

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.minimum_version = ssl.TLSVersion.TLSv1_2
context.maximum_version = ssl.TLSVersion.TLSv1_3
context.load_cert_chain(certfile, keyfile)
context.load_verify_locations(cafile=cafile)

context.verify_mode = ssl. CERT_REQUIRED
getciphers = context.get_ciphers()
#for gc in getciphers:
# print("get cipher",gc)

with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock:
sock.bind((HOST, PORT))
sock.listen(1)
with context.wrap_socket(sock, server_side=True) as ssock:
conn, addr = ssock.accept()
cert = conn.getpeercert()
pprint.pprint(cert)
v = conn.version()
print("version",v)
c = conn.cipher()
print("ciphers",c)
sock.close

When this ran, and the z/OS NJE node connected to it ($SN,SOCKET=LAPTOP), the output was

{'issuer': ((('organizationName', 'COLIN'),),
(('organizationalUnitName', 'CA'),),
(('commonName', 'DocZosCA'),)),
'notAfter': 'Jun 17 23:59:59 2025 GMT',
'notBefore': 'Jun 17 00:00:00 2024 GMT',
'serialNumber': '07',
'subject': ((('organizationName', 'RSA2048'),),
(('organizationalUnitName', 'SSS'),),
(('commonName', '10.1.1.2'),)),
'subjectAltName': (('IP Address', '10.1.1.2'),),
'version': 3}
version TLSv1.3
ciphers ('TLS_CHACHA20_POLY1305_SHA256', 'TLSv1.3', 256)

Showing the certificate, the level of TLS and the cipher spec used.

The messages on the z/OS console were

$SN,SOCKET=LAPTOP                                                        
$HASP000 OK
IAZ0543I NETSRV1 TCP/IP connection with IP Addr: 10.1.0.2 Port: 2175
Initiated
IAZ0543I NETSRV1 TCP/IP connection with IP Addr: 10.1.0.2 Port: 2175
Successful
IAZ0543I NETSRV1 TCP/IP connection with IP Addr: ::ffff:10.1.0.2 Port:
2175 ended due to TCP/IP error, rc: 1121

Setting up JES2 input NJE node (server) and AT-TLS

I got this working in response to a question about AT-TLS and JES2.

You need to configure the port and IP address of the destination node using AT-TLS.

I created the socket definitions

$ADDSOCKET(TLS),NODE=1,IPADDR=10.1.1.2,NETSRV=1,PORT=2275

Before you start

Get a working JES2 NJE, and AT-TLS environment. It makes it difficult to get the AT-TLS configured as well as getting NJE to work.

JES2 NJE needs a Netserver (NETSRV) to do the TCP/IP communication.

When you configure AT-TLS this intercepts the traffic to the IP address and port and does the TLS magic. This means you need a different netserver, and a tls specific port, and a TLS specific socket. It looks like the default TLS port is 2252. The doc says

SECURE=OPTIONAL|REQUIRED|USE_SOCKET
Indicates whether the NETSERV should accept only connection requests with a secure protocol in use such as TLS/SSL. When SECURE=REQUIRED is speci®edQ the NETSERV rejects all connection requests that do not specify a secure protocol is to be used for the connection. When SECURE=OPTIONAL is speciedQ the NETSERV allows connections with or without a secure protocol in use.
The default, USE_SOCKET, inherits the SECURE setting from the SOCKET statement associated with the NETSERV. If the SOCKET says SECURE=YES, then processing is the same as specifying
SECURE=REQUIRED on the NETSERV.
To specify that the NETSERV should use NJENET-SSL (2252) as the PORT it is listening on and the default port for outgoing connections, but not require all connections to use TLS/SSL, you must specify SOCKET SECURE=YES on the socket that is associated with the NETSERV and set the NETSERV to SECURE=OPTIONAL.

I do not understand this because AT-TLS will try to do a TLS handshake and fail if the session is not a TLS session.

It feels like the easiest way is to have a netserver just for TLS with its own port. I may be wrong.

In my PAGENT configuration, I took a working TLSrule and created

TTLSRule CPJES2IN 
{
LocalAddr ALL
RemoteAddr ALL
LocalPortRange 2252
Direction Inbound
Priority 255
TTLSGroupActionRef AZFGroupAction1
TTLSEnvironmentActionRef AZFEnvAction1
TTLSConnectionActionRef AZFConnAction1
}

This is for the inbound traffic on port 2252.

I defined the JES2 node

$TSOCKET(TLS),NODE=1,IPADDR=10.1.1.2,NETSRV=1,PORT=2252 

with the matching port=2252

I assigned this socket to netsrv1, and started it

$TNETSRV1,SOCKET=TLS
$SNETSRV1

I used a Python nje client to connect to z/OS. I used a modified version of the python NJE client, where I defined a certfile, keyfile and cafile.

I used

nje = njelib.NJE("N50","S0W1")
nje.set_debuglevel(1)
nje.setTLS is colin added code
#nje.setTLS(certfile="/home/colinpaice/ssl/ssl2/jun24/docec521june.pem",
# keyfile="/home/colinpaice/ssl/ssl2/jun24/docec521june.key.pem",
# cafile="/home/colinpaice/ssl/ssl2/jun24/docca256.pem")
connected = nje.session(host="10.1.1.2",port=2252,timeout=1)

Where the JES2 system is called S0W1, the node used is N50.

The z/OS IP address is 10.1.1.2, and the port is 2252.

There were no helpful messages to say the session was using TLS. I used Wireshark on the connection, and AT-TLS trace to check the TLS calls.

If I used a non TLS connection to the z/OS node I got

EZD1287I TTLS Error RC: 5003 Data Decryption    
LOCAL: ::FFFF:10.1.1.2..2252
REMOTE: ::FFFF:10.1.0.2..41288
JOBNAME: JES2S001 RULE: CPJES2IN

showing the AT-TLS definition was CPJES2IN.

RC 5003 will occur when the AT-TLS process is expecting an TLS message but receives a clear-text message – so no TLS request coming in.