Updating a web page from a server – redirect to a new page.

When using a web server as a back-end to an HTML page, for example filling in a form, the result from the back-end can be

  1. “Redirect to a new page”.
  2. A stream of HTML. This is an HTML document where the “boilerplate” or constant data, is intermixed with the variable data included inline. The HTML is displayed, and replaces the previous page.
  3. A stream of data containing data such as changed fields. The requesting page can use this stream to update its page.

Redirect to a new page

The server can send back status meaning redirect

#!/usr/bin/python3
import cgitb
import cgi
import sys, os, io

cgitb.enable(display=0, logdir="/home/colinpaice/first.log")
print('Status: 303 See Other2')
print("Content-Type: text/html")
// the url and any parmeters
print("Location: ../c3.html?colin=p1&error=p2")
# needs blank line to say end of headers
print()

This causes page c3.html to be displayed. The page can process the URL and process the options passed on the url.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<HTML lang="en">
<HEAD>
<script>
//  this function is  invoked on page load 
function js_onload_code (e){
  var url = document.location.href;
  // split the parameters from the url  base
  var p = url.split('?')[1]
  if (p)
  {
    var params = p.split('&');
    if (params)
    {
      var l = params.length;
      // display what was passed in
      alert("colin:"+url); 
      for (var i = 0 ; i < l; i++) 
      {
        // split it into kw"="value
        tmp = params[i].split('=');
        document.write(tmp[0] + "=" + tmp[1] +"!<br>");
        }
      } // if params
    } // if
}
// this causes the above script to be executed.
window.onload= js_onload_code ();
   
</script>
<BODY  >
<P>This is page c3.html
</BODY>
</HTML>

This displayed a page with url http://localhost/c3.html?colin=p1&error=p2

colin=p1!
error=p2!

This is page c3.html

If you repeatedly refresh this page you will get more and more data! It is better to use a field, then clear it before adding the information.

Collecting a wire-shark trace with TLS active for a browser

With TLS, the data transfer is encrypted. With TLS 1.3, part of the handshake is encrypted. In Wireshark this shows up as Encrypted Application Data.

You can configure wireshark to decrypt the data flow . You need to set an environment variable, and start the application from the terminal window, to pick up the environment variable.

export SSLKEYLOGFILE=$HOME/sslkeylog.log
google-chrome

This will cause the application( the google-chrome browser) to start and to write the TLS key data to the file.

Configure Wireshark to use this file:

  • Edit
  • Preferences
  • Expand the Protocols twistie
  • Scroll down to TLS ( typing T gets you near it)
  • Enter the Master-Secret log file name mine is /home/colinpaice/sslkeylog.log

Now, if you run the test you should get the data decrypted

Transport Layer Security
 TLSv1.3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
   Content Type: Change Cipher Spec (20)
   Change Cipher Spec Message
 TLSv1.3 Record Layer: Alert (Level: Fatal, Description: Decrypt Error)
  Opaque Type: Application Data (23)
    [Content Type: Alert (21)]
    Alert Message
      Level: Fatal (2)
      Description: Decrypt Error (51)

HTML sending stuff to the back-end server.

The simple and most common way of sending HTML data to a back-end server is using an anchor link

<a href="./echo.rexx">echo</a> 

This invokes the back-end program and displays the output. You have no control over the headers or what data is sent.

You can do more sophisticated stuff using the JavaScript fetch function, for example

  • specify different headers
  • change the data sent to the server, for example specify which data you want sent to the server, perhaps only send changed values rather than all input values
  • send the data to a different URL.

A quick aside on reading the JavaScript documentation and use of the arrow function

A lot of the documentation on JavaScript uses the arrow function, which can be useful, but I find distracting.

You can use a function such as mysubmit which takes an event:

function mysubmit(event)
{
console.log(event);
}
document.forms["target"].addEventListener('submit', mysubmit ) 

When the submit event occurs, “mysubmit” is invoked, and passed the event;

This can also be written, using inline functions as

document.forms["target"].addEventListener('submit', mysubmit(event) => { console.log(event;...} )
or
document.forms["target"].addEventListener('submit', (event) => { console.log(event;...} ) 

I have also seen it written as

function mysubmit(zevent)
{
console.log(zevent)
}
document.forms["target"].addEventListener('submit', (event) => mysubmit(event) ) 

At first glance this looks it could be simplified. However this is useful if you wanted to pass additional parameters to the function.

function mysubmit(zevent,a)
{
// a will have the value abc
console.log(zevent)
}
document.forms["target"].addEventListener('submit', (event) => mysubmit(event,'abc') ) 

I find the first example, document.forms[“target”].addEventListener(‘submit’, mysubmit ) easier to follow; and easier to write as you have to worry about matching the ending “})”s.

Using fetch

I used an HTML page with a form, so I could process the data. The fetch function sends the data to the specified URL, and waits for the response. You can specified headers and body and you get back a response object.

Setup

You need to associate a function with the “submit” of the form

 document.forms["target"].addEventListener('submit', mysubmit ) 

where mysubmit is the name of a callback function, for example

function submit(event) 
{ 
  event.preventDefault(); 
  displayChanged(event) 
  fetch("./echo.rexx", { 
      method: 'POST', 
      body: new URLSearchParams(new FormData(event.target)) // event.target is the form 
    })
    .then( check   )
    .then( zesult  ) 
    .catch(mycatch ) 
} 

Where

  • function submit(event) { defines the function, the parameter variable is called event.
  • event.preventDefault(); disables normal behaviour and lets me manage it.
  • displayChanged(event) my processing of the data before sending the data, see below.
  • fetch(“./echo.rexx”, { This sends the request to the backed “./echo.rexx”.
    • method: ‘POST’,
    • body: new URLSearchParams(new FormData(event.target)) Fetch has two parameters, the URL and a set of options, such as body, a set of headers{} etc. In this instance the second parameter is {method:”POST”, body: …. }.
  • }) // end of fetch
  • .then(check) this waits until the fetch finishes, see below
  • .then(zesult( ) see below
  • .catch( mycatch ) any exception
  • } // end of function

The .then processing is about using promises. Basically the .then has a wait for the response and return something to the next .then statement.

Check

This waits for the completion of the request.

function check(response) 
{ 
  if (!response.ok) { 
       throw new Error(`HTTP error! Status: ${response.status}`); 
  } 
  return response.text() 
 } 
  • It checks the “ok” flag in the data, if it is not OK, then throw an exception. The variable response.status has an error message.
  • Extract the body of the response and pass it on.

zesult

This waits for the completion of the previous “return response.text()”

function zesult(text) 
{ 
//console.log(text ); 
  let e = document.getElementById("errorField"); 
  e.innerHTML = text;                                                                     
 return (text) 
} 

and replaces the value of the piece of HTML with id=”errorfield”. For example

<p id="errorField" style="color:red ">No Errors yet</p> 

I do not know why two “.then”s are needed. It looks like it started another asynchronous process to extract the body from the response. In “check” if you display “response” it gives you some data. If you display “response.body” is gives a in-flight promise.

If you are tracing this, and you do not get into your code, check you have specified event.preventDefault(); before you do the fetch.

Looking at what data is being sent

Formdata allows you to work with the keyword=value data as generated by a form. You cannot get the data for other HTML elements outside of a form.

In my submit function I have displayChanged(event)

function displayChanged(event) 
{
// display the parameters
...
// Display what has changed
... 
}

Display the parameters

function displayChanged(event) 
{ 
  // display all the data 
  // extract into an iterable object  
 let f = new FormData(event.target)  ; 
 let data = ""
 for ([key, value] of f) { 
    data += ", " + key + ":" + value 
 } 
 console.log(data) 
...
}

For the data in my form, this displayed

,i1:initial value, text1:test@example.com, email:test@example.com, password:pw18,password25:pw25

where the bold names are the input field names.

Display what has changed

There is no direct way of processing just the changed variables, but you can fake it.

I had an input fields defined with an onchange call back.

<input id="password" name="password" value="pw18" onchange="change(this)" >
<script> 
function change(o){ 
  if  (o!= undefined) 
     o.setAttribute("changed","yes"); 
} 
</script>

If the field is changed, then the onchange -> change function is called. This sets an attribute “changed” to “yes”.

In the submit processing there

function displayChanged(event) 
{ 
...
  // display what has changed
  let er = event.target; 
  Array.from(er).forEach((r)  => 
  { 
    let id = r.getAttribute("id") 
    let ch = r.getAttribute("changed") 
    console.log( "id " + id + " changed " + ch ) 
  }) 
...
}

When I changed the two password fields, the console log had

id text1 changed null
id email1 changed null
id password21 changed yes
id password2 changed yes

Changing the parameters passed to the server.

You pass the parameters to the server using

fetch("./echo.rexx", { 
    method: 'POST', 
    body: new URLSearchParams(new FormData(event.target)) 

You can create a new instance of FromData, add keyword/value parameters to it, and pass it through to the back-end.

Above I explained how you can identify the “changed” data. You could take these elements and add to a new FormData, and pass those through to the server.

JavaScript: selecting an element

When writing some JavaScript to update elements in a page, I got confused selecting which elements to process. Below is my little crib sheet to help me remember.

Naming of parts.

You can have some HTML like

<div id="colin" name="request" class="myclass">requestor<div>

This division element can be referred to by the id=”colin”. It has an attribute name with a value of request. It has formatting parameters defined in the style sheet class=”myclass”.

If you have an element myDiv, you can use

let myclass=myDiv.getAttribute("class");
myDiv.setAttribute("class", "newclass");
myDiv.setAttribute("id", "newid");
myDiv.setAttribute("name", "newname");
myDiv.setAttribute("newattribute", "new");
myDiv.removeAttribute("name");

The id,class and any attribute look the same, but are slightly different when used to select elements.

Structure

You can have a low level element like a paragraph <p>..</p>

You can have more complex objects like a table <table>…</table> which can have table rows <tr>…</tr>, which can have table cells <td>…</td> which can have other elements such as <p>…</p>.

A division <div..>/div> can be used around text or an element to give it properties, such as a formatting or giving a word an id.

The high level structure is a document. You can create and add element to another element. You can remove elements. If you remove a table element, all of its children, <tr> and<td> are removed as well.

You can have elements which are not part of the document. You can add them to the document.

Locating an element in JavaScript

Originally there was just getElementById:

document.getElementById('newid');

This searches within document and its children for the first element with id=”newid”. If you have more than one element with id=”newid” only the first will be returned.

With a document

<!doctype html>
<html lang="en">
<head>
</head>
<body>
<p id="passed" class="myclass"></p>
<p id="p1" name="Colin">Colin</p>
<p id="p1" name="Jo">Jo</> 
</body>
</html>
var q= document.getElementById('passed');

returns an object called p#passed. This means it is a <p>aragraph object with id=”passed”.

It was quickly found that getElementById was not flexible enough, and so

querySelectorAll and querySelector were created. These are much more flexible.

  • let p=querySelector(“p”) returns the first element of type <p>.
  • let pall=querySelectorAll(“p”) returns a NodeList of all the <p> elements. Each element in the node list represents a <p> element. In my example HTML pall has a length of 3, because there are 3 <p> elements.
  • let p=querySelector(“#p1”) returns the first node with id=”p1″. It returns an element with “p#p1” saying this is a <p> element.
  • let pnl=querySelectorAll(“#p1”) with the above HTML returns a NodeList of length 2 where:
    • pnl[0] is the first node with id=”p1″. It returns an element with “p#p1”
    • pnl[1] is the second node with id=”p1″. It returns an element with “p#p1”.
  • let pnl=querySelectorAll(“p.myclass”) returns all “p” elements with class=”myclass”.
  • let pnl=querySelectorAll(“.myclass”) returns all elements with class=”myclass” regardless of type.
  • let nl=querySelectorAll(“[name=’colin’]”) returns all elements with attribute name=”colin”. With the above example data it returns a nodeList with one element which is a p#p1.

These selectors can be combined so for example you could ask for all <p> nodes within a <table> with id=”mytab” which have name=”colin”.

Finding data in a subtree

Within my HTML page I have

<table id="ztable" frame="box" rules="rows"> 
<tr id="template"> 
   <td id="col0" name="serial"><div><p>data</p></div></td> 
   <td id="col1" name="requestor"><div>requestor<div></td>    
</tr> 
<tr id="tr1"></tr>
</table> 

<table id="table2"> 
<tr id="tr1"></tr>
</table>

I can find the first table, and extract the rows out of it.

let tab = document.getElementById('ztable');
let tr = tab.querySelectorAll("tr");

This locates the table with id=”ztable” then searches within this for all <tr> tags. It returns a node list of length 2. It does not return the row in the second table(table2).

Not found?

Am I blind or is there no good documentation on how to write a web server application?

There is a lot of documentation on the internet on how to create a web page, for example using Java script or CSS to do clever things on a page. There is a lot of documentation on how to write a back-end application, with the choice of more than 10 different languages (Python, PHP, Java etc). But I could not find any good documentation on “start here”, how to get the back-end server to update a field on the web page, or what authorisation choices there are.

This blog post tries to cover some of the holes. It is what I have learned – it may not be the best; it may even be wrong, but I hope it helps.

If there is a good source of information, please tell me, and I’ll reference it.

What does a back-end do?

The back-end responds to a request such as

c3.html?colin=p1&error=p2&name=Colin%20Paice”

Where

  • c3.html is the name of the URL invoking the server. This could be page c3.html, or “myPython.py” an application script written in python, or “server”, a label which maps to an html page or a application name.
  • ? this divides the URL name from any parameters .
  • colin=p1. An HTML field with id called colin, has a value of “p1”
  • & a separator between arguments.
  • name=Colin%20Paice, an HTML field with id “name” has a value “Colin Paice”. Special characters like blank and & are replaced by their ASCII hex equivalent, so %20 is a blank.

Some requests pass data via STDIN, and the back-end application has to read from this stream to get the data.

The back-end can reply with a

  • A redirect to a different page (with parameters).
  • An HTML document where “constant” data such as “Please enter your Surname” is intermixed with “variable” data such as “Jones”. To update one field on the page, you have to send the whole document.
  • Other data, for example JSON, or HTML like <p id=”error” field=”firstName”> Invalid value </p> and the front end page can take this data and process it – for example to display the data, or highlight a value.

The request can come from an HTML page or as a REST request, where the application creates the string like c3.html?colin=p1&error=p2&name=Colin%20Paice” and uses an application such as CURL to send the request to the server, then parses the response.

As a developer for the back-end, I do not want to send the whole HTML document back – as I may not have it! I just want to send back a response to the request, for example field-1 has an invalid value, field-2 has an invalid length, the value of field-3 is not in the data base.

If you have multiple servers then a request may not go back to the same server because of load balancing.

Using TLS, it may do a new handshake for every request. TLS can cache session information for a short period to avoid the full handshake.

With some back-ends a request coming in starts a new thread, so there is a noticeable delay (half a second) when sending a request to a back-end. It is faster to use a static HTML home page than to call the back-end server to say “display the home page”.

State is not usually stored the back end, so you may want to return state information to the web page, and have it as a hidden, read only input field, which is sent to the back-end on the next request; or use cookies.

You need to decide what headers you need. You may want to use Cross-origin resource sharing (CORS) for protection.

You need to determine if the back-end can support the services. For example some z/OS services can easily be done using Rexx, but not using Python or PHP.

Basic design questions

What authorisation is needed?

  • Do you want to ensure the users use TLS to establish the session.
  • Do you want the server to send a certificate to the client, so the client authenticates the server.
  • Do you want the client to send a certificate to the server to authenticate the client, or make it optional?
  • Does the end user need to enter a userid and password. Do you have access to the enterprise wide identity system to be able to validate the userid or password, or will you just use the operating system’s checking.
  • You can have the back-end lookup the certificate and get back a userid.
  • Do you want the back-end to execute with the user’s authorisation, or the same authentication for all users.
  • Which ids are authorised to use what facilities, for example scripts, web pages, authorised operating system facilities, database tables etc..

What checking do you need to perform?

You may have checking on a web page, for example check a value is a number, or a string of a certain length. You also need to do the same checks in your back- end, as people may use a REST API to access your back-end, and no checking will have been done. With in-page checking, you get a response much faster than if the request goes to the back-end to validate

How do you transfer information from the front end to the back-end?

You can use cookies to store information in the clients front end. You need to consider Secure cookies and HttpOnly cookies, and should consider Same-site cookies.

You can have hidden input fields and read-only fields on a web page. These get passed to the back-end. A page called “developer.html” might have

<input type=”hidden” name=”action” value=”colinhidden”>
<input type=”text” name=”org” id=”name” value=”myorg.com” readonly=”readonly”>

<input type=”text” name=”role” id=”name” value=”Developer” readonly=”readonly”>

to pass action=”colinhidden” without the end user seeing it, and passing org=”myorg.com” with the user being able to see it, but not change it, and to pass “role= developer”. If you are using a REST API, you can create whatever values you like, as long as they are syntactically correct.

How do you transfer information from the back end to the front-end?

  • Using redirect does not work very well – it may be doable – but not very elegant.
  • Send a whole dynamically built HTML page – does not score well on the isolation of data and boilerplate.
  • Sending data back for the front end page to process. This seems to be the best solution.

The front end page can use javaScript to process the returned data. If the back-end has sent back:

<div id=”error” field=”name”> Invalid name specified>/div>
<div id=”error” field=”date”> Date is in the past</div>
<div id=”focus” which=”Surname”>

The JavaScript can do

  • Select all elements with a particular id, such as id=”error” and display them.
  • Take action over specific fields, for example for id=”error” then change the matching fields value to be red.
  • Take an action such as make the Surname field the focus.

You need to agree the protocol any records, and attributes, and what to do if there is a problem.

Why doesn’t ctrl-s work in ISPF edit? – ah it does now.

I had been editing a file, saving it, and finding the changes were not being picked up. Looking back, it was obvious; I was using CTRL-S the familiar Linux command, instead of F3 on ISPF.

I fixed this by configuring X3270 (on Linux).

My file /home/colin/.x3270pro now has

...
x3270.keymap: mine
! Definition of the 'mine' keymap
x3270.keymap.mine: #override \
    Alt<Key>4:          String("\\x00a2")\n\
    Ctrl<Key>backslash: String("\\x00a2")\n\
    <Key>Escape:    Clear()\n\
    <Key>End:        FieldEnd()\n\
    Ctrl<Key>Delete:   EraseEOF()\n\
    Ctrl<Key>Right:    NextWord()\n\
    Ctrl<Key>Left:    PreviousWord()\n\
    Ctrl<Key>Up:    Home()\n\
    <Key>Control_L: Reset()\n\
    <Key>Control_R: Reset()\n\
    <Key>Prior: PF(7)\n\
    <Key>Next: PF(8)\n\
    <Btn3Down>:   PA(1)\n\
    Ctrl<Key>1:   PA(1)\n\
    Ctrl<Key>s:   MoveCursor(3,15) String("save") Enter()\n\

When I started a new X3270 session, Ctrl -S went to the command line, typed save and pressed enter. Job Done ! The numbers are 0 based, so 3 means line 4 on the screen.

This makes life so much easier!

What’s my Appache HTTPD configuration?

I was trying to understand how my HTTPD configuration worked, and which file something was in. I was pleasantly surprised to find how easy it was to get this information. Rather than just list the configuration files as they were read, the data is displayed by function. This means I can see all the configuration information about “Environment variables”, or SAF.

It displays all the configuration keywords. For example for Module Name:mod_ibm_ssl.c it displays

  • SSLDisable – SSL is disabled for this server
  • SSLSNIMap – Add a new SNI/label mapping
  • SSLEnable – SSL is enabled for this server
  • SSLServerCert – Certificate within keyfile for this server
  • SSLClientAuth – The type of Client Authentication

Values are substituted, in my configuration files I had

colin.conf:define  urls  https://10.1.1.2   
...
RewriteRule "¬/(AdmRootca|Rootca)/ssl-cgi/(.*) ${urls}/$1/ssl-cgi-bin/$2 [R,NE,L] " 

This was displayed as

RewriteRule "^/(AdmRootca|Rootca)/ssl-cgi/(.*) https://10.1.1.2/$1/ssl-cgi-bin/$2 [R,NE,L] "

and I could check my configuration was correct.

You can change the startup to display the configuration at startup… but…

The documentation says you can add -DDUMP_CONFIG to the startup parameters. But when I tried this, it made the parameter list too long for the PARM=…

I had to change

//HTTPCP   PROC ACTION='start  -DDUMP_CONFIG ', 
//         DIR='/usr/lpp/ihsa_zos', 
//         CONF='/u/mqweb2/conf/httpd.conf'
//IHS      EXEC PGM=BPXBATCH,REGION=0M, 
//  PARM='SH &DIR/bin/apachectl -k &ACTION -f &CONF -DNO_DETACH   ', 
// MEMLIMIT=1236M  

to

//HTTPCP   PROC 
//  EXPORT SYMLIST=* 
//  SET ACTION='start  -DDUMP_CONFIG ' 
//  SET DIR='/usr/lpp/ihsa_zos' 
//  SET CONF='/u/mqweb2/conf/httpd.conf' 
//IHS      EXEC PGM=BPXBATCH,REGION=0M,PARMDD=PARMDD, 
// MEMLIMIT=1236M 
//PARMDD DD  *,SYMBOLS=EXECSYS 
SH &DIR/bin/apachectl 
 -k &ACTION 
 -f &CONF -DNO_DETACH 
 -DDUMP_CONFIG 
/* 

Then I got the configuration in STDOUT – though without the HTML links.

To display it online, you need to configure HTTPD

In my colin.conf file I had

LoadModule info_module modules/mod_info.so 

<IfModule mod_info.c> 
<Location /info> 
    SetHandler server-info 
#   Require all denied 
#   To allow access from a specific IP: 
#   Require ip 192.168.1 
</Location>
</IfModule> 

Of course you’ll enable security once it works…

I used

https://10.1.1.2:443/info

and it displayed the configuration. It has sections on

  • Loaded modules (I was pleased to see mod_info.so in the list)
  • Server settings
  • Hooks and providers
  • What I call “function type parameters” with the relevant configuration. For example for mod_alias.c it shows
    • Module directives
      • Alias – a fakename and a realname, or a realname in a Location
      • ScriptAlias – a fakename and a realname, or a realname in a Location
      • Redirect – an optional status, then document to be redirected and destination URL
      • Current configuration – and the information applicable to the function. For Module Name:mod_alias.c
        • In file: /u/mqweb2/conf/httpd.conf 
          • 467: Alias /icons/ “/u/mqweb2/icons/” 
          • 483: ScriptAlias /cgi-bin/ “/u/mqweb2/cgi-bin/”
        • In file: /u/mqweb2/conf/vhost80.conf  
          • 23: <VirtualHost *:80>  
          • 44:   ScriptAliasMatch /(PKIServ|Customers)/public-cgi/(.*) /usr/lpp/pkiserv/PKIServ/public-cgi/$
          • 2    : </VirtualHost>
      • For Module Name:mod_env.c
        • In file: /u/mqweb2/conf/vhost80.conf  
          • 23: <VirtualHost *:80>  
          • 27:   SetEnv LIBPATH “/usr/lpp/pkiserv/lib:/usr/lpp/ihsa_zos/lib:/usr/lpp/ihsa_zos/modules:/usr/lpp/ihsa_zos”  
          • 28:   SetEnv _PKISERV_CONFIG_PATH “/etc/pkiserv”    
          • : </VirtualHost>
        • In file: /u/mqweb2/conf/vhost443.conf  
          • 27: <VirtualHost *:443>  
          • 28:   SetEnv LIBPATH “/usr/lpp/pkiserv/lib:/usr/lpp/ihsa_zos/lib:/usr/lpp/ihsa_zos/modules:/usr/lpp/ihsa_zos”  
          • 29:   SetEnv _PKISERV_CONFIG_PATH “/etc/pkiserv”    
          • : </VirtualHost>

This is all very well thought through and easy to use.

Strong keys may be bad for your blood pressure.

I’ve just spent a couple of days trying to get a web server to use an Elliptic key with size 521. It works on Firefox, Curl, and OpenSSL, but not on Chrome. Weaker keys work of size 256 and 384 work fine, except in some cases a key size of 256 gave did not work on z/OS.

I’ve since found a page going back to 2015 discussing the dropping of several EC cipher specs. There is also discussion “NSA Suite B says that NSA uses curves P-256 and P-384” which (to some people) implied that other elliptic curves should not be used, and so were removed. There is another discussion that P256 is as strong as P521, but requires less CPU, and so P521 should not be used.

So the short answer is use Elliptic curves with key size 256 or 256, and do not use the others.

I was trying to get the Apache HTTPD server on z/OS to use a certificate, and it failed. I tried using a similar certificate from the openssl Simple server on Linux and it failed, so I’m guessing that my Chrome version (Version 114.0.5735.198 (Official Build) (64-bit)) does not support it.

The problem

In Chrome I got

This site can’t provide a secure connection 10.1.1.2 sent an invalid response.
ERR_SSL_PROTOCOL_ERROR

This is not entirely true, as the browser sent the Alert response to the server and it was not an ERR_SSL_PROTOCOL_ERROR. After this the browser code returned its caller saying “failed” . Using Wireshark, capturing encrypted data showed the data flowing from the browser to the server to port 443),

Transmission Control Protocol, Src Port: 38188, Dst Port: 443

Transport Layer Security
  TLSv1.3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
    Content Type: Change Cipher Spec (20)
    Change Cipher Spec Message
  TLSv1.3 Record Layer: Alert (Level: Fatal, Description: Decrypt Error)
    Opaque Type: Application Data (23)
    Alert Message
      Level: Fatal (2)
      Description: Decrypt Error (51)

In the Apache HTTPD error.log it had

SSL0209E: SSL Handshake Failed, ERROR processing cryptography. [10.1.0.2:38188 -> 10.1.1.2:443] 

Which is not very helpful. The problem occurred on the browser; all the server sees is Alert code 51.

Taking a system SSL trace on z/OS(GSK Server) gave me

Job HTTPCP8   Process 0501003C  Thread 00000032  read_tls13_alert 
  TLS 1.3 alert 51 received from ::ffff:10.1.0.2.38188. 

Diagnostics

Using tools like gsktrace on z/OS, and Wireshark to see the flow over the network. I could see that an alert was sent from the Chrome browser to server with type Decrypt Error(51). This meant it was a problem at the browser end, not the server end.

Wireshark capturing encrypted data.

If you use TLS 1.3 once the “change cipher spec” has been issued, all traffic is encrypted, and by default Wireshark cannot read it. For example

Transport Layer Security
 TLSv1.3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
   Content Type: Change Cipher Spec (20)
     Change Cipher Spec Message
 TLSv1.3 Record Layer: Application Data Protocol: http-over-tls
   Opaque Type: Application Data (23)
     Version: TLS 1.2 (0x0303)
       Encrypted Application Data: 489be8a0976798290766c9ee158c24f5863d18
      [Application Data Protocol: http-over-tls]d

and you cannot tell this is reporting an error.

Wireshark can decrypt this. You need to set an environment variable, and start the application from the terminal window, to pickup the environment variable.

export SSLKEYLOGFILE=$HOME/sslkeylog.log
google-chrome

This will cause the application( the google-chrome browser) to start and to write the TLS key data to the file.

Configure Wireshark to use this file:

  • Edit
  • Preferences
  • Expand the Protocols twistie
  • Scroll down to TLS ( typing T gets you near it)
  • Enter the Master-Secret log file name mine is /home/colinpaice/sslkeylog.log

Now, if you run the test you should get the data decrypted

Transport Layer Security
 TLSv1.3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
   Content Type: Change Cipher Spec (20)
   Change Cipher Spec Message
 TLSv1.3 Record Layer: Alert (Level: Fatal, Description: Decrypt Error)
  Opaque Type: Application Data (23)
    [Content Type: Alert (21)]
    Alert Message
      Level: Fatal (2)
      Description: Decrypt Error (51)

Diagnostics that were not helpful.

Chrome trace

Starting Chrome from a terminal session

google-chrome --enable-logging --v=1

The output was in ./.config/google-chrome/chrome_debug.log.

Using grep ERROR ./.config/google-chrome/chrome_debug.log gave me

...:ERROR:nss_util.cc(357)] After loading Root Certs, loaded==false: NSS error code: -8018
...:ERROR:ssl_client_socket_impl.cc(978)] handshake failed; returned -1, SSL error code 1, net_error -107
...:ERROR:ssl_client_socket_impl.cc(978)] handshake failed; returned -1, SSL error code 1, net_error -107

and no other useful information besides net_error -107. For these codes see here. 107 is an unhelpful message NET_ERROR SSL_PROTOCOL_ERROR.

Wireshark gave me decrypt error.

z/OS system SSL trace

Using GSKSRVR and CTRACE on z/OS. See here. This gave me

S0W1      MESSAGE   00000004  17:20:47.690772  SSL_ERROR 
  Job HTTPCP8   Process 0501003C  Thread 00000032  read_tls13_alert 
  TLS 1.3 alert 51 received from ::ffff:10.1.0.2.60830. 

Which shows the alert came from the browser.

Another way of getting system SSL trace.

The documentation says you can collect trace by changing the configuration. This allows you to trace just the pages/scripts that you want.

Creating a certificate on z/OS

On z/OS I used

RACDCERT ID(START1) GENCERT -                            
  SUBJECTSDN(CN('10.1.1.2') - 
             O('NISTECCTEST') - 
             OU('SSS')) -                                
   ALTNAME(IP(10.1.1.2))-                                
   NISTECC - 
   KEYUSAGE(   HANDSHAKE     )  - 
   SIZE(384) - 
   SIGNWITH (CERTAUTH LABEL('DOCZOSCA')) -               
   WITHLABEL('NISTECCTEST')     - 
                                                         

With different sizes 256,284, and 521. The IBM documentation says For NISTECC keys, valid key sizes are 192, 224, 256, 384, and 521 bits. I had problems with key size 521 bits.

On Linux I used

timeout="--connect-timeout 10"
enddate="-enddate 20240130164600Z" 
ext="-extensions end_user"
name="docec384"
key="$name.key.pem"
cert="$name.pem"

subj="-subj /C=GB/O=Doc2/CN="$name 
CA="docca256"
cafiles="-cert $CA.pem -keyfile $CA.key.pem "
rm $name.key.pem
rm $name.csr
rm $name.pem
passin="-passin file:password.file"
passout="-passout file:password.file"
md="-md sha384"
policy="-policy signing_policy"
caconfig="-config ca2.config"
caextensions="-extensions clientServer"
config="-config openssl.config"

openssl ecparam -name secp384r1 -genkey -noout -out $name.key.pem 
openssl req $config -new -key $key -out $name.csr -outform PEM -$subj $passin $passout 
openssl ca $caconfig $policy $ext $md $cafiles -out $cert -in $name.csr $enddate $caextensions 

openssl x509 -in $name.pem -text -noout|lessf

I used this as a template with different flavours of -name secp384r1

  • -name secp384r1
  • -name prime256v1 which worked
  • -name secp521r1 which did not work

You can get a list of valid names from the command openssl ecparam -list_curves.

Running a Linux web server

I tried using the openssl s_server, and could also reproduce this problem (with a much faster turnaround). I used

tls="-tls1_3  "
#cert=" -cert ./docec256.pem -key ./docec256.key.pem"
#cert=" -cert ./docecgen.pem -certform pem -key docecgen.key.pem -keyform pem" 
#cert=" -cert ./docec521.pem -certform pem -key docec521.key.pem -keyform pem"
cert=" -cert ./docec384.pem -certform pem -key docec384.key.pem -keyform pem" 
CA="-chainCAfile ./docca256.pem"
cipher1="-cipher TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384"

cipher=$cipher1
port="-port 4433 "

strict="-x509_strict -strict"
debug="-trace"

#  ca="-CAfile ./zpdt.ca.pem "
openssl s_server $port $tls  $cert $cipher $CA $debug $strict -www  

Chrome startup options

You can start chrome from a terminal with many startup options. See here

IKJ56500I COMMAND … NOT FOUND. Macro does not exist

Due to some random typing, when I try to edit a member I keep getting

  • IKJ56500I COMMAND M NOT FOUND
  • Macro does not exist

After some playing around, I found this was caused by having a list of PDS members, using the prefix command E, and putting something(M) in the Prompt column.

To fix it, use E and put / in the Prompt column. This gives the EDIT Entry Panel and shows Initial Macro..M . Blank out the M, and press PF3 to clear it.

Easy when you know how.

I expected this to show up under ISPF 2 (Edit) but the initial Macro field was blank.

One minute – debugging an HTML page

I had an HTML page with javascript and wanted to debug it and see what was going on. It was all pretty easy, but some things took a while to understand. I’ll describe some of the things I did using Chrome browser. Firefox has similar capabilities.

I found JavaScript debugging reference which looks pretty good.

Using the Chrome Debugger Tools, part 3: The Sources Tab is pretty comprehensive.

Getting started

  • Display an html page.
  • To get into developer tools use Ctrl+Shift+I

You get a display like

  1. The web page (squashed up) to a narrow column
  2. Elements – you can see the HTML components. As you move your mouse over the elements in the list, the fields in the web page are highlighted
  3. Sources – you can see the source of the program with the javascript etc. Errors in the file will get flagged on this page
  4. Debug” switch. The icon next to it is the “step over”
  5. Watch” – You can specify which variables you want displayed permanently
  6. “Scope” you can list and display all the variables available to you at that point in the web page
  7. An example of the source

Display and edit the HTML

Clock on the Elements tab (2 in the above picture) to display the source.

You may have your javaScript within the page, or referenced by http://./reply.js. You can see these under the page sub tab, or as a tab in the source (7). You may have to click on the name on the “Page” tab, to get it displayed in the source pane.

This shows sssserver.html is the main one, and checkCipher.js has been included. Some of these may be displayed as tabs above the sourc

  • It gives a good high level view of the program for example it may have <head>…</head>. The … show there is omitted content. Click on the … to display the content.
  • As you move your mouse over the HTML the web page elements will be highlighted. Moving over the <body> shows the whole web page, moving over a <p> shows just the paragraph.
  • As you click on some html, it displays the CSS on the right had side.
    • It tells you what CSS is being used for example the “p” tag with option “display”:block
    • As you move your mouse over the CSS picture it highlights the data on the html page
  • You can edit the HTML, add elements, delete elements etc

Set breakpoint

  • Click on the Sources (3) tab
  • Click on the line number of interest – it toggles blue.
  • Run the page

Step through the code

  • Click on the Sources (3) tab.
  • Click on the pause button (4); it gets grayed out
  • Run your script. I clicked the submit button.
  • The display changes and it shows “Debugger paused”.
    • Open the “watch” twistie and click “+”. You can now enter a variable name to display the object.
      • Open the object’s twistie to display the attributes of the object
      • Open the “Scope” twistie. Open the “Local” twistie. This will display all of the local variables. Open the variable’s twistie to see all its attributes. I do not think you can change the data. Use ctrl+alt+left mouse to expand or compress the twistie.
  • Open the “Breakpoint” twistie and select “Pause on uncaught exception” and “Pause on caught exception“. This will stop when it detects a problem
  • If you click the icon next to the Pause Icon (or F10) it will step through the code.
  • If you click on a line of javascript and use right-click you select “Continue to here”.
  • Within an active code segment, hovering over a variable will display the attributes. (You can also go to “Scope” and display from that window. Clicking on the window object( for example) gives all the operations you can do on that object (for example all of the on…. method names) .
  • If your code has console.log(“something”) this will appear in the console tab.

To get out of the current debug press the browser’s stop (X) and browser’s “reload/refresh” button.

Where did it spend its time?

If you click on the “Network” tab it shows where time was spent on the network. For example which files were got, and how long it took to get the data. It gives information on

  • Which file was loaded.
  • Status – 200 is OK.
  • Type eg .gif.
  • Initiator – which page did the request come from.
  • Size ( if it was downloaded) or “cache” if it was already in the browser cache.
  • Time in milliseconds.
  • “Waterfall” breaks down the time spent

Under the “Performance” tab you can record the activity, and display it.

Use Control+E to start, then Ctrl+E to stop it.

It displays information like

What do all of the keys do?

If you click on the settings wheel, and select Shortcuts, it displays all the options and key combinations.

Debugging a java script file?

Chrome caches these. Disable this by going to the Network table and click on “disable cache”