Zowe: messages on z/OS when using Zowe

ZWEAO503E The server is not ready to handle the request: /apicatalog/api/v1/

I got this when the ZAAS service was stopped, use the f zowe,appl=DISPLAY to display the status of the various tasks.

You can use f zowe,appl=start(ZAAS) to start it. Check the zowe sysprint data set for any error messages.

I also got the following message in the z/OSMF logs

CWWKS2907E: SAF Service IRRSIA00_CREATE did not succeed because user adcdb has insufficient authority to access APPL-ID IZUDFLT. SAF return code 0x00000008. RACF return code 0x00000008. RACF reason code 0x00000020.

BPXTLS failed: rc=-1, return code=143, reason code=0x0be80000

Error: service has no impersonation; make sure process user has sufficient authority:
z/OS: program control flag must be set, UPDATE access to BPX.SERVER and BPX.DAEMON SAF resources is required
Other platforms: impersonation is not supported

This is actually from BPX1TLS, phread_security_np. There are not many cases where 0be80000 is mentioned. This link mentions PTKTDATA profiles.

LoadBalancer does not contain an instance for the service zaas

This was a tricky one to track down.

Zowe works with z/OSMF. They communicate with certificates. This means the z/OSMF trust store keyring needs the CA of the Zowe server certificate, and the Zowe trust store keyring needs the CA of the z/OSMF server key.

Not only that , if verifyCertificates STRICT is specified in the zowe.yaml file, then Zowe will check the server’s certificate sent from z.OSMF. This includes the altname or altIP, the altIP is the IP address of the connection. An internal IP address is 127.0.0.0. An external IP address is almost anything else. On my system I have addresses 127.0.0.1 and 10.1.1.2 and 10.1.2.6. You can display them using the TSO NETSTAT HOME command.

The zOSMF certificate did not have an altip specified, and so failed the Zowe checks. I had to set the Zowe option verifyCertificates NOSTRICT for it to work.

You can also get this message when the system is partially up.

ZWEAM701E; InvalidKeyException: Unrecognized RSA or RSASSA-PSS key algorithm name

The request to the URL ‘/zaas/api/v1/auth/login’ has failed: SignatureException: Unable to compute RS256 signature. Cause: Unrecognized RSA or RSASSA-PSS key algorithm name. caused by: InvalidKeyException: Unrecognized RSA or RSASSA-PSS key algorithm name. messageAction:Refer to specific exception details for troubleshooting.

It looks you Zowe server needs an RSA key for use with JWT. I changed from an Elliptic key to an RSA and it worked.

Using the web browser

403 Impersonation error

• Error: response status is 403
• Download
• Impersonation error

In the logs/zssServer….log it has

BPXTLS failed: rc=-1, return code=139, reason code=0x0be803d1
Error: service has no impersonation; make sure process user has sufficient authority:
z/OS: program control flag must be set, UPDATE access to BPX.SERVER and BPX.DAEMON SAF resources is required

This is because Zowe does not have access to the profiles.

Action:
Check the profiles are defined, and the Zowe userid has UPDATE access to them.

Header X-Zowe-Auth-Failure: ZWEAG160E No authentication provided in the request.

I actually got the message ZWEAG160E No authentication provided in the request 5 times.

The header is collecting all errors during the treatment of the request. Because the response was 503 the Gateway retries it 5 times. This is strange, but expected behaviour.

I also got an error message in the z/OSMF log /global/zosmf/data/logs/zosmfServer/logs/trace.log

CWWKS2907E: SAF Service IRRSIA00_CREATE did not succeed because user colin2 has insufficient authority to access APPL-ID IZUDFLT. SAF return code 0x00000008. RACF return code 0x00000008. RACF reason code 0x00000020.

Give the userid access to the profile. It may be easier to connect the userid to the group. In my case to group IZUUSER.

I gave the userid access and then got

IYUCM0008W: The user ID colin is not authorized to resource type Navigation Task, resource name Software Services

X-Zowe-Auth-Failure: ZWEAG160E No authentication provided in the request

{
“additionalInfo”: null,
“debug”: null,
“httpStatus”: 403,
“messageID”: “IYUCM0008W”,
“messageText”: “IYUCM0008W: The user ID colin is not authorized to resource type Navigation Task, resource name Software Services.”,
“requestMethod”: “GET”,
“requestUri”: “/zosmf/provisioning/rest/1.0/scr”
}

X-Zowe-Auth-Failure: ZWEAG160E No authentication provided in the request.

I’m not sure about this. Certificate authentication should work, but I needed to specify

–basic –user colin2:password

Python

Fatal (HANDSHAKE_FAILURE): Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)

This post said

I got the Insufficient buffer remaining for AEAD cipher fragment error when trying to use the Python client (browser and curl worked normally). After lots of trial and error, it seems the error was related to the SSL certificates not having properly configured extension fields, especially keyUsage and extendedKeyUsage, and the fact that urllib3 doesn’t include /etc/ssl/certs/ca-certificates.crt by default.

Another post said “This is an OpenJDK issue with TLS 1.3 “occurring between the OpenShift default route and Mule applications with re-encypt route enabled.”

Keystore file message is confusing

I saw the following in a GW trace record

[35mZWESVUSR [0;39m [36mINFO [0;39m((o.a.t.u.n.N.certificate)) Connector
[https-jsse-nio-0.0.0.0-7554], TLS virtual host [default], certificate type [UNDEFINED] configured from keystore [/u/tmp/zowec/.keystore] using alias [CONN1.IZUDFLT] with trust store [null]

it was confusing because I was not using a keystore file. This looks like a bug in Spring, and can be ignored.

ZWEAG121E Authorization header is missing, or the request body is missing or invalid for URL ‘/zaas/api/v1/auth/login’

• messageAction: Provide valid authentication.
• messageReason: The authorization header is missing, or the request body is missing or invalid.

I got this when trying to do a certificate login – and no password.

• The certificate used did not have a mapping to a userid.
• The userid was revoked

ZWEAO500E: The service has encountered a situation it doesn’t know how to handle.

Please contact support for further assistance. More details are available in the log under the provided message instance ID.

Case 1

I got this when I changed the a logging value using the REST API. I had

“configuredLevel”: “WARN2″‘

which is invalid.

Case 2

In the log I got

java.lang.ClassCastException: com.ibm.crypto.plus.provider.ECPublicKey incompatible with java.security.interfaces.RSAPublicKey

This is because the key type in my keyring was an Elliptic curve, but the code assumesit is RSA, and uses an RSA function – which does not work with an Elliptic key.

HTTPIE: Https error: ConnectionError: HTTPSConnectionPool(host=…, port=…)

Max retries exceeded with url: … (Caused by NewConnectionError(‘: Failed to establish a new connection: [Errno 111] Connection refused’)) while doing a GET request to URL: https://…

The back end was not active so the TLS hand shake failed.