Colin’s “TCPIP on z/OS” message explanations

Purpose

This blog post is a repository of my interpretation of the messages from the Z/OS communications server family of products. Ive tried to add more information, or explain what some of the values are. it is aimed at search engines, not as a readable article.

EZZ7853I AREA LINK STATE DATABASE

This message can come from

• OSPF external advertisements : The DISPLAY TCPIP,tcpipjobname,OMPROUTE,OSPF,EXTERNAL
• OSPF area link state database: The DISPLAY TCPIP,tcpipjobname, OMPROUTE, OSPF, DATABASE, AREAID=area-id

in topic DISPLAY TCPIP,,OMPROUTE.

Type

1. Router links advertisement
2. Network links advertisements
3. Network summaries
4. Autonomous System(whole network) summaries
5. Autonomous System(whole network) external advertisements (DISPLAY TCPIP, tcpipjobname, OMPROUTE, OSPF,EXTERNAL)

EZZ0318I HOST WAS FOUND ON LINE 8 AND FIRST HOP ADDRESS OR AN = WAS EXPECTED

I got this with

ROUTE 2001:db8::7/128 host 2001:db8:1::3    IFPORTCP6      MTU 5000 

Which has a first hop address! The problem was /128. Remove this and it worked. If you then issue TSO NETSTAT ROUTE it gives

DestIP:   2001:db8::7/128 
  Gw:     2001:db8:1::3 
  Intf:   IFPORTCP6         Refcnt:  0000000000 
  Flgs:   UGHS              MTU:     5000 

EZZ7904I Packet authentication failure, from 10.1.1.1, type 2

An OSPF packet of the specified type was received. The packet fails to authenticate.

System programmer response

Verify the authentication type and authentication key specified for the appropriate interfaces on this and the source router. The types and keys must match in order for authentication to succeed. If MD5 authentication is being used and OMPROUTE is stopped or recycled, ensure that it stays down for at least 3 times the largest configured dead router interval of the OSPF interfaces that use MD5 authenticaiton, in order to age out the authentication sequence numbers on routers that did not recycle.

Types are

• 0 Null authentication
• 1 Simple password
• 2 Cryptographic authentication

See OSPF Version 2.

From the message description, this could be a timing issue.

EZZ7921I OSPF adjacency failure, neighbor 10.1.1.1, old state 128, new state 4, event 10

EZZ7921I.

I got this restarting frr on Linux.

The Neighbor State Codes can be one of the following:

• 1 Down
• 2 Attempt
• 4 Init (session has (re) started
• 8 2-way
• 16 ExStart
• 32 Exchange
• 64 Loading
• 128 Full. the router has sent and received an entire sequence of Database Description Packets.

The Neighbor Event Codes can be one of the following:

• 7 SeqNumberMismatch
• 8 BadLSReq
• 10 1-way. An Hello packet has been received from the neighbor, in which this router is not mentioned. This indicates that communication with the neighbor is not bidirectional. For example the remote end is restarting.
• 11 KillNbr
• 12 InactivityTimer
• 13 LLDown
• 15 NoProg. This event is not described in RFC1583. This is an indication that adjacency establishment with the neighbor failed to complete in a reasonable time period (Dead_Router_Interval seconds). Adjacency establishment restarts.
• 16 MaxAdj. This event is not described in RFC2328. This indicates that OMPROUTE has exceeded the futile neighbour state loop threshold (DR_Max_Adj_Attempt). Even if a redundant parallel interface (primary or backup) exists, OMPROUTE continues to attempt to establish adjacency with the same neighbouring designated router over the existing or alternate interface.

EZZ7905I No matching OSPF neighbor for packet from 10.1.1.1, type 4

• EZZ7905I No matching OSPF neighbor for packet from 10.1.1.1, type 4
• EZZ7904I Packet authentication failure, from 10.1.1.1, type 2

I got these when I was using OSPF Authentication_type=MD5, and the Authentication_Key_ID did not match.

BPXF024I

You get messages prefixed by this message if SYSLOGD is not running.

For example

BPXF024I (TCPIP) Oct 6 10:11:10 omproute 67174435 : EZZ8100I OMPROUTE subagent Starting

With the SYSLOGD running you get

EZZ8100I OMPROUTE SUBAGENT STARTING

TELNET and AT-TLS

EZZ6035I TN3270 DEBUG CONN DETAIL 1035-00 Policy is invalid for the conntype specified.

EZZ6035I TN3270 DEBUG CONN DETAIL 
IP..PORT: 10.1.0.2..34588
CONN: 0000004E LU: MOD: EZBTTACP
RCODE: 1035-00 Policy is invalid for the conntype specified.
PARM1: PARM2: SECURE PARM3: POLICY NOT APPLCNTRL

POLICY NOT APPLCNTRL

The AT-TLS policy needs

TTLSEnvironmentAdvancedParms CSQ1-ENVIRONMENT-ADVANCED 
{ 
  ApplicationControlled         On 
...
}

Now you know, it is obvious that APPLCNTRL in the message means ApplicationControlled!

PARM2: SECURE PARM3: NO POLICY

EZZ6035I TN3270 DEBUG CONN   DETAIL                      
  RCODE: 1035-00  Policy is invalid for the conntype specified.      
  PARM1:          PARM2: SECURE   PARM3: NO POLICY                   

There is no AT-TLS policy for the port being used. The message does not tell you which port or policy is being used. The operator command “D TCPIP,TN3270,PROFILE” shows which ports are in use.

EZZ6060I TN3270 PROFILE DISPLAY 968                            
  PERSIS   FUNCTION        DIA  SECURITY    TIMERS   MISC      
 (LMTGCAK)(OPATSKTQSSHRTL)(DRF)(PCKLECXN23)(IPKPSTS)(SMLT)     
  L******  ***TSBTQ***RT*  TJ*  TSTTTT**TT  IP**STT  SMD*      
----- PORT:  2023  ACTIVE           PROF: CURR CONNS:      0   

The TS under security mean TLS connection, Secure Connection.

Use the Unix commands pasearch -t 1>a oedit a to display the configuration and search for “port”. The port value may be specified – or it may be within a range.

LocalPortFrom: 2023 LocalPortTo: 2025

EZZ6035I TN3270 RCODE: 1030-01 TTLS Ioctl failed for query or init HS.

PARM1: FFFFFFFF PARM2: 00000464 PARM3: 77B77221

The PARM1 value is the return value, the PARM2 value is the return code, and the PARM3 value is the reason code for the ioctl failure; these values are defined in z/OS UNIX System Services Messages and Codes.

• Error numbers. 464 is ENOTCONN:The socket is not connected
• Reason codes 7221: The connection was not in the proper state for retrieving.

I got this when

• there was problems with the System SSL configuration, such as invalid certificate name,
• when the z/OS certificate was not suitable eg the key needed to be bigger
• the HandshakeRole ServerWithClientAuth was specified – it should be HandshakeRole Server
• Breton Imhauser said this could also include a TCP connection flood – crude Denial Of Service attempt of TN3270. This is what it looks like when a set of remote clients are repeatedly establishing a tcp connection with your TN3270 and hanging up. They establish the TCP connection and FIN-ACK it without telnet negotiation. User may claim it was a “heartbeat” test of the host.

In my /etc/syslog.conf I have

daemon.debug /var/log/SSHDdebug

There were additional messages in this file after the TLS handshake problem.

EZZ6035I TN3270 DEBUG CONFIG EXCEPTION RCODE: 600F-00 System SSL initiation failed.

PARM1: 000000CA PARM2: 00000000 PARM3: GSK_ENVIRONMENT_INIT

AT-TLS did not have access to the keyring. For example need access to

RDEFINE RDATALIB START1.MQRING.LST UACC(NONE)
PERMIT START1.MQRING.LST CLASS(RDATALIB) ID(TCPIP) ACCESS(CONTROL)
tso setropts refresh raclist(rdatalib)

and perhaps access to

PERMIT IRR.DIGTCERT.LISTRING CLASS(FACILITY) ID(TCPIP) ACCESS(READ)

1030-02 – also to do with keyrings.

EZZ6035I TN3270 DEBUG TASK EXCEPTION TASK: MAIN MOD: EZBTZMST
RCODE: 1016-01 Port Task setup failed.
PARM1: 0000102B PARM2: 00000BCF PARM3: 00000000
EZZ6006I TN3270 CANNOT LISTEN ON PORT 3023, CONNECTION MANAGER TERMINATED, RSN =102B

This was caused by

PORT 
...
   3023 TCP *   SAF     VERIFY 

and getting

EZD1313I REQUIRED SAF SERVAUTH PROFILE NOT FOUND EZB.PORTACCESS.S0W1.TCPIP.VERIFY               

Define the profiles and give the userid access to it.

OMPRoute

EZZ7815I Socket 11 bind to port 521, address :: failed, errno=111:EDC5111I Permission denied., errno2=74637246

This was caused by

PORT
   520 UDP OMPROUTE            ; RouteD Server 
   521 UDP OMPROUTE            ; RouteD Server for IP V6 

The name after the UDP (OMPROUTE) did not match my job name which was trying to use it.

EDC5111I Permission denied. errno2=0x744C7246.

0x744C7246 744C7246. This problem occurred with using port 22 (Telnet).

Changing to port 2222 showed that it was just port 22, the other configuration worked.

Commenting out the RESTRICTLOWPORTS and the PORT reservation for “22 SSHD” showed it was one of those.

Using the RESTRICTLOWPORTS parameter to control access to unreserved ports below port 1024 (an application cannot obtain a port in the range 1 – 1023 that has not been reserved by a PORT or PORTRANGE statement, unless the application is APF-authorized or has OMVS superuser [UID(0)] authority).

The solution was to use port reservation such as

    22 TCP SSHD* NOAUTOLOG  ; OpenSSH SSHD server

EZZ7811I COULD NOT ESTABLISH AFFINITY WITH INET, ERRNO=1011:

EDC8011I A NAME OF A PFS WAS SPECIFIED THAT EITHER IS NOT CONFIGURED OR IS NOT A SOCKETS PFS., ERRNO2=11B3005A

I had RESOLVER_CONFIG=//’ADCD.Z24C.TCPPARMS(TCPDATA)’ pointing to an invalid data set.

EZZ7937I THE IPV6 OSPF ROUTING PROTOCOL IS DISABLED

The message in the documentation is pretty useless.

It means there was no valid IPV6 interfaces defined, and no IPV6 addresses.

EZZ7956I OSPF area 0.0.0.3 not configured, interface JFPORTCP6 not installed

The documentation

I was missing an IPv6_area for the interface

IPv6_AREA Area_Number=0.0.0.3; 
IPv6_OSPF_Interface 
      Name = JFPORTCP6 
      Attaches_To_area=0.0.0.3 
      Prefix=2001:db8:8::/64 
      ; 
IPv6_Default_Route 
      Name=JFPORTCP6 
      Next_Hop=2300::1 
      ; 
IPv6_OSPF 
        RouterID = 7.7.7.7 

EZZ8125I IPV6 OSPF ROUTERS NONE

The documentation is useless.

I got NONE even though I had a router.

EZZ7886I NOT CONNECTED TO AREA SPECIFIED ON … DISPLAY COMMAND

I got this response to the F P1,IPV6OSPF,database command. You do not specify an area!