The bear traps when using enclaves

I hit several problems when trying to use the enclave support.

In summary

1. The functions to set up and use an enclave are available from C, but the functions to query and display usage are not available from C (and so not available from Java).
2. Some functions caused an infinite loop because they overwrote the save area.
3. Not all classify functions are available in C.  For example ClientIPAddr
4. I had problems in 64 bit mode.
5. Various documentation problems
6. It is not documented that you need to pass the connection token to __server_classify(_SERVER_CLASSIFY_CONNTKN, (char * ) connToken. You get errno2 errno2=0x0330083B.  Home address space does not own the connect token
   from the input parameter list.
7. You can query the CPU used by your enclave using the IWMQTME macro (in supervisor state!). I had to specify CURRENT_DISP=YES to cause the dispatcher to be called to update the CPU figures.  By default the CPU usage figures are updated at the end of a dispatch cycle.  On my low use system, my transactions were running without being redispatched, and so the CPU “used” was reported as 0.

In more detail…

Minimum functionality for C programs.

You cannot obtain the CPU used by the enclaves from a C program, as the functions are not defined.  I had to write my own assembler code to called the assembler macros to obtain the information.  Some of these macros require supervisor state.

Many macros clobber the save area

Many macros, use a program call to execute a function.  Other functions such as  IWMEQTME use a BASR instruction.  This function then does a standard save of the registers.  This means that you need to have a standard function save area.  Without this, the callers save area was used, and this overwrote the register, and Branch back… just branched to after the macro.

Instead of a function like

EDEL     RMODE  ANY 
EDEL     AMODE  31 
EDEL     CSECT 
          USING *,12 
          STM  14,12,12(13) 
          LR   12,15 
          L    6 0(1)  the work area  
          L    2,4(1)  ADDRESS OF THE passed data              
          IWM4EDEL ETOKEN=0(2),MF=(E,0(6),COMPLETE),                   XX 
                CPUTIME=8(2),ZAAPTIME=16(2),ZIIPTIME=24(2),            XX 
                RSNCODE=32(2),RETCODE=36(2) 
          LM   14,12,12(13) 
          SR   15,15 
          BR   14 

I needed to add in code to create a save area, for example with a different macro

QCPU     RMODE  ANY 
QCPU     AMODE  31 
QCPU     CSECT 
** CAUTION THE IWMEQTME CORRUPTS SAVE AREA SO PROGRAM NEEDS ITS OWN
** SAVE AREA 
      USING *,12 
      STM  14,12,12(13) 
      LR   2,1 
      LR   12,15 
      LA    0,WORKLEN 
      STORAGE OBTAIN,LENGTH=(0) 
      ST     1,8(,13) FORWARD CHAIN IN PREV SAVE AREA 
      ST     13,4(,1) BACKWARD CHAIN IN NEXT SAVE AREA 
      LR     13,1     SET UP NEW SAVE AREA/REENTRANT WORKAREA 
      L    2,0(2)  ADDRESS OF THE CPUTIME 
      IWMEQTME CPUTIME=8(2),ZAAPTIME=16(2),ZIIPTIME=24(2),          X 
            CURRENT_DISP=YES,                                       X 
            RSNCODE=4(2),RETCODE=0(2),MF=(E,32(2),COMPLETE) 
      LR   3,15 
* free the resgister save area
      LR     1,13               ADDRESS TO BE RELEASED 
      L     13,4(,13)          ADDRESS OF PRIOR SAVE AREA 
      LA    0,WORKLEN           LENGTH OF STORAGE TO RELEASE 
      STORAGE RELEASE,           RELEASE REENTRANT WORK AREA        X 
            ADDR=(1),            ..ADDRESS IN R1                    X 
            LENGTH=(0)           ..LENGTH IN R0 
      L    14,12(13) 
      LR  15,3 
      LM   0,12,20(13) 
 SR   15,15 
      BR   14   

Problems using a 64 bit program

I initially had my C program in 64 bit mode. This caused when I wrote some stub code to use the assembler interface, as the assembler macros are supported in AMODE 31, but my program, and storage areas were 64 bit, and the assembler code had problems.

Various documentation problems

1. It is not documented that you need to pass the connection token to __server_classify(_SERVER_CLASSIFY_CONNTKN, (char * ) connToken. You get errno2 errno2=0x0330083B.  Home address space does not own the connect token
   from the input parameter list
2. _SERVER_CLASSIFY_SUBSYSTEM_PARM Set the transaction subsystem parameter. When specified, value contains a NULL-terminated character string of up to 255 characters containing the subsystem parameter being used for the __server_pwu() call.  This applies to _Server_classify_ as well as __server_pwu().   The sample applies for  _SERVER_CLASSIFY_TRANSACTION_CLASS , _SERVER_CLASSIFY_TRANSACTION_NAME, _SERVER_CLASSIFY_USERID.
3. Getting report and server class back from __server-classify
   1. It is  _SERVER_CLASSIFY_SRVCLSNM not _SERVER_CLASSIFY_SERVCLSNM.
   2. You use _SERVER_CLASSIFY_RPTCLSNM@, _SERVER_CLASSIFY_SERVCLS@, _SERVER_CLASSIFY_SERVCLSNM@ without the @ at the end.   I think this is meant to imply these are pointers.
   3. They did not work for me.  I could not see when the fields are available.   The classify work is only done during the CreateWorkUnit() request.  I request it before this function, and after this function and only got back a string of hex 0s.