Tag: striped

Setting up striping for MQ logs and other experiences with SMS.

You get improved throughput using striped data sets. Products like MQ and DB2 have logs and page sets which can exploit VSAM striping.   This blog post tells you what you need to configure to be able to use it.

Why do striped logs have higher throughput?

When writing to a dataset, the duration of the request is composed of three parts

  1. Issuing the request and getting it to the IO subsystem on the mainframe.
  2. Getting from the IO subsystem on the mainframe down to the Storage Controller
  3. Transferring the data.

The time depends on the amount of data to transfer.  Striping uses more than one volume,  so less data is written to each device, and so the response time is shorter.

When writing pages to the MQ log which is not striped, all the pages are sent down the channel to one disk.

When using an MQ log with 4 stripes, you have 4 volumes.

  1. Page 1 goes to volume A
  2. Page 2 goes to volume B
  3. Page 3 goes to volume C
  4. Page 4 goes to volume D
  5. Page 5 goes to volume A
  6. Page 6 goes to volume B

The time to send 2 pages to volume A,  etc should be less than the time taken to send 6 pages to a non striped volume.

When I worked for IBM I had JCL to define the logs which was set up with striping.   The storage manager person had set up the SMS definitions so it was easy for me.  I recently tried to set up stripes for my MQ logs on my personal z/OS system and I had to set up my own SMS definitions.  In theory it was easy, but it took me a long time because I did not know about one tiny little SMS command.

SMS, Storage Classes, and Data Classes ( and Management classes)

When I first experienced System Managed Storage(SMS) there were lots of new terms to learn.  I’ll give a 10 second summary of SMS.  

  • Datasets have attributes, such as size in MB, record length, is it a PDSE or a sequential file.
  • Disk volumes have attributes .  You want production datasets on new disks, because they are faster than old disks.  You want to keep the backup copy of a dataset on a different set of disks to the original data set.
  • Some datasets you want to backup daily, have multiple backups and take off them site.  Some data sets you backup once a week, and have only one copy.   If the data set has not been used for a month, then migrate it to tape. 

The SMS classes are

  1. Data Class.  Specify the data set attributes; What record length, space allocation, PDSE or sequential file.
  2. Storage Class defines the criteria for the allocation of data sets.  It defines if the volumes need to be “dual copy” or not, or have a performance response time better than a specified value
  3. Management Class – back this up daily.

You have ACS scripts, which can do processing like, if the HLQ is SYS1.** then set Management Class =  frequent_backup.  If the data set name is like MQS.**.PROCLIB then use Data Class  = BIGALLOC which allocates 50 Cylinders.

You configure the SMS classes using the interactive ISMF tool.

Setting up MQ logs with stripes

You need JCL like

//IDCAMS EXEC PGM=IDCAMS,REGION=4M 
//SYSPRINT DD SYSOUT=* 
//SYSIN DD * 
DEFINE CLUSTER - 
  (NAME (MQS.STRIPED.LOGCOPY1.DS01) - 
  LINEAR - 
  DATACLAS(DCEXT ) - 
  STORAGECLASS(SCMQS ) - 
  VOLUMES( A4USR1 USER00) - 
  SHAREOPTIONS(2 3) - 
  CYL(1000) ) - 
  DATA - 
    (NAME(MQS.STRIPED.LOGCOPY1.DS01.DATA) ) 

LISTCAT ENTRIES(MQS.STRIPED.LOGCOPY1.DS01) ALL

 

You need to pick a Data Class which has extended format. 

Data Set Name Type . . . . . : EXTENDED
 If Extended . . . . . . . . : REQUIRED

You need to specify a storage class with striping configured. I thought it would be easy to have a field in the Storage Class called “Number of stripes”, but no, it is way more complex that this.

There is a Storage Class field called Sustained Data Rate(SDR). As I see it, the crazy reasoning behind its use, is that at one time, 3390’s could sustain a data rate of 4 MB/second. If you want more than this you clearly need to use more 3390s. By specifying an SDR value of 16 with 3390s, SMS could work out you want at least 4 stripes – see; crazy!

I changed the storage class to have the SDR value of 16.

Performance Objectives
Direct Millisecond Response . . . : 1
Direct Bias . . . . . . . . . . . :
Sequential Millisecond Response . : 1
Sequential Bias . . . . . . . . . :
Initial Access Response Seconds . :
Sustained Data Rate (MB/sec) . . . : 16


Aside: Another crazy. If you want to cache a dataset – such as a PDSE, there is no storage class switch saying “CACHE=Y|N”. You specify the response time by setting “Sequential MilliSecond Response=1″. MSR= 1 says “This Must be Cached”, MSR= 10 says “This may be cached if SMS has the capacity”. These are only advisories.

You also need to specify the volumes to be used. VOLUMES( A4USR1 USER00) . Your ACS may be set up to choose the appropriate volumes (with the right attributes and enough space). My ACS is very simple, and I have to explicitly specify the SMS volume names.

Did it work? No.

I read the books, and it looked like I had done the right thing. I enabled SMS tracing of allocations using the operator command

SETSMS VOLSELMSG(ON)

This gave me in my JCL

IGD17385I =====SUMMARIZED ANALYSIS MESSAGES ON DEFINING DATA SET MQS.STRIPED.LOGCOPY1.DS01 =====
IGD17386I VOLSELMSG(ON,0) TYPE(ALL) JOBNAME(* ) ASID() 076 STEPNAME( ) DSNAME(* )
IGD17387I DS_TYPE(VSAM) SC(SCMQS) DC(DCEXT) GS(N) SPACE(4000KB) BESTFIT(N) STRIPING(N)
IGD17290I THERE WERE 3 CANDIDATE STORAGE GROUPS OF WHICH THE FIRST 3 078
WERE ELIGIBLE FOR VOLUME SELECTION.
THE CANDIDATE STORAGE GROUPS WERE:SGMQS SGBASE SGEXTEAV
IGD17269I 247 VOLUMES WERE REJECTED BECAUSE THEY WERE NOT ONLINE
IGD17269I 247 VOLUMES WERE REJECTED BECAUSE THE UCB WAS NOT AVAILABLE

I could see the Storage Class, and the Data Class being used, but why did it have STRIPING(N)?

After a day’s worth of struggling, I eventually dumped and wrote a program to format the SMS DCOLLECT definitions . I could see the SDR value for the storage class was 0! My changes had not been picked up.

I used the operator command

setsms scds(SYS1.S0W1.DFSMS.SCDS)

to reload the definitions,reran my job and it worked. Because I still had SMS VOLumeSELectMSG(ON) enabled, the job produced

IGD17387I DS_TYPE(VSAM) SC(SCMQS) DC(DCEXT) GS(N) SPACE(4000KB) BESTFIT(N) STRIPING(Y)

and

IGD17070I DATA SET MQS.STRIPED.LOGCOPY1.DS01
ALLOCATED SUCCESSFULLY WITH 2 STRIPE(S).
IGD17172I DATA SET MQS.STRIPED.LOGCOPY1.DS01
IS ELIGIBLE FOR EXTENDED ADDRESSABILITY
IDC0508I DATA ALLOCATION STATUS FOR VOLUME A4USR1 IS 0
IDC0508I DATA ALLOCATION STATUS FOR VOLUME USER00 IS 0
IDC0181I STORAGECLASS USED IS SCMQS
IDC0181I DATACLASS USED IS DCEXT
IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0

The listcat command

LISTCAT ENTRIES(MQS.STRIPED.LOGCOPY1.DS01) ALL

gave me 

STRIPE-COUNT-----------2
...
VOLUME                      
   VOLSER------------A4USR1  
   STRIPE-NUMBER----------1  
 VOLUME                      
   VOLSER------------USER00  
   STRIPE-NUMBER----------2

Success!