I was using a Python program to access a z/OS service, and found there were times when my code did not clean up and close the resource.
It took me an afternoon to find out how to do it. I found pyzfile by daveyc an excellent example of how to cover Python advanced topics.
pyzfile example
The documentation has
from pyzfile import *
try:
with ZFile("//'USERID.CNTL(JCL)'", "rb,type=record",encoding='cp1047') as file:
for rec in file:
print(rec)
except ZFileError as e:
print(e)
Breaking this down
Understanding the “with”
try:
with ZFile("//'USERID.CNTL(JCL)'", "rb,type=record",encoding='cp1047') as file:
...
do something with file
...
except ZFileError as e:
print(e)
When the with ZFile(…) as file: is executed the code conceptually does
- standard set up processing
- open the file and return the handle
- do processing using the file handle
- when ever it leaves the with code section perform the close activity
Note:This could have been done with
try:
open the file
...
do something
...
except:
...
finally: # do this every time
if the file was opened:
close the file
but this is not quite so tidy and compact as the with syntax
In more detail…
- The def __init__(self,..): method is invoked and passed the parameters. It saves parameters using statements like self.p1
- The __enter__(self): is invoked passing the instance data(self). It seems to have no other parameters.
- In the pyzfile, the code issues return self._open(). This invokes the function _open to open the data set.
- When the with processing completes, it invokes the function __exit__(self, exc_type, exc_value, exc_traceback): This is invoked whether the code returned normally, or got an exception.
- In the pyzfile, the code issue executes self.close(). So however the “with” processing ends, the close is always executed
Handing errors
I’ve seen that using the “with” clause, people tend to throw exceptions when problems are found
For example with the pyfile code there is
class ZFileError(Exception):
""" ZFile exception """
def __init__(self, message: str, amrc: dict = None):
self.message = message
self.amrc = amrc
if amrc is None:
self.amrc = {}
super().__init__(self.message)
def __str__(self) -> str:
return self.message
def amrc(self):
"""
Returns the amrc dict at the time of error.
:return: The ``__amrc`` structure at the time of error.
"""
return self.amrc
class ZFile:
...
def _open(self):
...
self.handle = open...
if not self.handle:
raise ZFileError(f"Error opening file '{self.filename}':
{self.lib.zfile_strerror().decode('utf-8')}")
return self
Understanding the “for”
The code above had
with ZFile("//'USERID.CNTL(JCL)'", "rb,type=record",encoding='cp1047') as file:
for rec in file:
print(rec)
How does the “for” work?
The key to this code are the functions
##################################################
# Iterators
##################################################
def __iter__(self):
return self
def __next__(self):
ret = self.read()
if not ret:
raise StopIteration
return ret
When the for statement is processed it processes the __next__ function. This does the work of getting the next record and returning it.
There is a lot of confusing documentation about iterators, iteration and iterables. Let’s see if my description helps clarify or just adds more confusion.
Something is iter-able of you can do iteration on it; where iteration means taking each element in turn.
In Python a list is iter-able
for l in [0,1,2,3,]
print(l)
will iterate over the list and return the element from the list
0
1
2
3
Records in a file are a bit more abstract, you cannot see the whole file, but you can say get the next record – and move through the file until there are no more records.
An iterator is the mechanism by which you iterate. Think of it as a function. The Python documentation is pretty clear.
Most people define
def __iter__(self):
return self
For most people, just specify this. The PhD class may use something different.
The mechanism of “for” uses the __next__ function
def __next__(self):
ret = self.read()
if not ret:
raise StopIteration
return ret
Which obtains the next element of data. If there are no more elements, then raise the StopIteration exception.
If you do not handle the StopIteration exception, then Python handles it for you and leaves the processing loop.
Conclusion
With both of these techniques “with” and “for” I could extract records from a z/OS resource.
I’ve used the “with” and “for” with yield to hide implementation detail
# create the function to read the file
def readfile(name):
try:
with ZFile(name, "rb,type=record,noseek") as file:
for rec in file:
yield rec
except ZFileError as e:
print(e)
# process the file using for ... readline()
def reader(...):
for line in readfile("//'IBMUSER.RMF'"):
do something with the data
ColinPaice 