Re-reading File When Cycling Over A File Object In Python

From this context: import itertools lines = itertools.cycle(open('filename')) I'm wondering how I can implement that same 'feature' but rereading the file when it reaches the end,

Solution 1:

I'd use:

itertools.chain.from_iterable(itertools.starmap(open, itertools.repeat(("filename",))))

or:

itertools.chain.from_iterable(itertools.starmap(lambda: open("filename"), itertools.repeat(())))

You can also write a generator comprehension (I think I like this best!):

(line for _ in itertools.repeat(()) for line in open("filename"))

Here's the imperative (statement-based) equivalent:

defcycle_file(filename):
    whileTrue:
        for line inopen(filename):
            yield line

Or, with Python 3.3 (using PEP 380 subgenerator delegation):

defcycle_file(filename):
    whileTrue:
        yieldfromopen(filename)

One problem with all of these is that (on a GC platform e.g. Jython) the file will not be closed until the file object is GCed, which could happen some time later. To prevent the open file leaking, you have to call close on it or use a contextmanager (with statement). This comes out naturally in the imperative form:

defcycle_file(filename):
    whileTrue:
        withopen(filename) as f:
            for line in f:
                yield line

or

defcycle_file(filename):
    whileTrue:
        withopen(filename) as f:
            yieldfrom f

Trying to close the file with a generator comprehension becomes highly contrived:

(line forfin (itertools.chain(f, (f forfin (f,) if f.close() and False))
                forfin (open("filename") for_in itertools.repeat(())))
 forlinein f)

It'd be nice if Python had a way to specify that an opened file should close itself upon reaching the end of the file, or a way to tell a contextmanager-iterator to close itself on StopIteration.

Solution 2:

Something like

defcycle_file(f):
    whileTrue:
        ln = f.readline()
        if ln == "":
            f.seek(0)
            continueyield ln

Except that it might be nice to put in a check for empty files, which I'll leave to you.