Python: Is The Split Function Evaluated Multiple Times In A List Comprehension?

There is something I've been wondering about for a while. Is the split executed once or multiple times in this list comprehension? l = [line.split()[i] for i in indexes] I current

Solution 1:

The expression on the left in a list comprehension is evaluated anew for each element, yes.

If you only need it evaluated once, you need to do exactly what you did; call it first and store the result to be re-used in the list comprehension.

From the List displays documentation:

In this case, the elements of the new list are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce a list element each time the innermost block is reached.

Emphasis mine.

You could also disassemble the list comprehension using the dis.dis() function:

>>>import dis>>>dis.dis(compile('[line.split()[i] for i in indexes]', '', 'eval'))
  1           0 BUILD_LIST               0
              3 LOAD_NAME                0 (indexes)
              6 GET_ITER            
        >>    7 FOR_ITER                22 (to 32)
             10 STORE_NAME               1 (i)
             13 LOAD_NAME                2 (line)
             16 LOAD_ATTR                3 (split)
             19 CALL_FUNCTION            0
             22 LOAD_NAME                1 (i)
             25 BINARY_SUBSCR       
             26 LIST_APPEND              2
             29 JUMP_ABSOLUTE            7
        >>   32 RETURN_VALUE        

The FOR_ITER opcode starts the loop (with the JUMP_ABSOLUTE closing it), and each time a LOAD_NAME line, LOAD_ATTR split and CALL_FUNCTION are executed. In other words, the bytecodes 13 through to 19 implement the line.split() part, and it is executed each time through the loop, which runs from bytecodes 7 through to 29.

(Python 3 note: list comprehensions got their own scope and you'll need to extract the code object from the outer code object constants; dis.dis(compile('[line.split()[i] for i in indexes]', '', 'eval').co_consts[0])).

Solution 2:

I'll add that your example of LC is an replacement for:

l = []
for i in indexes:
    l.append(line.split()[i])

So,the answer is most definitely Yes, it's evaluated each time per iteration.

Solution 3:

As @Dalen says, when reasoning about a list comprehension, I believe you can expect it to behave just as if you'd done the same thing without a comprehension. @Martijn shows how you can verify this by looking at the assembly.

My answer

1. Provides an easier method to verify the behavior yourself (in the sense that you don't need to read assembly).

2. Shows the behavior for a function in multiple places within the list comprehension (I found this question when I was wondering about how often an "outer function" (see below) is called).

Code:

definner_func(i):
    print('called inner')
    return i

defouter_func(n):
    print('called outer')
    returnrange(n)

l = [inner_func(i) for i in outer_func(5)]

This will print called outer once and called inner 5 times, verifying that, just as in a normal for loop, the outer function is executed once and the inner function once per loop.