Strange behaviour with Python Generator

  • A+
Category:Languages

I was running a piece of code that unexpectedly gave a logic error at one part of the program. When investigating the section, I created a test file to test the set of statements being run and found out an unusual bug that seems very odd.

I tested this simple code:

array = [1, 2, 2, 4, 5] # Original array f = (x for x in array if array.count(x) == 2) # Filters original array = [5, 6, 1, 2, 9] # Updates original to something else  print(list(f)) # Outputs filtered 

And the output was:

>>> [] 

Yes, nothing. I was expecting the filter comprehension to get items in the array with a count of 2 and output this, but I didn't get that:

# Expected output >>> [2, 2] 

When I commented out the third line to test it once again:

array = [1, 2, 2, 4, 5] # Original array f = (x for x in array if array.count(x) == 2) # Filters original ### array = [5, 6, 1, 2, 9] # Ignore line  print(list(f)) # Outputs filtered 

The output was correct (you can test it for yourself):

>>> [2, 2] 

At one point I outputted the type of the variable 'f':

array = [1, 2, 2, 4, 5] # Original array f = (x for x in array if array.count(x) == 2) # Filters original array = [5, 6, 1, 2, 9] # Updates original  print(type(f)) print(list(f)) # Outputs filtered 

And I got:

>>> <class 'generator'> >>> [] 

TL;DR: Why is updating a list in Python changing the output of another generator variable? This seems very odd to me.

 


As others have mentioned Python generators are lazy. When this line is run:

f = (x for x in array if array.count(x) == 2) # Filters original 

nothing actually happens yet. You've just declared how the generator function f will work. Array is not looked at yet. Then, you create a new array that replaces the first one, and finally when you call

print(list(f)) # Outputs filtered 

the generator now needs the actual values and starts pulling them from the generator f. But at this point, array already refers to the second one, so you get an empty list.

If you need to reassign the list, and can't use a different variable to hold it, consider creating the list instead of a generator in the second line:

f = [x for x in array if array.count(x) == 2] # Filters original ... print(f) 

Comment

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: