next() doesn't play nice with any/all in python next() doesn't play nice with any/all in python python python

next() doesn't play nice with any/all in python


While this is the default behaviour in Python versions up to and including 3.6, it's considered to be a mistake in the language, and is scheduled to change in Python 3.7 so that an exception is raised instead.

As PEP 479 says:

The interaction of generators and StopIteration is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently, StopIteration raised accidentally inside a generator function will be interpreted as the end of the iteration by the loop construct driving the generator.

From Python 3.5 onwards, it's possible to change the default behaviour to that scheduled for 3.7. This code:

# gs_exc.pyfrom __future__ import generator_stopdef error():    return next(i for i in range(3) if i==10)all(error() for i in range(2))

… raises the following exception:

Traceback (most recent call last):  File "gs_exc.py", line 8, in <genexpr>    all(error() for i in range(2))  File "gs_exc.py", line 6, in error    return next(i for i in range(3) if i==10)StopIterationThe above exception was the direct cause of the following exception:Traceback (most recent call last):  File "gs_exc.py", line 8, in <module>    all(error() for i in range(2))RuntimeError: generator raised StopIteration

In Python 3.5 and 3.6 without the __future__ import, a warning is raised. For example:

# gs_warn.pydef error():    return next(i for i in range(3) if i==10)all(error() for i in range(2))

$ python3.5 -Wd gs_warn.py gs_warn.py:6: PendingDeprecationWarning: generator '<genexpr>' raised StopIteration  all(error() for i in range(2))

$ python3.6 -Wd gs_warn.py gs_warn.py:6: DeprecationWarning: generator '<genexpr>' raised StopIteration  all(error() for i in range(2))


The problem isn't in using all, it's that you have a generator expression as the parameter to all. The StopIteration gets propagated to the generator expression, which doesn't really know where it originated, so it does the usual thing and ends the iteration.

You can see this by replacing your error function with something that raises the error directly:

def error2(): raise StopIteration>>> all(error2() for i in range(2))True

The final piece of the puzzle is knowing what all does with an empty sequence:

>>> all([])True

If you're going to use next directly, you should be prepared to catch StopIteration yourself.

Edit: Nice to see that the Python developers consider this a bug and are taking steps to change it in 3.7.