next() doesn't play nice with any/all in python
While this is the default behaviour in Python versions up to and including 3.6, it's considered to be a mistake in the language, and is scheduled to change in Python 3.7 so that an exception is raised instead.
As PEP 479 says:
The interaction of generators and
StopIteration
is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently,StopIteration
raised accidentally inside a generator function will be interpreted as the end of the iteration by the loop construct driving the generator.
From Python 3.5 onwards, it's possible to change the default behaviour to that scheduled for 3.7. This code:
# gs_exc.pyfrom __future__ import generator_stopdef error(): return next(i for i in range(3) if i==10)all(error() for i in range(2))
… raises the following exception:
Traceback (most recent call last): File "gs_exc.py", line 8, in <genexpr> all(error() for i in range(2)) File "gs_exc.py", line 6, in error return next(i for i in range(3) if i==10)StopIterationThe above exception was the direct cause of the following exception:Traceback (most recent call last): File "gs_exc.py", line 8, in <module> all(error() for i in range(2))RuntimeError: generator raised StopIteration
In Python 3.5 and 3.6 without the __future__
import, a warning is raised. For example:
# gs_warn.pydef error(): return next(i for i in range(3) if i==10)all(error() for i in range(2))
$ python3.5 -Wd gs_warn.py gs_warn.py:6: PendingDeprecationWarning: generator '<genexpr>' raised StopIteration all(error() for i in range(2))
$ python3.6 -Wd gs_warn.py gs_warn.py:6: DeprecationWarning: generator '<genexpr>' raised StopIteration all(error() for i in range(2))
The problem isn't in using all
, it's that you have a generator expression as the parameter to all
. The StopIteration
gets propagated to the generator expression, which doesn't really know where it originated, so it does the usual thing and ends the iteration.
You can see this by replacing your error
function with something that raises the error directly:
def error2(): raise StopIteration>>> all(error2() for i in range(2))True
The final piece of the puzzle is knowing what all
does with an empty sequence:
>>> all([])True
If you're going to use next
directly, you should be prepared to catch StopIteration
yourself.
Edit: Nice to see that the Python developers consider this a bug and are taking steps to change it in 3.7.