[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Nick Coghlan]

On 18 November 2014 14:50, Guido van Rossum <guido at python.org> wrote:

> Nick,
>
> I think we've gone through enough clarifications of the PEP now to be
> clear on the proposal. I saw in one of your earliest replies (right after
> Chris posted his first draft) that you're hesitant to support the PEP
> because of what would have to change to contextlib. What I couldn't quite
> read is whether you think that the proposal by itself is not an
> improvement, or whether you're just worried about compatibility.
>

I think it's an improvement - I really like the fact that it brings
generators into line with your reasoning in the with statement PEP that
flow control constructs should be locally visible. At the moment, "raise
StopIteration" in a generator context is effectively non-local flow
control, as it means any function call (explicit or implicit) or yield
point may gracefully stop generator execution, rather than only return
statements.

>From the point of view of a generator *consumer*, I'm now happy that the
backwards incompatibility is limited to cases where you throw in
StopIteration and want to check if you got the same StopIteration instance
back - you'll now also need to check for RuntimeError with __cause__ set to
the StopIteration instance you threw in. You can construct scenarios where
such a check will give a false positive, but they're getting seriously
contrived at that point.

That's obscure enough that I think it's on par with other behavioural
tweaks we've included in the "Porting to Python X.Y" guides in the past.


> Apparently you know of a large group of users who use an older 3rd party
> version of contextlib, and for whom that older, 3rd party contextlib should
> keep working with future versions of Python 3 without updating their
> version of contextlib -- did I get that right? What exactly is the
> constraint there that makes their version of contextlib immutable even
> though the version of Python they are using may move forward?
>

I don't even remember how that came up now, but it was entirely
hypothetical, and I think it can be ignored as a concern. As you say, being
able to update to Python 3.5 without also being able to update to a new
version of contextlib2 would just be weird.

Even if such a strange scenario did somehow come up, it would still be
possible to switch to a conditional import where they used the stdlib
version if available, and only fell back to contextlib2 on earlier versions
of Python.


> Separate from this special case, I am also worried about backward
> compatibility, and I have yet to get a good idea for how widespread code is
> that depends on StopIteration bubbling out from generators. I also don't
> have a good idea how often this issue bites users, but I have a feeling it
> does bite. E.g. this quote from c.l.py (
> https://mail.python.org/pipermail/python-list/2014-November/680775.html):
>
> """
> I did find it annoying occasionally that raising StopIteration inside a
> generator expression conveys a different behavior than elsewhere. It did
> take me quite a while to understand why that is so, but after that it
> did not cause me much of a headache anymore.
> """
>

One advantage of needing a __future__ import on the generator author side
is that you always have the option of changing your mind, and *never*
making the new behaviour the default.

That wouldn't be a *good* outcome, but I don't think it would be
intolerable.

OTOH, I'm also not sure the status quo is sufficiently problematic to be
worth changing. Yes, it's a little weird, but is it *that* much weirder
than the unavoidable issues with exceptions thrown in __next__,
__getitem__, __getattr__ and other special methods where a particular type
of exception is handled directly by the interpreter?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141120/472b8ddf/attachment.html>