Skip to content

bpo-31718: Raise ValueError instead of SystemError when calling methods of uninitialized io.IncrementalNewlineDecoder objects#3913

Closed
orenmn wants to merge 10 commits into
python:masterfrom
orenmn:bpo31718-dont-raise-sys-err
Closed

bpo-31718: Raise ValueError instead of SystemError when calling methods of uninitialized io.IncrementalNewlineDecoder objects#3913
orenmn wants to merge 10 commits into
python:masterfrom
orenmn:bpo31718-dont-raise-sys-err

Conversation

@orenmn

@orenmn orenmn commented Oct 7, 2017

Copy link
Copy Markdown
Contributor

In addition:

  • Add tests to test_io to verify that indeed ValueError is raised instead of SystemError.
  • While we are here, fix refleaks in io.IncrementalNewlineDecoder.__init__().

https://bugs.python.org/issue31718

Comment thread Modules/_io/textio.c Outdated
PyErr_SetString(PyExc_ValueError,
"IncrementalNewlineDecoder.__init__ not called");
PyErr_Format(PyExc_ValueError,
"%.200s.__init__() not called",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert this change. The problem is that IncrementalNewlineDecoder.__init__ not called. This can happen if subclass' __init__ doesn't call super().__init__. Subclass's __init__ can be called, but self->decoder == NULL if IncrementalNewlineDecoder.__init__ not called.

Comment thread Lib/test/test_io.py
Comment thread Modules/_io/textio.c Outdated
PyObject *errors)
/*[clinic end generated code: output=fbd04d443e764ec2 input=89db6b19c6b126bf]*/
{
Py_XDECREF(self->decoder);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a race condition, destructor of self->decoder can call IncrementalNewlineDecoder.__init__ . Use Py_XSETREF().

@orenmn

orenmn commented Oct 7, 2017

Copy link
Copy Markdown
Contributor Author

Sorry about messing up so much.

@serhiy-storchaka

Copy link
Copy Markdown
Member

LGTM. I'll merge if @benjaminp doesn't have comments.

@benjaminp benjaminp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things:

  1. Most of the _io module classes abstract away the reinitialization logic with a macro CHECK_INITIALIZED. It would be nice to do that here.
  2. Checking for initialization should be the first thing a method does. In this implementation, .reset() can partially reset the object before raising an exception.

@orenmn

orenmn commented Oct 10, 2017

Copy link
Copy Markdown
Contributor Author

Note that i added an ok field to nldecoder_object. I would change CHECK_INITIALIZED_DECODER() to rely on the decoder field if you think it is too expensive.
Also, i added CHECK_INITIALIZED_DECODER() to the beginning of the getter of newlines. I would remove that if you think this is too much of a change to the behavior.

@serhiy-storchaka

Copy link
Copy Markdown
Member

Python implementation of .reset() can partially reset the object before raising an exception.

I don't understand why the ok field is added. __init__() can fail only if PyUnicode_FromString("strict"), but the error field is not used and could be removed (but see bpo-31722). I would set error = PyUnicode_FromString("strict") if error == NULL before changing any fields (if we want to keep an unused field).

@orenmn

orenmn commented Oct 14, 2017

Copy link
Copy Markdown
Contributor Author

@benjaminp @serhiy-storchaka so should the C implementation allow reset() to partially reset the object?

@csabella csabella requested a review from benjaminp May 17, 2019 22:39

@auvipy auvipy left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rebase is needed

@csabella csabella requested review from benjaminp and serhiy-storchaka and removed request for benjaminp February 7, 2020 12:19
@csabella

csabella commented Feb 7, 2020

Copy link
Copy Markdown
Contributor

@orenmn, please resolve the merge conflicts. Thank you!

@orenmn

orenmn commented Feb 7, 2020

Copy link
Copy Markdown
Contributor Author

@csabella I am really sorry, but I am not working on CPython nowadays..

@csabella

csabella commented Feb 7, 2020

Copy link
Copy Markdown
Contributor

@orenmn, no problem and thanks for responding. I'm going to close this pull request for now so that someone else can continue with it if they are interested. If you come back to it later on and it's not resolved yet, feel free to reopen it at that time. Thanks again!

@csabella csabella closed this Feb 7, 2020
@serhiy-storchaka

Copy link
Copy Markdown
Member

It would be better to not close abandoned PRs which fix a bug or implement a valuable enhancement, but recreate them, update and merge.

@csabella

csabella commented Feb 7, 2020

Copy link
Copy Markdown
Contributor

Sorry, but that was my point in closing the abandoned PRs, to allow someone the opportunity to do just that. If someone becomes aware that a PR exists but isn't being worked on, then it gives that person a chance to pick up the issue and continue it. Otherwise these PRs are just being left open and no one seems to be aware of them. I thought I was implementing a good workflow for the older issues, but perhaps it should be discussed more and then formalized?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants