Skip to content

bpo-31956: Add start and stop parameters to array.index()#4435

Closed
Phaqui wants to merge 9 commits into
python:masterfrom
Phaqui:fix-issue-31956
Closed

bpo-31956: Add start and stop parameters to array.index()#4435
Phaqui wants to merge 9 commits into
python:masterfrom
Phaqui:fix-issue-31956

Conversation

@Phaqui

@Phaqui Phaqui commented Nov 17, 2017

Copy link
Copy Markdown
Contributor

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just take list.index as an example.

Comment thread Modules/arraymodule.c Outdated

v: object
start: Py_ssize_t = 0
stop: Py_ssize_t = 9223372036854775807

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this 9223372036854775807 mean? I have idea, but this looks totally confusing.

And there is an integer overflow on 32-bit platform.

@Phaqui Phaqui Nov 17, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it somewhere (in cpython), though I can't remember where. I'll try to readjust with clinic, taking list_index() as an example, and commit the updates asap.

EDIT: Updated.

@bedevere-bot

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@Phaqui

Phaqui commented Nov 17, 2017

Copy link
Copy Markdown
Contributor Author

I have made the requested changes; please review again.

@bedevere-bot

Copy link
Copy Markdown

Thanks for making the requested changes!

@serhiy-storchaka: please review the changes made to this pull request.

Comment thread Modules/arraymodule.c Outdated
stop = len;
}

for (i = start; i < stop; i++) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An arbitrary Python code can be executed in PyObject_RichCompareBool(). It can change the size of the array. As result, the index can become out of the array limits.

Use the same loop condition as in list.index to prevent this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was able to reproduce this, using an object that contained a reference to the array, which in its __eq__ mutates the array, causing an assertion failure. At this point in time, my code has boiled down to an almost identical copy of list_index, except I do not see the reason for truncating the stop index, as is done there.

Comment thread Modules/arraymodule.c

for (i = 0; i < Py_SIZE(self); i++) {
len = Py_SIZE(self);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove redundant empty lines.

Comment thread Modules/arraymodule.c Outdated

if (stop < 0) {
stop += len;
} else if (stop > len) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"else if" should be on a separate line (PEP 7). But actually this branch is not needed due to the comment below.

Comment thread Doc/library/array.rst Outdated
*x* in the array.
*x* in the array. The optional arguments *start* and *stop* can be specified to
search for *x* within a subsection of the array, but the returned index is
still relative to the start of the array, not the subsection.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the words "but the returned index is still relative to the start of the array, not the subsection" are redundant since this directly follows from the first sentence.

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but somebody other should review the documentation.

Comment thread Doc/library/array.rst Outdated
Raises :exc:`ValueError` if *x* is not found.

.. versionchanged:: 3.7
Added optional start and stop parameters.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*start* and *stop*

@@ -0,0 +1,4 @@
array.index() now supports optional start and stop arguments.
As with general sequences, they can be negative, and the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this is redundant too. This is common behavior of index() methods.

…omatically confirm to the standard, no need to point that out
Comment thread Doc/library/array.rst


.. method:: array.index(x)
.. method:: array.index(x[, start[, stop]])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can document the default start value:

array.index(x, start=0[, stop])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to specify start=0, stop=sys.maxsize, but there were objections against this, because this looks like start and stop are keyword parameters. There were issues about changing the documentation back to [, start[, stop]].

Comment thread Modules/arraymodule.c

v: object
start: slice_index(accept={int}) = 0
stop: slice_index(accept={int}, c_default="PY_SSIZE_T_MAX") = sys.maxsize

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to not document the default value at Python level? Try to remove "= sys.maxsize".

Maybe you should put "/" (currently the following line) between start and stop?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to not make stop the keyword argument until other index() methods accept keyword arguments.

The signature of array.index() matches the signature of list.index() and this is the best that we can to do now. Due to limitations of pydoc, inspect and Argument Clinic the default value for stop is displayed as a concrete number, but this is a separate issue.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right, I didn't notice that other methods don't accept keywords.

>>> "abc".index("c", stop=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: index() takes no keyword arguments

I like the idea of accepting keyword arguments in other index() methods :-)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is an open issue for this. But first I want unify the code for all index() methods. Currently index() methods of str/bytes/bytearray use slightly different code.

Comment thread Modules/arraymodule.c

Return index of first occurrence of v in the array.

Raises ValueError if the value is not present.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we don't document the exact exception raised by a method. Please remove this addition.

Maybe we should start to document them, but I would prefer to have a discussion about that, before starting to document them.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstrings of other index() methods document this.

Comment thread Modules/arraymodule.c
start = 0;
}
if (stop < 0) {
stop += Py_SIZE(self);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is stop is still negative here?

Maybe you can use PySlice_AdjustIndices() here with step=1? What do you think @serhiy-storchaka?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All correct if stop is still negative. The following loop just will not be executed.

Comment thread Modules/arraymodule.c
if (stop < 0) {
stop += Py_SIZE(self);
}
for (i = start; i < stop && i < Py_SIZE(self); i++) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that you kept the "< Py_SIZE()" check if the array is mutated during the PyObject_RichCompareBool() call? If yes, please add a short comment to explain that, since it can be surprising.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested to do this. This is an exact copy from list.index(). Do you want to add a comment in list.index()?

@bedevere-bot

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@serhiy-storchaka serhiy-storchaka added the type-feature A feature request or enhancement label Feb 26, 2018
@vstinner

Copy link
Copy Markdown
Member

Ping @Phaqui: do you still plan to work on this PR? https://bugs.python.org/issue31956#msg350405 cc @nanjekyejoannah

@nanjekyejoannah nanjekyejoannah left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner @serhiy-storchaka , I think the author of this PR is waiting for consesus from either or both of you following the discussion from BPO here : https://bugs.python.org/issue31956#msg350405

@kamilturek

Copy link
Copy Markdown
Contributor

@Phaqui @vstinner @serhiy-storchaka
Is there any work planned to complete this PR? I am happy to take it over if necessary.

@Phaqui

Phaqui commented Mar 26, 2021

Copy link
Copy Markdown
Contributor Author

Sure, the code is mostly written. There were arguments about documentation, and details, and thus did not go in, but as far as I can remember, the code itself is pretty much list.index(), and should therefore in essence be ready to go.

@vstinner vstinner closed this May 3, 2021
@vstinner vstinner deleted the branch python:master May 3, 2021 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting changes type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants