Skip to content

fix: support list in document class (#1557)#1569

Merged
JoanFM merged 10 commits into
docarray:mainfrom
maxwelljin:fix-docarray-list
May 26, 2023
Merged

fix: support list in document class (#1557)#1569
JoanFM merged 10 commits into
docarray:mainfrom
maxwelljin:fix-docarray-list

Conversation

@maxwelljin

Copy link
Copy Markdown
Contributor

This pull request resolves the TypeError in BaseDocIndex._flatten_schema that occurs when a non-class type, such as List[str], is used as an argument for issubclass(). As observed in issue #1557, this causes the HnswDocumentIndex initialisation to crash.

As Johannes suggested, I use a custom issubclass function that can handle non-class types by returning False instead of failing. This change prevents crashes when a non-class type is passed as input.

@JoanFM JoanFM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, add tests showing the underlying issue is fixed

Comment thread docarray/index/abstract.py Outdated
@JoanFM JoanFM linked an issue May 24, 2023 that may be closed by this pull request
Comment thread docarray/index/abstract.py Outdated
##########################################################

@staticmethod
def custom_issubclass(x, A_tuple) -> bool:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this a function instead of a staticmethod, and let's more it to utils._internal._typing.
Maybe we can also rename it to safe_issubclass to indicate a bit more about what makes it custom?

Comment thread docarray/index/abstract.py Outdated
Comment thread docarray/index/abstract.py Outdated
Comment thread docarray/index/abstract.py Outdated
Traditional 'issubclass' calls can result in a crash if the input is list/tuple.

:param x: A class 'x'
:param A_tuple: A class, or a tuple of classes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make this argument lowercase, i.e. a_tuple

Comment thread docarray/index/abstract.py Outdated
Comment on lines +713 to +714
This is a modified version of the built-in 'issubclass' function to support non-class input.
Traditional 'issubclass' calls can result in a crash if the input is list/tuple.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice docstring!

@maxwelljin

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback! I have made the following changes:

I have added tests (docarray/tests/hnswlib/test_index_get_del.py) to demonstrate that the issue is fixed. These tests verify the functionality and ensure the problem is resolved.
I have converted the method to a function and moved it to utils._internal._typing, as suggested. Additionally, I renamed it to "safe_issubclass" to provide a clearer indication of its purpose.
I have also included support for other origins like "Dict" in addition to "list" and "tuple" and handled cases like "TypeVars" that are not classes but still need to be checked.

Once again, thank you for the valuable feedback

Comment thread docarray/utils/_internal/_typing.py Outdated
cls.__qualname__ = cls.__qualname__[: -len(cls.__name__)] + new_name
cls.__name__ = new_name

def safe_issubclass(x, a_tuple) -> bool:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add unittests specific to this method, also add proper type hinting

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for suggestions. I've commited an unit test to this method.

@maxwelljin maxwelljin force-pushed the fix-docarray-list branch from 59ba829 to 8a13e18 Compare May 26, 2023 07:10
@maxwelljin maxwelljin closed this May 26, 2023
@maxwelljin maxwelljin reopened this May 26, 2023
@JoanFM JoanFM requested review from JoanFM and JohannesMessner May 26, 2023 08:47
@JoanFM JoanFM merged commit 7a7a83a into docarray:main May 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not able to index when using List[str] in custom document class

3 participants