Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type annotations for hnswlib #13529

Merged
merged 4 commits into from
Feb 23, 2025
Merged

Add type annotations for hnswlib #13529

merged 4 commits into from
Feb 23, 2025

Conversation

lbhm
Copy link
Contributor

@lbhm lbhm commented Feb 23, 2025

This PR adds type annotations for hnswlib, a popular library for nearest neighbor search in vector spaces.

As this is my first time contributing to typeshed, I'd be happy to receive feedback on a couple of aspects:

  • stubtest raises the error hnswlib.Index is inconsistent, metaclass differs and reports pybind11_builtins.pybind11_type as the metaclass. However, since pybind11 is not a runtime dependency of hnswlib, I understand that I'm not allowed to import it for type annotations. What is the best practice to handle this?
  • pyright reports Variable not allowed in type expression whenever I use NDArray from Numpy as a type annotation. Numpy is a dependency of hsnwlib so I understand that I can use it in type annotations. How can I fix this?

This comment has been minimized.

This comment has been minimized.

@srittau
Copy link
Collaborator

srittau commented Feb 23, 2025

Not a full review, but some remarks:

  • You used object as type annotation in some places. This means that a function or methods will accept literally any object (like the built-in str() function does). This is very uncommon. If unsure about the correct type hints, you can leave them unannotated for now – or use Incomplete if that isn't possible.
  • Regarding the stubtest error: It's probably best to add a @tests/stubtest_allowlist.txt file and add hnswlib.Index to it, with a comment explaining the situation. See other stubtest_allowlists in this repository.

@lbhm
Copy link
Contributor Author

lbhm commented Feb 23, 2025

Thank you for the quick response @srittau. I added a stubtest allowlist as you suggested.

As for the usage of object, I followed the contributor guideline of annotating the implementation as it is here. The relevant methods of hnswlib.Index accept py::object as input (e.g., here). Internally, the input is subsequently cast into a Numpy array, so practically, the method would accept anything that can be cast into a Numpy array. I was not sure if there is a proper type alias for this. I am not a pybind11 expert myself, so I didn't want to narrow down the input type too much. Is there a best practice for such a situation?

This comment has been minimized.

@srittau
Copy link
Collaborator

srittau commented Feb 23, 2025

The easiest for now is using Incomplete (an alias of Any) or leaving the argument unannotated. This is a clear sign that this needs more work and is basically identical to using object in argument positions.

@lbhm
Copy link
Contributor Author

lbhm commented Feb 23, 2025

Alright, I replaced object with Incomplete and left some arguments unannotated according to the rules of flake8-pyi.

Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@srittau srittau merged commit 9135645 into python:main Feb 23, 2025
43 checks passed
@lbhm lbhm deleted the types-hnswlib branch February 23, 2025 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants