Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PySlice::new converts isize to c_long which is incorrect on Windows #2768

Closed
Narsil opened this issue Nov 22, 2022 · 1 comment
Closed

PySlice::new converts isize to c_long which is incorrect on Windows #2768

Narsil opened this issue Nov 22, 2022 · 1 comment
Labels

Comments

@Narsil
Copy link

Narsil commented Nov 22, 2022

Bug Description

When trying to take a large slice with PySlice::new(start, stop, step) one, can overflow and get back an erroneous answer on Windows platform.

Core of the issue seems to be here:

https://docs.rs/pyo3/latest/src/pyo3/types/slice.rs.html#43-52

https://doc.rust-lang.org/stable/std/ffi/type.c_long.html Which is said to be i32 on Windows platform.

It seems using c_longlong https://doc.rust-lang.org/stable/std/ffi/type.c_longlong.html should fix the issue along with pyo3::ffi::PyLong_FromLongLong(start as std::ffi::c_longlong) which are defined to be at least i64 on most systems (including Windows x64bit where isize is i64 and c_long is not, which is the core of this issue)

Steps to Reproduce

Create a slice with either value being over i32::MAX. (it will silently overflow to negative values).

Backtrace

No response

Your operating system and version

Windows (any afaik)

Your Python version (python --version)

Any (afaik)

Your Rust version (rustc --version)

1.65 (but any)

Your PyO3 version

0.17.2

How did you install python? Did you use a virtualenv?

pyenv but all are affected afaik

Additional Info

Original report: huggingface/safetensors#95

Temporary fix in the lib itself huggingface/safetensors#99

Happy to contribute a PR if wanted.

@davidhewitt for visibility.

@Narsil Narsil added the bug label Nov 22, 2022
@birkenfeld
Copy link
Member

PyLong_FromSsize_t should be used; isize is what Python calls Py_ssize_t.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants