Docs: clarify handling of Python references while Rust holds the GIL

[birkenfeld]

The relationship between periods holding the GIL and object reference lifetime is not clearly described in the library documentation, leading to bugs like #1547 and #1556. The guide makes a mention in "Advanced topics", but this reads like something that isn't important to know, calling it "invisible".

There should be at least mention in the library reference (maybe just on the topmost level in "Ownership and Lifetimes"?) and also in the guide in a more introductory section.

[davidhewitt]

Agreed that better documentation is needed here. I'd held off because I kept thinking #1308 was just around the corner, but then I got sidetracked addressing lots of other bugfixes and cleanups.

The worst part of the current system, in my opinion, is that to make it safe PyO3 can only clear the pool on #[pyfunction] return, or when the Python GIL is actually dropped. (It used to be cleared when every GILGuard was dropped, but then it was super trivial to create use-after-free, see #864.) This leads to the extremely long held-object lifetimes which trip users up in #1547 and #1556.

[birkenfeld]

Right, that means that the advice in https://pyo3.rs/v0.13.2/advanced.html isn't actually accurate anymore?

[davidhewitt]

Closing as the Bound API will replace the GIL Refs discussed here.