Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unsafe variable pointer #1691

Closed
wants to merge 2 commits into from
Closed

Unsafe variable pointer #1691

wants to merge 2 commits into from

Conversation

mejedi
Copy link
Contributor

@mejedi mejedi commented Feb 18, 2025

Enable (unsafe) direct access to mmap-ed variable memory. It makes scanning large data areas more efficient and enables advanced synchronisation patterns, such as atomics.

Introduce a new package so that a user has to explicitly opt in for unsafe features.

@mejedi mejedi requested a review from a team as a code owner February 18, 2025 11:46
@mejedi mejedi force-pushed the unsafe-variable-pointer branch from 22cdea2 to 1f2dad6 Compare February 18, 2025 12:14
Permit offset + size == len. Detect integer overflow in offset + size.

Signed-off-by: Nick Zavaritsky <[email protected]>
Enable (unsafe) direct access to mmap-ed variable memory. It makes
scanning large data areas more efficient and enables advanced
synchronisation patterns, such as atomics.

Introduce a new package so that a user has to explicitly opt in for
unsafe features.

Signed-off-by: Nick Zavaritsky <[email protected]>
@mejedi mejedi force-pushed the unsafe-variable-pointer branch from 1f2dad6 to 65932b7 Compare February 18, 2025 12:19
@ti-mo
Copy link
Collaborator

ti-mo commented Feb 18, 2025

My plan was to revive #1607 (the VariablePointer part) and put it behind a CollectionOptions flag. We're not going to commit to this sort of API.

Comment on lines -63 to +72
end := size - 1
end := size

qt.Assert(t, qt.IsTrue(mm.bounds(0, 0)))
qt.Assert(t, qt.IsTrue(mm.bounds(end, 0)))
qt.Assert(t, qt.IsFalse(mm.bounds(end, 0)))
qt.Assert(t, qt.IsTrue(mm.bounds(end-8, 8)))
qt.Assert(t, qt.IsTrue(mm.bounds(0, end)))

qt.Assert(t, qt.IsFalse(mm.bounds(end-8, 9)))
qt.Assert(t, qt.IsFalse(mm.bounds(end, 1)))
qt.Assert(t, qt.IsFalse(mm.bounds(0, size)))
qt.Assert(t, qt.IsTrue(mm.bounds(0, size)))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I fail to understand something? Especially qt.IsFalse(mm.bounds(0, size))).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix posted in #1694.

@mejedi
Copy link
Contributor Author

mejedi commented Feb 18, 2025

@ti-mo

My plan was to revive #1607 (the VariablePointer part) and put it behind a CollectionOptions flag. We're not going to commit to this sort of API.

To double-check we are on the same page: did you plan to add it before the proposal gets through? Even if it gets accepted it is taking at least another two years before the feature becomes available in Golang and cilium/ebpf bumps its minimum requirement.

If the collection option is omitted, does the .Pointer() fail?

IMO it is cleaner to have a separate package than to introduce a method that fails by default. What if the proposal is rejected? Do we rip out the .Pointer() and the collection option?

The unsafe.VariablePointer() does not promise any semantics beyond what the current golang can deliver. When a better solution becomes available, we can deprecate it.

@lmb ?

@ti-mo
Copy link
Collaborator

ti-mo commented Feb 20, 2025

@mejedi To confirm what we discussed offline: the proposal has gotten positive traction, and mknyszek told me the feature has a decent chance of making it into the runtime. I'll push to get it formally accepted, it needs a few edits. Then, it's a matter of getting it implemented.

nack on adding an unsafe package. Let's go forward with the VariablePointer proposal in my other PR and put it behind a CollectionOptions.UnsafeVariableExperiment flag. Leaving the option to false will use the existing safe implementation and make VariablePointer return an error. MemoryPointer will remain unexported for now, I want to limit the experiment to variables only.

@ti-mo ti-mo closed this Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants