Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove custom doubling strategy + add examples to VecAllocEx #9058

Merged
merged 3 commits into from
Jan 31, 2024

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jan 29, 2024

Which issue does this PR close?

Closes #9057

Rationale for this change

While writing documentation as suggested by @crepererum on #9025 (comment) I found that the accounting with push_accounted and allocated_size were different and inconsistent

What changes are included in this PR?

  1. Add examples + tests
  2. Remove custom doubling strategy (use the default rust Vec implementation)

Are these changes tested?

Yes, with doc tests

Are there any user-facing changes?

Allocation accounting is now different

/// // push more data into the vec
/// for _ in 0..10 { vec.push_accounted(1, &mut allocated); }
/// assert_eq!(allocated, 64); // underlying vec has space for 10 u32s
/// assert_eq!(vec.allocated_size(), 64);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocated size was different than what was obtained by push_accounted prior to this change


// growth factor: 2, but at least 2 elements
let bump_elements = (self.capacity() * 2).max(2);
let bump_size = std::mem::size_of::<u32>() * bump_elements;
Copy link
Contributor Author

@alamb alamb Jan 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be sizeof<T> rather than sizeof<u32> but perhaps I am missing something

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks correct. I suspect this was the cause of the discrepancy in the original code

@alamb alamb changed the title Add examples to VecAllocExt and remove custom doubling strategy Remove custom doubling strategy + add examples to VecAllocEx Jan 29, 2024
let new_capacity = self.capacity();
if new_capacity > prev_capacty {
// capacity changed, so we allocated more
let bump_size = (new_capacity - prev_capacty) * std::mem::size_of::<T>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it make sense to do a checked_mul here similar to the checked_add below?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either that or remove the checked_add because push would panic first now that it's called first instead of at the end

Copy link
Contributor

@erratic-pattern erratic-pattern Jan 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking about it more: if accounting is tracking multiple vecs, then this makes sense. multiplication should never overflow because push would panic first in that case, but the checked_add could potentially overflow since accounting could be some value greater than capacity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is a good point. I will add a comment to clarify

Copy link
Contributor Author

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review @kallisti-dev

let new_capacity = self.capacity();
if new_capacity > prev_capacty {
// capacity changed, so we allocated more
let bump_size = (new_capacity - prev_capacty) * std::mem::size_of::<T>();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is a good point. I will add a comment to clarify

Copy link
Contributor

@crepererum crepererum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

@alamb alamb merged commit f2ac2c6 into apache:main Jan 31, 2024
22 checks passed
@alamb alamb deleted the alamb/double branch January 31, 2024 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove custom doubling allocation strategy in
3 participants