Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storr::encode64() overhead #572

Closed
wlandau opened this issue Nov 4, 2018 · 6 comments
Closed

storr::encode64() overhead #572

wlandau opened this issue Nov 4, 2018 · 6 comments

Comments

@wlandau
Copy link
Member

wlandau commented Nov 4, 2018

#571 + profvis helped me see this.

screenshot_20181103_232309

See also richfitz/storr#44.

A few ideas:

  1. A documentation-only fix: recommend a custom cache from storr::storr_rds(mangle_key = FALSE).
  2. Use storr_rds(mangle_key = FALSE) by default for new caches and just mangle the file names.
  3. Migrate to a specialized caching system for drake using times stamps and no key mangling.
@wlandau
Copy link
Member Author

wlandau commented Nov 4, 2018

I think the best way to solve this is through richfitz/storr#88. We could use mangle_key = "selective" in new_cache() and is_default_cache() and handle back compatibility in darke_fetch_rds(). Simple on drake's end.

@wlandau
Copy link
Member Author

wlandau commented Nov 4, 2018

If richfitz/storr#89 does not work out, we could change the default mangle_key to FALSE and use custom wrappers to mangle our own keys manually. As always, back compatibility is tricky.

@wlandau wlandau changed the title Bottleneck: storr::encode64() Work around storr::encode64() Nov 4, 2018
@wlandau wlandau changed the title Work around storr::encode64() storr::encode64() overhead Nov 4, 2018
@wlandau
Copy link
Member Author

wlandau commented Nov 5, 2018

For the custom wrappers, make.names() would work fine for drake's purposes, and it is likely to be much faster than any key mangling done by storr.

@wlandau
Copy link
Member Author

wlandau commented Nov 6, 2018

Hopefully we can handle this through richfitz/storr#90.

@wlandau
Copy link
Member Author

wlandau commented Dec 21, 2018

Fixed via #633.

@wlandau wlandau closed this as completed Dec 21, 2018
@wlandau
Copy link
Member Author

wlandau commented Dec 21, 2018

See also #630.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant