Skip to content
This repository has been archived by the owner on Sep 11, 2023. It is now read-only.

[util] improve performance of type check #1440

Merged
merged 1 commit into from
Nov 20, 2019

Conversation

clonker
Copy link
Member

@clonker clonker commented Nov 19, 2019

Small change to improve performance of check whether is something a list of int by a factor of roughly 20:
Old implementation on list of 1000000 random ints:

861 ms ± 9.3 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

New implementation on list of 1000000 random ints:

48.3 ms ± 932 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Fixes #1439

@marscher
Copy link
Member

That's brilliant, did you come up with the groupby type idea? 👍

@marscher marscher merged commit fea443f into markovmodel:devel Nov 20, 2019
@marscher
Copy link
Member

the failed MSM tests are due to a change of version in openblas, I have reported it on Conda-forge (since it is actually on an increase in build number, not source code version) conda-forge/blas-feedstock#40

@clonker
Copy link
Member Author

clonker commented Nov 20, 2019

What can I say, cosmic radiation! 🤷‍♂️

@marscher
Copy link
Member

The speed penalty of the old impl has to arise from the boolean and operations per element, not the type check. Since groupby reduces the boolean operations to the amount of types in the list, it is blazingly fast ⚡

@clonker clonker deleted the typecheck_performance branch February 25, 2021 08:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MSM estimation 10x slower when dtraj is a list vs array
2 participants