Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvement for expr slots using Mapping-derived class for bindings #28

Merged
merged 4 commits into from
May 8, 2024

Conversation

martinwellman
Copy link
Collaborator

This PR deals with issue #27, and addresses both points in that issue. It addresses performance issues with expr slots in a slot_derivation. The first point in the issue was that bindings are recalculated any time an expr slot is found, and the recalculation occurs even if the bindings are the same. The second point was that when the bindings are calculated, ALL bindings are calculated, whether that binding is used or not. To address this, I created a Bindings class in object_transformer.py that calculates a binding only when requested with the get method. This class is derived from the Mappings class which behaves like a regular dictionary. The code for calculating the bindings is the same as the code that was previously found in ObjectTransformer.map_object() for expr slots, it has been placed in Bindings.get_ctxt_obj_and_dict() and the resulting bindings are cached in Bindings.bindings. Bindings.get_ctxt_obj_and_dict() is called from Bindings.getitem, using only the item requested. Also, eval_utils.eval_expr_with_mapping() has been added, which is the same as eval_expr but a Mapping object is used as the bindings parameter, instead of **kwargs (eval_expr has also been changed to simply call eval_expr_with_mapping).

One problem with this is that the cache keys used by ObjectIndex will be the same for the various different objects passed to ObjectIndex.bless, even if different dictionary keys are passed in from the source object. This means a possibly incorrect cached ProxyObject might be returned. To deal with this the cache is cleared by calling object_index.clear_proxy_object_cache(). This is a bit messy so suggestions for cleaning this up are welcome.

An important thing to keep in mind is that if an unrestricted evaluation is required for executing the code from an expr slot, then ALL bindings/variables are required to execute the code with asteval.Interpreter. This can be found in the except block where bindings.get_ctxt_obj_and_dict() is called (with None as a parameter to specify that ALL bindings are requested).

A simpler solution that is a bit cleaner (but not as good performance-wise) is found in expr-opt-b.

Fixes #27

@martinwellman
Copy link
Collaborator Author

Resolved original conflict with branch main. Should be ready to be merged. @cmungall

@martinwellman martinwellman merged commit 2fe7d8e into main May 8, 2024
5 checks passed
@martinwellman martinwellman deleted the expr-opt-a branch May 8, 2024 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance for calculating bindings used by expr slot derivations
2 participants