Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor way of working with RQ meta #9082

Open
wants to merge 39 commits into
base: develop
Choose a base branch
from

Conversation

Marishka17
Copy link
Contributor

@Marishka17 Marishka17 commented Feb 10, 2025

Motivation and context

This PR changes the way we work with RQ meta. The previous implementation tried to standardize the fields saved in the meta and how to access them, however, we had to remember the structure of nested objects. This approach explicitly defines the structure of the meta being used.
Extracted from #9075
Partially depends on #9077

How has this been tested?

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@cvat-ai cvat-ai deleted a comment from github-actions bot Feb 10, 2025
Copy link
Contributor

github-actions bot commented Feb 10, 2025

❌ Some checks failed
📄 See logs here

Copy link
Contributor

github-actions bot commented Feb 10, 2025

❌ Some checks failed
📄 See logs here

@Marishka17 Marishka17 changed the title [WIP] Refactor way of working with RQ meta [do not merge] Refactor way of working with RQ meta Feb 10, 2025
@codecov-commenter
Copy link

codecov-commenter commented Feb 12, 2025

Codecov Report

Attention: Patch coverage is 77.97468% with 87 lines in your changes missing coverage. Please review.

Project coverage is 73.23%. Comparing base (de42098) to head (9b83548).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9082      +/-   ##
===========================================
+ Coverage    73.18%   73.23%   +0.05%     
===========================================
  Files          446      447       +1     
  Lines        45661    45833     +172     
  Branches      3907     3907              
===========================================
+ Hits         33418    33568     +150     
- Misses       12243    12265      +22     
Components Coverage Δ
cvat-ui 77.07% <ø> (-0.02%) ⬇️
cvat-server 70.17% <77.97%> (+0.12%) ⬆️

@Marishka17 Marishka17 changed the title [do not merge] Refactor way of working with RQ meta Refactor way of working with RQ meta Feb 12, 2025
@Marishka17 Marishka17 force-pushed the mk/refactor_working_with_rq_meta branch from 3348a62 to 7ea0523 Compare February 14, 2025 16:02
@Marishka17 Marishka17 requested a review from nmanovic as a code owner February 17, 2025 10:53
@Marishka17 Marishka17 changed the title Refactor way of working with RQ meta [WIP] Refactor way of working with RQ meta Feb 19, 2025
@cvat-ai cvat-ai deleted a comment from github-actions bot Feb 21, 2025
@Marishka17 Marishka17 force-pushed the mk/refactor_working_with_rq_meta branch from b99d1b2 to 9ce85ac Compare February 21, 2025 11:31
@Marishka17 Marishka17 changed the title [WIP] Refactor way of working with RQ meta Refactor way of working with RQ meta Feb 21, 2025
Comment on lines +136 to +137
def to_dict(self):
return self.meta
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these redundant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I also think so, but I made this way because:

  1. ImmutableRQMetaAttribute expects objects with a meta attribute and I would not prefer to use obj.to_dict()[SomeField]
  2. in some places we need to have the "dict" representation of an user or request (BaseRQMeta.for_job(rq_job).user.to_dict()). In such cases, I also don't like calling BaseRQMeta.for_job(rq_job).user.meta because it is less informative.
    So, I kept to_dict only for UserMeta and RequestMeta.


@classmethod
def for_job(cls, job: RQJob):
return cls(job=job)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return cls(job=job)
return cls(job=job, meta=job.meta)

This would simplify the rest of the class: no need for the if in def meta, no need for the assert in __init__.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for the assert in init

frankly speaking, we must check that job.meta and meta refer to the same object in that case


user = request.user

return cls.for_meta(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the class redundant here? You're creating a dictionary, wrapping it, then immediately unwrapping it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, right now there is no reason to do so. In the previous implementation, I thought it would be better to check that the generated meta matches the expected structure.

self, *, job: RQJob | None = None, meta: dict[RQJobMetaField, Any] | None = None
) -> None:
assert (job and not meta) or (meta and not job), "Only job or meta can be passed"
def __init__(self, *, job: RQJob | None = None, meta: dict[str, Any] | None = None) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def __init__(self, *, job: RQJob | None = None, meta: dict[str, Any] | None = None) -> None:
def __init__(self, *, job: RQJob | None = None, meta: dict[str, Any]) -> None:

Since you've updated for_job, meta is always passed, so you don't have to support it being missing anymore.

@Marishka17 Marishka17 force-pushed the mk/refactor_working_with_rq_meta branch from f10eff1 to 227c877 Compare February 24, 2025 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants