Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(workflow): workflow revert command #2956

Merged
merged 8 commits into from
Jun 29, 2022

Conversation

mohammad-alisafaee
Copy link
Contributor

@mohammad-alisafaee mohammad-alisafaee commented Jun 17, 2022

Description

  • renku workflow revert command. There are some deviations from the story definition:

    1. --soft is renamed to --metadata-only
    2. --plans is renamed to --plan
    3. The story say that generations should be changed if the activity is the newest execution and has some downstream activities. The implementation always reverts the generations to an earlier version (if any).
    4. ActivityGateway.get_all_activities won't return deleted activities by default.
    5. When an activity has downstream activities, it doesn't prompt to ask if it should revert the activity. Instead, it always fails and asks users to pass --force in such cases.
  • When deleting a plan, it creates a derivative plan and marks it as deleted.

Fixes #2342
Fixes #2969

NOTE

We don't detect files that are used in datasets and might still delete them if they are not used in a workflow.

@mohammad-alisafaee mohammad-alisafaee marked this pull request as ready for review June 22, 2022 07:08
@mohammad-alisafaee mohammad-alisafaee requested a review from a team as a code owner June 22, 2022 07:08
Copy link
Member

@Panaetius Panaetius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing how quick this was done, great work!

We should also have someone from ORDES review this (with some instructions) once we settled on a stable state.

communication.confirm(prompt_text, abort=True, warning=True)

for derivative_plan in get_derivative_chain(plan):
derivative_plan.delete(when=when)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a break from how we do deletions on datasets, where we just add a new derivation that has invalidated_at set.

I wonder if we should bring these in line. Also not sure how it affects KG if they get modified Plan nodes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already mark all activities in the activity chain as deleted. If that's an issue for KG then we shouldn't do that and #2954 cannot be done. I'll talk to KG about this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I added renku workflow remove, the idea way that a Plan chain ends in a node that is deleted, it means all versions are deleted. I think the filtering out of deleted plans in other commands was don't in parallel in a separate PR and only the nodes that are invalidated are ignored (but not their parents). I think the latter is wrong and we should treat it the same as datasets (an invalidation invalidates the whole chain).

For activities it's fine since they can't be edited (just removed).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've talked to Kuba about this and it's OK to set a property that didn't have a value before (prov:wasInvalidatedAt in this case). This may affect project's fork if deletion was done after the fork (all forked projects see the plan as deleted) but they will fix this in future by name-spacing forks since this happens for all other properties as well.

@mohammad-alisafaee mohammad-alisafaee force-pushed the 2342-activity-revert branch 3 times, most recently from aa17dd5 to 86abbeb Compare June 27, 2022 15:57
Copy link
Member

@Panaetius Panaetius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor change, otherwise looks great 👍

@mohammad-alisafaee mohammad-alisafaee enabled auto-merge (squash) June 29, 2022 09:45
@mohammad-alisafaee mohammad-alisafaee merged commit cb0e73d into develop Jun 29, 2022
@mohammad-alisafaee mohammad-alisafaee deleted the 2342-activity-revert branch June 29, 2022 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a final derivative when deleting workflows add renku revert activity command
3 participants