Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade_check should check python_callable is not used in TriggerDagRunOperator #17899

Closed
2 tasks done
aran3 opened this issue Aug 29, 2021 · 2 comments
Closed
2 tasks done
Labels
kind:feature Feature Requests upgrade-check upgrade-check CLI won't fix

Comments

@aran3
Copy link
Contributor

aran3 commented Aug 29, 2021

Apache Airflow version

2.1.2

Operating System

Red Hat Enterprise Linux Server 7.6 (Maipo)

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

What happened

airflow upgrade_check should check python_callable is not used in TriggerDagRunOperator.
As of #6317, python_callable is no longer supported (in any 2.x version) and dag tasks using it will fail to parse.
This is not currently verified by upgrade_check and I caught it only after upgrading to 2.x.

What you expected to happen

upgrade_check to verify parameters of TriggerDagRunOperator and warn that upgrade_check should be removed and replaced with passing conf dictionary directly.

How to reproduce

run airflow upgrade_check on a dag with the following task (from PR):

def _trigger_task(context, object):
    object.payload = {"message": "Hello world"}
    return object

trigger_task = TriggerDagRunOperator(
    task_id='test_trigger_dagrun',
    trigger_dag_id="example_trigger_target_dag",
    python_callable=_trigger_task,
    dag=dag,
)

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@aran3 aran3 added area:core kind:bug This is a clearly a bug labels Aug 29, 2021
@boring-cyborg
Copy link

boring-cyborg bot commented Aug 29, 2021

Thanks for opening your first issue here! Be sure to follow the issue template!

@eladkal
Copy link
Contributor

eladkal commented Aug 29, 2021

Closing for two reasons:

  1. There are no further releases of upgrade check tool as 1.10 is EOL.
  2. In any case this is not something that can be added to the upgrade check as the conf param doesn't exist in TriggerDagRunOperator of 1.10 - It wasn't backported so this specific change is a breaking change. The upgrade checker will not suggest changes that results in broken dags on 1.10

@eladkal eladkal closed this as completed Aug 29, 2021
@eladkal eladkal added kind:feature Feature Requests upgrade-check upgrade-check CLI won't fix and removed kind:bug This is a clearly a bug area:core labels Aug 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:feature Feature Requests upgrade-check upgrade-check CLI won't fix
Projects
None yet
Development

No branches or pull requests

2 participants