Airflow ClickHouse Plugin v1.2.0 is not available on PyPI

[bryzgaloff]

A publishing pipeline failed: https://github.com/bryzgaloff/airflow-clickhouse-plugin/actions/runs/7716663176

It requires several adjustments:

• Version of installed Airflow should follow the tag, currently it is hardcoded to 1.1.0.
• Constraint file should be specified when installing the plugin.
• Bonus: add instructions to the README about the constraint file, referencing Airflow docs: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-tools

[bryzgaloff]

As a quick fix, you may install it from Test PyPI: the package has already been published there. Use python -m pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple airflow-clickhouse-plugin[common.sql]==1.2.0 to install.

[epikhinm]

Hi @bryzgaloff !
Do you have any plans here? Do you want to release a new version and skip 1.2.0?
Do you need any help with releasing 1.2.0?

[kakvablyat]

+1

[bryzgaloff]

Hi everyone, and thank you for waiting for the issue resolution.

There is a problem in the PyPI publishing workflow, and I have not had much time recently to fix this CI pipeline. I am in the process of fixing it. I hope to complete by the end of this week. I have tons of emails and messages, so I am prioritizing this task much higher now.

Here is a quick summary of what has to be fixed:

• The pipeline has a hardcoded version 1.1.0. But it has to be a pushed git tag.
• The pipeline should fail if a tag does not follow semver format (1.2.0, not v1.2.0). Maybe PyPI performs the check when a package is uploaded and rejects wrong versioning, but I need to check it on a test upload first.
• Airflow for PyPI packages tests should be installed with --constraint: otherwise, tests are likely to fail.
• Ideally, the PyPI tests should be run with all supported versions combinations, but I have not made my final decision if it is required.
• A preliminary local test should be run before uploading to PyPI.

These steps are intended to avoid publishing a broken package version to PyPI (which is an irreversible action: packages cannot be overwritten even in Test PyPI). A special challenge is that the pipeline itself is not runnable if the package version is already uploaded, which makes testing complicated.

[bryzgaloff]

Hello everyone! I'm pleased to announce that airflow-clickhouse-plugin==1.2.0 is now available on PyPI: airflow-clickhouse-plugin/1.2.0 🎉

I've applied a quick fix to address the failed workflow steps, ensuring that you don't need to wait any longer for this release. The publishing pipeline has been successfully executed. So, this version has been tested and is fully functional.

I'm closing this issue now as its main objective has been achieved. However, please note that this was a draft version of the pipeline. I'll continue my work in pypi-workflow-fix-#75 branch to implement proper fixes and conduct further testing.

Excitingly, v1.2.0 has been officially published! Happy coding! 😊

Thank you everyone for your patience kindly waiting for the fixes to be applied 🙏