Update package metadata

[ofek]

Background

Hello there! The Python packaging ecosystem has standardized on the interface for build backends (PEP 517/PEP 660) and the format for metadata declaration (PEP 621/PEP 631). As a result, the use of setup.py files is now heavily discouraged.

So, I'm spending my free time updating important projects so that they are modernized and set an example for others 😄

Summary of changes

This implements PEP 621, obviating the need for setup.py, setup.cfg, and MANIFEST.in. Support has not landed in setuptools, so builds will now use hatchling. It is quite stable and actively maintained, the only reason the version is 0.x is because 1.0.0 will drop support for Python 2. It's also on the major distribution channels such as conda-forge, Debian, Fedora, etc.

I've done this for such projects as pipx, all Datadog Agent integrations, etc.

Notes

• This fixes the .gitignore file to ignore the entire intended directories, see https://git-scm.com/docs/gitignore#_pattern_format
• The hatch_build.py can be removed if you want things to be more static

[facelessuser]

I appreciate you talking the time to do this. I will be evaluating this. I apologize though if it takes a bit of time. Switching the build backend is no a small decision as I'll have to be familiar enough to maintain it, and whatever is chosen will most likely be chosen to use across all my packages.

I see you are the author of hatch, I at least know where to go to ask questions 🙂. I may have some mildly more complicated questions as the project that has the most complicated setup is https://github.com/facelessuser/Rummage. If I can't migrate that one, then migrating this one may wait.

I'll be taking some time to run through this and let you know my final decision.

[facelessuser]

NOTE TO SELF: not sure if the includes are all correct. Will need to double-check that we are including what we expect in the sdist and not what we don't.

[facelessuser]

Okay, so quick question. Assuming I want to run an arbitrary pre-processor step. Like I need to generate some files before the build begins. For example, I have a package Backrefs that needs to generate a compatible Unicodedata table that corresponds with the current Python version. It'll generate the necessary source code, then I will build the package.

I assume I need to use the initialize build hook: https://ofek.dev/hatch/latest/plugins/build-hook/#hatchling.builders.hooks.plugin.interface.BuildHookInterface.initialize?

[ofek]

Yup exactly!

from hatchling.builders.hooks.plugin.interface import BuildHookInterface

class CustomBuildHook(BuildHookInterface):
    def initialize(self, version, build_data):
        ...

[facelessuser]

I think the only other thing I am uncertain of is whether setup_required is still supported. I'm just going to have to figure out how to integrate babel support for Rummage...though I don't know how much that functionality is actually used. I had one guy who added the necessary translation files for Russian, but I think they are out of date, and no one has ever complained 🙃. Regardless, I'd like to at least not abandon the functionality.

I may take a stab at backrefs to make sure I understand the system. And then try and go as far as I can on Rummage. If I can successfully work through the needed requirements for those two packages, I think hatch will fulfill my general needs in regards to all the packages I offer through PyPI.

[ofek]

Those deps would go in either build-system.requires:

[build-system]
requires = [
    "babel",
    "hatchling>=0.21.0",
]
build-backend = "hatchling.build"

or defined as a dependency of your hook:

[tool.hatch.build.hooks.custom]
dependencies = [
  "babel",
]

The latter is preferred so disabling the hook will not install its dependencies in the build environment.

[facelessuser]

Cool, makes sense. That leaves me with figuring out how to actually call babel. Hopefully, integration is straight forward. You've been a lot of help. I'll let you know if I get stuck. I may be able to carve out some time today to dig in today. This is looking promising.

[ofek]

Hmm, looks like babel installs some kind of distutils-specific logic that Rummage calls when building wheels:

• https://github.com/facelessuser/Rummage/blob/129ac8cb226afce5fb0915c0b81e752ab510dc58/setup.py#L18
• https://github.com/python-babel/babel/blob/bc7a51bc8809f9d62dca43a634d3b803dc054cf0/setup.py#L83-L84
• https://github.com/python-babel/babel/blob/bc7a51bc8809f9d62dca43a634d3b803dc054cf0/babel/messages/frontend.py#L123

I'm assuming something like:

import os

from hatchling.builders.hooks.plugin.interface import BuildHookInterface

class CustomBuildHook(BuildHookInterface):
    def initialize(self, version, build_data):
        if self.target_name != 'wheel':
            return

        # Lazily import in case hook is disabled but file is still loaded
        from babel.messages.frontend import compile_catalog

        compiled_catalog = compile_catalog()
        compiled_catalog.directory = os.path.join(self.root, 'rummage', 'lib', 'gui', 'localization', 'locale')

        compiled_catalog.finalize_options()
        compiled_catalog.run()

Note that the if branch can be removed if you define this as a target-specific hook so it only runs for wheels:

[tool.hatch.build.targets.wheel.hooks.custom]
dependencies = [
  "babel",
]

[facelessuser]

Another unclear point. zip_safe=False. How do you differentiate with Hatch? Also, I'm not entirely clear on the replacement for include_package_data = True. Is that the artifacts section?

[ofek]

zip_safe=False

That's a legacy setuptools thing, see python/mypy#8802 (comment)

replacement for include_package_data = True

All file extensions are included by default, not just .py. This is generally what users expect and matches the behavior of other modern build backends.

[facelessuser]

I read the comment (python/mypy#8802 (comment)), what I am unclear of is what this means. Does this mean that packages are not zipped by default anymore?

[facelessuser]

Looks like babel is going to be a pain to get working...

[facelessuser]

never mind, think I got it

[ofek]

I read the comment (python/mypy#8802 (comment)), what I am unclear of is what this means. Does this mean that packages are not zipped by default anymore?

I think it's about the old .egg format.

[facelessuser]

I think the metadata plugin isn't running. I'm doing the same thing in Rummage, and the PKG-INFO has no classifiers. I even tried to throw an exception, and nothing broke 😕.

[facelessuser]

I think this change needs to define [tool.hatch.metadata.hooks.<PLUGIN_NAME>] in the toml to get the metadata hook to run

[ofek]

Whoops, fixed!

[facelessuser]

I think Hatch can do everything I need it to do. I was able to convert the most cumbersome project, so if I can manage that, I think I understand things well enough to adopt in all of the easier projects.

[facelessuser]

Closing this in favor of #1644 which builds upon this PR and cleans up some stuff