Possibility to mark dependencies as dev-only that don’t go into package metadata

[hynek]

I like using optional dependencies like tests or docs for local development.

However, it has the annoying side-effect of putting that into package metadata which make it look like attrs depends on some wild stuff.

It would be nice to have dev-only dependency groups and my recent foray into PDM showed me there’s precedent for that: https://pdm.fming.dev/latest/usage/dependency/#add-development-only-dependencies

Syntax is the same, just different key:

[tool.pdm.dev-dependencies]
tests = ["pytest"]

FWIW, I’d be perfectly content with something like this which is less standard-bending:

[tool.hatch]
dev-only-groups = ["tests", "dev", "docs"]

---

I realize there’s certain overlap with hatch environments, but I don’t want to break the Python-standard ways of installing packages. At least for now (fact aside that I can’t use them for now :))

[funkyfuture]

what about a subcommand that let's you install the dependencies of a given environment into the currently active environment? i assume that you want to work with these packages in an interactive shell. b/c everything else can be done with environments as they are designed and implemented, is my impression.

[gwerbin]

What's the reason for avoiding Hatch environments? If someone else is working on your project, they'll need Hatch on their system one way or another, so I don't see the benefit in avoiding them in favor of a dev optional dependency group.

A dev-only optional dependency group seems more "standard-bending" than just using a Hatch environment, because the former actually changes the behavior of a standard feature, while the latter is a different thing altogether. The PDM dev-only dependencies are morally identical to (and less flexible than) having extra dependencies in a Hatch environment.

It is reasonable to offer tests and docs dependency groups in your published package, because downstream users might want to run tests and build docs without setting up a full dev environment. I also don't see much concern with these entering the Wheel metadata, as they explicitly are listed with an extra == condition.

[jamesdow21]

Official standard for this is PEP 735 – Dependency Groups in pyproject.toml, which was accepted on October 10, 2024

[funkyfuture]

okay, so beside an implementation, hatch needs to define how dependency groups can be referenced in for environments. i'm proposing two variants to get the discussion started.

one proposal would allow inclusion of references that are marked with an @ prefix in the mentioned list. eg:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
  "mypy",
  "@stubs"
]

the only rationale for the @ prefix is its frequent availability on various keyboard layouts.

the other proposal introduces a new key for that table:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
  "mypy",
]
dependency-groups = [
  "stubs"
]

of these two i personally prefer the latter because it tells readers directly what is being referenced.

[hartungstenio]

okay, so beside an implementation, hatch needs to define how dependency groups can be referenced in for environments. i'm proposing two variants to get the discussion started.

one proposal would allow inclusion of references that are marked with an @ prefix in the mentioned list. eg:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
"mypy",
"@stubs"
]
the only rationale for the @ prefix is its frequent availability on various keyboard layouts.

the other proposal introduces a new key for that table:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
"mypy",
]
dependency-groups = [
"stubs"
]
of these two i personally prefer the latter because it tells readers directly what is being referenced.

PEP-735 specifies how to include a dependency group in another dependency group. We could use the same syntax:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
  "mypy",
  {include-group = "stubs"},
]

[funkyfuture]

i like the lexical re-use, but there's no need for nesting in this case, thus:

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
  "mypy",
  
]
include-group = [ 
    "stubs",
]

but then the singular form is unfortunate.

[jimisola]

Either syntax works. But, is there not a common standard for the pyproject.toml or it depends on the build / package tool? Are there any pros to using the PEP735 or cons not using it?

That said, having the functionality in hatch is more important.

[jamesdow21]

Closest analogy that is currently supported by Hatch would be features, which allows an environment to specify which of the project's optional dependencies/extras should be installed, which works like

[project.optional-dependencies]
foobar = ["foo", "bar >= 2.0"]

[tool.hatch.envs.example]
features = ["foobar"]

Following that precedent would look like @funkyfuture's preferred syntax (with the caveat that there could be some bikeshedding over what name to use as the key in the table)

Since the value should be an array of strings, I would say that the key should be plural, and it seems like in general nouns are preferred over verbs.

The simplest answer to me seems to be to just call it dependency-groups, which then matches @funkyfuture's original second proposal

[dependency-groups]
stubs = ["types-lxml", "types-beautifulsoup4"]

[tool.hatch.envs.mypy]
dependencies = [
  "mypy",
]
dependency-groups = [
  "stubs"
]

[funkyfuture]

that also aligns well with the features key. imo this discussion suffices to start an implementation, b/c the structure is agreed upon. and whether it's dependency-groups or include-groups can easily be adjusted before it is finally included.

Either syntax works.

yes, that's the general linguistic understanding.

But, is there not a common standard for the pyproject.toml or it depends on the build / package tool? Are there any pros to using the PEP735 or cons not using it?

the standard tells us how these groupings are to be defined, not how they should be referred to in hatch-specific configuration data.

That said, having the functionality in hatch is more important.

that is false in its division of design and implementation:

1. there are two difficult things in programming … one of them is naming things and this aspect of design is essential for an intuitive user experience.
2. you wanna define these things once for the rest of time, b/c renaming things that turned out to be suboptimal in later releases is just a hassle for maintainers and users.

[jimisola]

But, is there not a common standard for the pyproject.toml or it depends on the build / package tool? Are there any pros to using the PEP735 or cons not using it?

the standard tells us how these groupings are to be defined, not how they should be referred to in hatch-specific configuration data.

Ok. The fragmention of pyproject.toml seems unnecessary, but if only the functionality and not the syntax is defined in the standard then that's it.

That said, having the functionality in hatch is more important.

that is false in its division of design and implementation:

1. there are two difficult things in programming … one of them is naming things and this aspect of design is essential for an intuitive user experience.
2. you wanna define these things once for the rest of time, b/c renaming things that turned out to be suboptimal in later releases is just a hassle for maintainers and users.

I think you misinterpreted me here. I can personally be a pedantic (or fanatic or whatever one wants to call it) about naming things for the reasons that you mentioned among other things. But, the functionality is more important than naming - not saying that naming is not important.

Why? Let's take it to the extreme and play devil's advocate, the dwelling on naming will continue for all eternity what do you think that the user experience will be then?

[Midnighter]

I'm having a hard time reading the state of this feature. Is there something that can be contributed to assist specification or implementation? Or is it rather something you want to tackle in core devs but it's not at the top of the list at the moment?

[funkyfuture]

i wouldn't know of other core devs than ofek. myself has other priorities.