Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv: add meta.repository field #294347

Merged
1 commit merged into from
Mar 29, 2024
Merged

Conversation

lolbinarycat
Copy link
Contributor

Description of changes

closes #293838

the new meta.repository will, by default (i.e. unless it is specified manually by the package), pull its value from src.meta.homepage, which is set by fetchFromGitHub and other fetchFrom* functions. if this cannot be found, it will be set to null.

rationale for name: chosen to make sense if added to search.nixos.org. meta.sourceCode or meta.sourceTree would make sense if not for the fact that search results already have a "Source" field which points to the location of the package.nix file in nixpkgs.

Things done

  • checked that the meta field is readable for packages built with stdenv and rustPlatform.buildRustPackage
  • checked that the meta field correctly defaults to null for packages that use fetchzip
  • checked that derivations are not changed, so this PR will not cause any rebuilds
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 8.has: documentation This PR adds or changes documentation 6.topic: stdenv Standard environment labels Mar 8, 2024
@lolbinarycat lolbinarycat marked this pull request as draft March 8, 2024 21:05
@ofborg ofborg bot added 8.has: clean-up 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Mar 8, 2024
@lolbinarycat lolbinarycat marked this pull request as ready for review March 9, 2024 00:28
@lolbinarycat
Copy link
Contributor Author

@Mindavi note that meta.repository does not default to meta.homepage, it defaults to src.meta.homepage, which is automatically set by fetchers such as fetchFromGitHub to point to the source repository.

setting meta.homepage will not affect meta.repository

@@ -45,6 +45,10 @@ Release branch. Used to specify that a package is not going to receive updates t

The package’s homepage. Example: `https://www.gnu.org/software/hello/manual/`

### `repository` {#var-meta-repository}

HTTP-browsable source tree of the package. Automatically set if the package uses a `fetchFrom*` fetcher for its `src`. Example: `https://github.com/forthy42/gforth`
Copy link
Member

@AndersonTorres AndersonTorres Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not every repository is HTTP(S)-browsable.
Indeed FTPs were a very common thing some short time ago, and there are projects of hosting forges on I2P networks.

Further, automatically filling this attribute is not a good idea. It should be explicitly set when desired.

Suggested change
HTTP-browsable source tree of the package. Automatically set if the package uses a `fetchFrom*` fetcher for its `src`. Example: `https://github.com/forthy42/gforth`
Source repository of the package.
Examples:
- `https://git.savannah.gnu.org/git/gforth.git`
- `https://github.com/NixOS/nixops`
- `https://nest.pijul.com/pijul/pijul`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not every repository is HTTP(S)-browsable.

no, but meta.repository is specifically for http-browsable source trees. it exists for maintainer convenience and to be displayed by sites like search.nixos.org

Copy link
Member

@AndersonTorres AndersonTorres Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only HTTP-browsable repositories? Why such discrimination?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because this field is meant to be presented via a web browser.

would you accept a gemini or gopher link as a value for meta.homepage?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I would.

Not a long time ago browsers displayed Gopher and even FTP. They dropped this because of code base cleanup.

What will be next? Dropping non-conformant forges like viewcvs?

Further, with some hack modern web browsers can rely on extensions that allow looking at those sites.

Further, special purpose clients still exist for them.

Further, Emacs has support for them:

https://www.emacswiki.org/emacs/CategoryGopher/

https://www.emacswiki.org/emacs/AngeFtp/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndersonTorres could you elaborate on this?

Further, automatically filling this attribute is not a good idea. It should be explicitly set when desired.

I think this actually a great idea! When fetching from a repo, linking to that exact repo seems to be the sane thing to do, and exactly what I would expect as a user. Of course there are cases where that isn't possible (if you have multiple sources, when fetching from an unknown forge that we don't know how to translate, etc.), but those are the only ones that should be handled manually, IMO.

The other option of adding the new meta field to every single package manually results in additional (and unnecessary) maintainer load.

@@ -71,7 +71,8 @@ in
if version == "8.11.0+0.11.1" then version
else builtins.replaceStrings [ "+" ] [ "." ] version
}.tbz";
sha256 = release."${version}".sha256;
# abort/syntax error will fail package set eval, but throw is "fine"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because without it, ofborg-eval fails due to niche consequences of lazy evaluation.

basically, evaluating meta now evaluates src, which causes problems for this package specifically.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewording it, your meta.eyecandy field broke Nixpkgs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndersonTorres If you want your criticism to be meaningful I'd suggest not leaving it up for interpretation.

Copy link
Member

@AndersonTorres AndersonTorres Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How exactly is this open to interpretation? Asking sincerely.

Creating a meta.XYZ field introduced a breaking change.
A breaking change in an otherwise completely unrelated piece of code.

How can this possibly differ from

Rewording it, your meta.eyecandy field broke Nixpkgs.

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't consider useful metadata to be eye candy, that's why I'm asking for clarification.

How that relates to the field's relationship to the derivations src attribute is the actual question we should be following up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't consider useful metadata to be eye candy, that's why I'm asking for clarification.

Neither me. Indeed Aleksanaa coninced me of a similar idea.

However the arguments from this PR's creator about which values are acceptable revolve around eyecandy.
The usefulness of this new field, according to themm, is basically "only HTTP-browsable values are allowed because I want to see them on my web browser that supports only HTTP".

Comment on lines 448 to 473
lib.attrsets.optionalAttrs (attrs ? src.meta.homepage) {
# should point to an http-browsable source tree, if available.
# fetchers like fetchFromGitHub set it automatically.
# this could be handled a lot easier if we nulled it instead
# of having it be undefined, but that wouldn't match the
# other attributes.
repository = attrs.src.meta.homepage;
} // {
Copy link
Member

@AndersonTorres AndersonTorres Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said before, we should not rely on setting this automatically.

Since you and @Aleksanaa are arguing for including this, it should be explicitly set by the package maintainer, never inferred.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further it should be a list of repositories, since it is very common a repo having backup mirrors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is clear that you are misunderstanding the point of this field.

it is not for use by update scripts, that is what src.gitRepoUrl is for.

As I said before, we should not rely on setting this automatically.

you say this, but you do not specify why.

please give an example of when setting this automatically would lead to undesired behavior. several (most?) of the meta.* fields are set automatically.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB packagers would have to explicitly set it to null if there's no browsable repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there's no browsable repo, src.meta.homepage will not be set, and thus meta.repository will not be set either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is clear that you are misunderstanding the point of this field.

It is clear you gave no reason why this field should point to an HTTP-browsable site to begin with.

it is not for use by update scripts, that is what src.gitRepoUrl is for.

This is irrelevant.
First, nothing hinders this field to be useful for other applications besides search.nixos.org. Second, nothing should hinder this field to be used by other applications besides search.nixos.org.

Third, what is the relevance? When I said this should be made for update scripts?
I merely suggested this could be a multi-valued field, since this is a typical occurrence in nature.

As I said before, we should not rely on setting this automatically.

you say this, but you do not specify why.

please give an example of when setting this automatically would lead to undesired behavior. several (most?) of the meta.* fields are set automatically.

Behold!

erro-crasso

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is clear you gave no reason why this field should point to an HTTP-browsable site to begin with.

it is supposed to be presented to the user via sites like search.nixos.org

therefore it should be a link that can be opened by a browser

This is irrelevant.

the existance of a similar but semantically different field is extremely relevant.

there is no reason for this field to represent a machine-readable git repository, as there is already another field that represents that.

First, nothing hinders this field to be useful for other applications besides search.nixos.org.
Second, nothing should hinder this field to be used by other applications besides search.nixos.org.

yes, that is correct. any other application that wants a human-readable, HTTP-browseable source code repository can use this field, because that is what this field represents.

Third, what is the relevance? When I said this should be made for update scripts?

yes, that is exactly the relevance

I merely suggested this could be a multi-valued field, since this is a typical occurrence in nature.

yes, while mirror repositories are common, i don't think modeling that is worth the added complexity.

  1. most fetchers already handle mirrors in some way
  2. having the user pick from a list of mirrors may cause confusion, needlessly clutter the UI, and make using the value more difficult

if you can show me a practical usecase where multiple values would be helpful, i can update this to allow multiple values.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some packages have multiple sources, like trellis, and thus multiple repositories.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough.

Copy link
Member

@mweinelt mweinelt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chances are, that with the introduction of repository, the homepage field will lead to more end-user facing documentation, which is totally fine.

But if the repository URL is now auto-computed, that means there is no actual link in the package definition to follow, which is a downgrade for the development experience.

Note that this will be the third URL in meta: homepage, downloadPage and now repository. I wonder specifically about the redundancy between downloadPage and repository.

@piegamesde
Copy link
Member

Honestly, between downloadPage and repository I prefer the latter. Given that I use NixOS and can just get the software with nix-shell -p or whatever, I rarely have any need to visit a project's downloads page. However, visiting a repository is a common use case.

@eclairevoyant
Copy link
Contributor

why this field should point to an HTTP-browsable site to begin with

The use cases I personally have are:

  • Checking upstream for reports of issues and perhaps fixes for the issues I run into at build-/run-time (for packaging)
  • Debugging issues that aren't reported (for packaging, usually)
  • Cursory examination of the code (as an end-user)

None of these entail any interest in contributing any code upstream, nor is it even git-specific (!) as some of the other discussion implied - projects using hg, fossil, etc. often have browsable repos as well.

Secondly, when I was new to nixpkgs, I found it fairly confusing to click "Source" on the search page and end up in nixpkgs. I guess this is how other distros (Arch, Debian) also work, it's not as if nixpkgs is unique in this regard, but it's not intuitive to necessarily show packaging concerns on the search page. And if we are doing so already, then I don't see the scope creep of showing yet another useful link with relevance to packaging (and disambiguating them properly, e.g. "nixpkgs source" vs "upstream source" or such).

But if the repository URL is now auto-computed, that means there is no actual link in the package definition to follow

I can see that, but I'd think that preferable to a bunch of PR churn (thinking back to meta.mainProgram). In that case it was far more important to ensure it was set correctly, if at all, whereas in this case of meta.repository, I'd rather not have a bunch of new PRs just to set the one field on existing packages, or even force contributors to set such a field on new packages.

if you can show me a practical usecase where multiple values would be helpful

Mirrors exist to at least increase availability; if one repo is down, I'd still like to be able to view the source.

@lolbinarycat
Copy link
Contributor Author

Chances are, that with the introduction of repository, the homepage field will lead to more end-user facing documentation, which is totally fine.

But if the repository URL is now auto-computed, that means there is no actual link in the package definition to follow, which is a downgrade for the development experience.

we could set a guideline that at least one of homepage or repository has to have an actual value withing the derivation expression.

Note that this will be the third URL in meta: homepage, downloadPage and now repository. I wonder specifically about the redundancy between downloadPage and repository.

  1. fourth, actually, changelog also exists.
  2. downloadPage is described as "The page where a link to the current version can be found". it's not clear if that refers to binaries, source tarballs, or both. additionally, it is not widely used as far as i can tell.

@lolbinarycat
Copy link
Contributor Author

@eclairevoyant

when I was new to nixpkgs, I found it fairly confusing to click "Source" on the search page and end up in nixpkgs. I guess this is how other distros (Arch, Debian) also work, it's not as if nixpkgs is unique in this regard, but it's not intuitive to necessarily show packaging concerns on the search page. And if we are doing so already, then I don't see the scope creep of showing yet another useful link with relevance to packaging (and disambiguating them properly, e.g. "nixpkgs source" vs "upstream source" or such).

this exact disambiguation is one of the reasons i didn't name it something like sourceCode.

i think if you just put an "upstream repository" link next to the links in the search results that would be fine, but i suppose that's a problem for another time.

Mirrors exist to at least increase availability; if one repo is down, I'd still like to be able to view the source.

that's certainly a nice thought in theory, but here's what would be needed to be able to use it in practice:

  1. the repository needs to have a mirror. this is only really common for popular projects.
  2. package maintainers need to manually add and maintain a list of available mirrors. there is mirrorsFile, but i don't think it's available at runtime.
  3. the repository needs to go down
  4. you need to be able to find the url of that mirror, either by manually checking the derivation file, or by it somehow being presented on serach.nixos.org (which doesn't even handle multiple homepage values currently)

is it worth the effort to do all that just so you don't have to use the wayback machine or search "PACKAGE_NAME source code mirror"?

it would be a fairly simple change to make check-meta allow multiple values, but i'm mostly concerned about the additional work that it will require in search.nixos.org (mainly in figuring out how to display multiple repos)

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 20, 2024

It is clear you gave no reason why this field should point to an HTTP-browsable site to begin with.

it is supposed to be presented to the user via sites like search.nixos.org

First, this is a backwards thinking: Nixpkgs can exist without the search engine, the search engine can't exist without Nixpkgs.

Second, what is the problem in providing a pointer to a repository that is not HTTP-reachable? Why the search engine can't return non-HTTP links as pieces of info?

There is a truckload of non-HTTP-links info already, such as licenses, descriptions, platforms...

Or should we provide an OSI link for each license and another for each broken platform?

therefore it should be a link that can be opened by a browser

  1. Again, eycandy.
  2. Non Sequitur. A pointer to a repository is still a point to a repository regardless the browser's capabilities of reaching and eyecanding rendering it.

This is irrelevant.

the existance of a similar but semantically different field is extremely relevant.

Can you follow the context of what I wrote? "This is irrelevant" is explained right below: being useful to automation scripts is not a demerit for a pointer to a repository.

there is no reason for this field to represent a machine-readable git repository, as there is already another field that represents that.

No, there is not.

Relying on src.some-obfuscated-ill-documented-field-never-intended-to-be-exported is a really bad idea.
One year ago a change in the algorithms that generated git archive tarballs caused a massive haul over the Internet: https://lwn.net/Articles/921787/

Curiously Nixpkgs was not affected because our fetchers do not deal with the archives but rather with their contents.

Hell, we have in-house examples!
The whole pkgs/data/fonts needed to be rewrote because some leaked implementation details of fetchzip changed!

yes, that is exactly the relevance

The relevance of meta.repository is helping update scripts?

yes, while mirror repositories are common, i don't think modeling that is worth the added complexity.

You can easily replicate the meta.license approach.

And, complexity? You are complaining about complexity, when you are writing such a complicated code that it broke an otherwise completely unrelated code?

1. most fetchers already handle mirrors in some way

Why are you bringing the fetchers?
Can you fix in a single subject?

Just because your meta code broke some fetcher, it does not follow we should talk about both.

2. having the user pick from a list of mirrors may cause confusion, needlessly clutter the UI, and make using the value more difficult
  1. A user (that is indeed a programmer) that should be used to multi-valued fields?
  2. We already have lists of maintainers, licenses and platforms, and no one complained about confusion, needless clutter, and difficulties of using the value.
  3. The search.nixos.org webmaster is free to pick only the head of meta.repository if the "cluttering" is such a serious concern.

if you can show me a practical usecase where multiple values would be helpful, i can update this to allow multiple values.

Oh boi....

You yourself employed an unofficial mirror to GForth!

https://github.com/forthy42/gforth

Gforth mirror on GitHub (original is on Savannah)

Or what? Savannah is not eyecandy enough when compared to GitHub's bells and whistles for mentally-overloaded programmers?

https://git.savannah.gnu.org/cgit/gforth.git/

@lolbinarycat
Copy link
Contributor Author

stop saying "eyecandy"

i don't care about appearances.

you keep telling me what i want, and i'm tired of it.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 22, 2024

I want to pick some ideas tomorrow.
But I want to bring an idea that was said to me yesterday:

meta.repository is an attribute suitable for srcs, especially those made from fetchFrom.*.
The outer derivation can/should inherit it, instead of overloading src.meta.homepage.

@lolbinarycat
Copy link
Contributor Author

meta.repository is an attribute suitable for srcs, especially those made from fetchFrom.*. The outer derivation can/should inherit it, instead of overloading src.meta.homepage.

i can try to make fetchFrom* set src.meta.repository in addition to src.meta.homepage, but removing an existing field seems like a great way to break existing tools.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 24, 2024

I said nothing about removing fields. (Which is a bit ironic!)

But this is irrelevant - you are not convinced about defining meta.repositories as a list while you are still using an unofficial mirror as an example...

@lolbinarycat
Copy link
Contributor Author

ok so instead of meta.repository = src.meta.homepage, you want meta.repository = src.meta.repository = src.meta.homepage?

@AndersonTorres
Copy link
Member

ok so instead of meta.repository = src.meta.homepage, you want meta.repository = src.meta.repository = src.meta.homepage?

meta.repository = src.meta.repository should suffice.

@lolbinarycat
Copy link
Contributor Author

ok, but src.meta.repository does not currently exist.

as i understand it, you want to introduce it as a new field that will have a value identical to src.meta.homepage in all cases.

@AndersonTorres
Copy link
Member

ok, but src.meta.repository does not currently exist.

Yes, and it is a chicken-egg problem.

@lolbinarycat
Copy link
Contributor Author

can we just add the field in it's current state then come back and add src.meta.repository in a second PR?

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 25, 2024

Yes, but Can we just make the field a list, like in meta.license?

After all you did not answered the arguments about making it a list of URLs, even after still using a MIRROR of GFORTH in your example.

Ah! Another reason for keeping mirrors: SLAPPers and preservation of abandonware!

@@ -444,7 +448,29 @@ let
let
outputs = attrs.outputs or [ "out" ];
in
{
optionalAttrs (attrs ? src.meta.homepage || attrs ? srcs && isList attrs.srcs && any (src: src ? meta.homepage) attrs.srcs) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, another suggestion:

Since you are grabbing src.meta.homepage regardless its existence (and dealing with the non-existence gracefully), please grab the yet-non-existent src.meta.repository too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aleksanaa any objection? I would 👍 to listen.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want a different implementation, please open another PR yourself. The author's purpose is already very clear.

Copy link
Member

@AndersonTorres AndersonTorres Mar 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a different implementation, just an add-on. Have you read the "too" in the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, it's merged now, so you are free to open your own PR that builds on top of it.

Copy link
Member

@AndersonTorres AndersonTorres Mar 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't wanna lose my vacation days.

(You know what?...)
#300286

@lolbinarycat
Copy link
Contributor Author

@a-n-n-a-l-e-e squashed!

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

built stdenv & tests on aarch64/x64 darwin

@ghost ghost merged commit 2809c84 into NixOS:master Mar 29, 2024
21 of 23 checks passed
@adisbladis
Copy link
Member

The implementation in this PR suffers from some performance issues.
Addressed in #300177.

To future nixpkgs mergers:
check-meta.nix is one of the hottest nixpkgs paths and any performance hit here is noticeable.
Please check the ofborg performance report before merging.

adisbladis added a commit that referenced this pull request Mar 30, 2024
stdenv/check-meta: Fix performance regressions introduced in #294347
@K900
Copy link
Contributor

K900 commented Mar 30, 2024

This broke eval for packages where src can fail to eval, which is pretty much every package with binary-only sources. Reverting in #300247, this will need to be fixed and relanded along with #300177.

K900 added a commit that referenced this pull request Mar 30, 2024
@ghost
Copy link

ghost commented Mar 30, 2024

The implementation in this PR suffers from some performance issues. Addressed in #300177.

To future nixpkgs mergers: check-meta.nix is one of the hottest nixpkgs paths and any performance hit here is noticeable. Please check the ofborg performance report before merging.

looked at the performance and it was less than 0.2 % increase in cpu time. however, other eval runs reported different increases with the same change which seems that the perf result for the change is lost in the noise. additionally, it doesn't appear that #300177 has any effect when compared to no-op changes like r-ryantm updates.

@ghost
Copy link

ghost commented Mar 30, 2024

This broke eval for packages where src can fail to eval, which is pretty much every package with binary-only sources. Reverting in #300247, this will need to be fixed and relanded along with #300177.

blurg. thanks for reverting.

@ghost
Copy link

ghost commented Mar 30, 2024

This broke eval for packages where src can fail to eval, which is pretty much every package with binary-only sources. Reverting in #300247, this will need to be fixed and relanded along with #300177.

blurg. thanks for reverting.

https://hydra.nixos.org/build/254689522/nixlog/1/tail
nix-instantiate -A brscan4 --show-trace --argstr system aarch64-linux

@lolbinarycat
Copy link
Contributor Author

This broke eval for packages where src can fail to eval, which is pretty much every package with binary-only sources.

shouldn't a binary-only package like that just lack the src attribute altogether? or it would just point to a binary tarball?

the code should check if src exists before trying to access it.

also, a specific example would be appreciated so i can replicate the issue.

@lolbinarycat
Copy link
Contributor Author

the brscan4 failure seems to be exclusive to aarch64, since nix eval .#brscan4.meta works fine on my end.

@K900
Copy link
Contributor

K900 commented Mar 30, 2024

It is specific to platforms where it's not supported, because that's where it throws, yes.

@lolbinarycat
Copy link
Contributor Author

what do people think about just wrapping this in tryEval or deepSeq so this doesn't happen?

@lolbinarycat
Copy link
Contributor Author

hmm problem is tryEval doesn't catch index errors (and idk why i thought deepseq would do anything)

@lolbinarycat
Copy link
Contributor Author

alternatively, we could make it default to nil or an empty list, and remove the optional and //, that way evaluating other meta attributes doesn't evaluate repository.

@AndersonTorres
Copy link
Member

Some useful ideas on catching throws can be found on this nice code from @matejc (in memerian from Nixpkgs):

https://blog.matejc.com/blogs/myblog/generate-package-info

@lolbinarycat
Copy link
Contributor Author

tryEval doesn't catch the kind of error that is causing eval to fail. try builtins.tryEval ({}.a) in the repl to see.

i opted instead to just try to avoid the error.

@AndersonTorres
Copy link
Member

builtins.hasAttr doesn't work?

@lolbinarycat
Copy link
Contributor Author

we already use the ? operator to check if src has the relevant fields, but the problem is, just evaluating src enough to check if it has an attribute is sometimes enough to cause an unrecoverable error. the only other option would be making sure src evaluates on every package, even on unsupported platforms.

@lolbinarycat
Copy link
Contributor Author

these packages may as well be setting src = if builtins.currentSystem == "aarch64-linux" then abort "someerror" else realsrc

@AndersonTorres
Copy link
Member

Aborts, contrary to throws, are not catch-able - this needs another thing...

then
[ attrs.src ]
else
lib.filter (src: src ? meta.homepage) attrs.srcs;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just noticed something while testing a rewrite: this filter statement needs to be moved outside the if, because otherwise the case of attrs.src not having a meta attribute is not handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing this in the newest version

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: stdenv Standard environment 8.has: clean-up 8.has: documentation This PR adds or changes documentation 9.needs: community feedback 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: new meta attr for source code repository
10 participants