Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpmspec does not expect debuginfo rpm for a subpackage #1878

Closed
fepitre opened this issue Jan 5, 2022 · 8 comments · Fixed by #3085
Closed

rpmspec does not expect debuginfo rpm for a subpackage #1878

fepitre opened this issue Jan 5, 2022 · 8 comments · Fixed by #3085
Assignees

Comments

@fepitre
Copy link

fepitre commented Jan 5, 2022

I've attached here https://gist.github.com/fepitre/de06639902f5da5f8614723da76a71c4#file-qubes-qrexec-spec a spec file for which I'm trying to predict resulting RPM files.

I've attempted to query with either --builtrpms or --rpms (default) but the resulting RPMs:

$ rpmspec -q --qf '%{name}-%{version}-%{release}.%{arch}.rpm\n' qubes-qrexec.spec 
qubes-core-qrexec-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-libs-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-devel-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-debuginfo-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-debugsource-4.1.16-1.fc35.x86_64.rpm

differs from what Mock produces:

DEBUG: Wrote: /builddir/build/RPMS/qubes-core-qrexec-devel-4.1.16-1.fc35.x86_64.rpm
DEBUG: Wrote: /builddir/build/RPMS/qubes-core-qrexec-libs-4.1.16-1.fc35.x86_64.rpm
DEBUG: Wrote: /builddir/build/RPMS/qubes-core-qrexec-debugsource-4.1.16-1.fc35.x86_64.rpm
DEBUG: Wrote: /builddir/build/RPMS/qubes-core-qrexec-libs-debuginfo-4.1.16-1.fc35.x86_64.rpm
DEBUG: Wrote: /builddir/build/RPMS/qubes-core-qrexec-4.1.16-1.fc35.x86_64.rpm

In this case, debuginfo package is for the subpackage libs. I admit it is impossible to predict if a debuginfo package will exist without building it but, should debuginfo/debugsource built RPMs be expected also for subpackages? For example, in this case --builtrpms would produce:

qubes-core-qrexec-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-libs-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-libs-debuginfo-4.1.16-1.fc35.x86_64.rpm <- new one
qubes-core-qrexec-devel-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-debuginfo-4.1.16-1.fc35.x86_64.rpm
qubes-core-qrexec-debugsource-4.1.16-1.fc35.x86_64.rpm
@pmatilai
Copy link
Member

This is just an artifact of the implementation - the top level debug packages are created by a different (much older) mechanism and thus they always exist at parse time already, whereas the sub-package debug packages are created inside the files processing which doesn't happen during spec parse.

There's no easy fix for that I'm afraid.

@DemiMarie
Copy link
Contributor

There's no easy fix for that I'm afraid.

Would it require a major refactoring?

@ffesti
Copy link
Contributor

ffesti commented Jan 18, 2022

No, refactoring does not help here. This is a general problem: Those debuginfo packages' existence depends on the content of their associated sub package. The content does not exist until those packages are built. But the purpose of the rpmspec command is to get the list of package before/without building the package.

@fepitre
Copy link
Author

fepitre commented Jan 18, 2022

No, refactoring does not help here. This is a general problem: Those debuginfo packages' existence depends on the content of their associated sub package. The content does not exist until those packages are built. But the purpose of the rpmspec command is to get the list of package before/without building the package.

I agree. The only workaround I've found for now is to add -debuginfo/-debugsource for the package and all its subpackages (except devel) to have list off all possible packages before build.

@ffesti
Copy link
Contributor

ffesti commented Jan 24, 2022

In the end we have to accept we can't get a correct list and we have to decide whether the list errs on to many or too little packages or both. As we are currently considering dynamic sub packages we will end up with too little no matter what. So I am hesitant to opt for too many here.

@fepitre
Copy link
Author

fepitre commented Jan 24, 2022

@ffesti if you want a bit of context for why I've reached this, I'm currently creating a build system that runs actual build inside a cage (either a container or an ephemeral VM) and I only copy-back expected RPMs. Having too many allows to skip if it does not exist. On the other hand, having unexpected RPMs raise a build failure. The idea is to not inject unexpected RPMs inside a repository where all is done automatically.

@pmatilai
Copy link
Member

Perhaps the sanest behavior we can do is only generate debuginfo packages on an actual build, and never any on rpmspec query. That way it's at least somehow consistent with itself.

As for "unexpected" packages, debuginfo are consistently named and thus easily filtered.
I wouldn't build too much structure around the notion of unexpected packages as Dynamic Spec Generation will change that quite fundamentally.

@pmatilai pmatilai added the RFE label Nov 8, 2023
@pmatilai pmatilai added this to RPM Nov 8, 2023
@github-project-automation github-project-automation bot moved this to Backlog in RPM Nov 8, 2023
@pmatilai pmatilai moved this from Backlog to Todo in RPM Nov 8, 2023
@pmatilai pmatilai moved this from Todo to Priority in RPM Apr 9, 2024
@pmatilai
Copy link
Member

pmatilai commented Apr 9, 2024

This is actually quite closely related to #2204 which we now have to fix.

@pmatilai pmatilai moved this from Priority to In Progress in RPM Apr 15, 2024
@pmatilai pmatilai self-assigned this Apr 15, 2024
pmatilai added a commit to pmatilai/rpm that referenced this issue Apr 15, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Replace the logic parts with actual C code where the logic is more
straightforward to follow, taking advantage of dynamic spec generation
so debuginfo packages are only ever generated during an actual build.

There's a crazy amount of details buried in such a tiny piece, commented
in the code now.

A noteworthy point here is that the presence of the old %install hack now
causes an explicit failure, something like:

    error: line 4: %package debuginfo: package foo-debuginfo already exists
    error: parsing failed

This explicit break is very much intentional as we want distros to remove
those %install hacks from their macro files.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
@pmatilai pmatilai moved this from In Progress to In Review in RPM Apr 15, 2024
pmatilai added a commit to pmatilai/rpm that referenced this issue Apr 15, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Replace the logic parts with actual C code where the logic is more
straightforward to follow, taking advantage of dynamic spec generation
so debuginfo packages are only ever generated during an actual build.

There's a crazy amount of details buried in such a tiny piece, commented
in the code now.

A noteworthy point here is that the presence of the old %install hack now
causes an explicit failure, something like:

    error: line 4: %package debuginfo: package foo-debuginfo already exists
    error: parsing failed

This explicit break is very much intentional as we want distros to remove
those %install hacks from their macro files.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
ffesti added a commit to ffesti/rpm that referenced this issue Apr 16, 2024
! This removes the %ifnarch noarch check. We need to find a solution for
this before merging (or decise it is just an optimization we don't
really need)

All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Turn this isto a proper macro that drops the package definition into a
.SPECPART file. This way debuginfo can be part of the %install script
without messing up the parsing.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
pmatilai added a commit to pmatilai/rpm that referenced this issue Apr 16, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Replace the logic parts with actual C code where the logic is more
straightforward to follow, taking advantage of dynamic spec generation
so debuginfo packages are only ever generated during an actual build.

There's a crazy amount of details buried in such a tiny piece, commented
in the code now.

A noteworthy point here is that the presence of the old %install hack now
causes an explicit failure, something like:

    error: line 4: %package debuginfo: package foo-debuginfo already exists
    error: parsing failed

This explicit break is very much intentional as we want distros to remove
those %install hacks from their macro files.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
ffesti added a commit to ffesti/rpm that referenced this issue Apr 22, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Turn this isto a proper macro in __spec_install_template that drops the
package definition into a .SPECPART file. This way debuginfo can be part
of the %install script without messing up the parsing.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
pmatilai added a commit to pmatilai/rpm that referenced this issue May 10, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
pmatilai added a commit to pmatilai/rpm that referenced this issue May 13, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

As a nice bonus, this makes debuginfo work for packages that don't use
%setup. Add an explicit test for this in the "rpmbuild %caps" test.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
pmatilai added a commit to pmatilai/rpm that referenced this issue May 13, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

As a nice bonus, this makes debuginfo work for packages that don't use
%setup. Add an explicit test for this in the "rpmbuild %caps" test.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Co-authored-by: Florian Festi <[email protected]>

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
pmatilai added a commit to pmatilai/rpm that referenced this issue May 13, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

Note that the old %install hack MUST BE REMOVED from distros now.

As a nice bonus, this makes debuginfo work for packages that don't use
%setup. Add an explicit test for this in the "rpmbuild %caps" test.
specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Co-authored-by: Florian Festi <[email protected]>

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
@ffesti ffesti closed this as completed in 8535694 May 13, 2024
@github-project-automation github-project-automation bot moved this from In Review to Done in RPM May 13, 2024
dmnks pushed a commit to dmnks/rpm that referenced this issue May 17, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

Note that the old %install hack MUST BE REMOVED from distros now.

As a nice bonus, this makes debuginfo work for packages that don't use
%setup. Add an explicit test for this in the "rpmbuild %caps" test.
specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Co-authored-by: Florian Festi <[email protected]>

Fixes: rpm-software-management#2204
Fixes: rpm-software-management#1878
(cherry picked from commit 8535694)
dmnks pushed a commit that referenced this issue May 17, 2024
All these years, enabling debuginfo has required distros to hijack the
spec %install section with a macro like this:

    %install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
    %%install\
    %{nil}

This for a widely used, longtime upstream supported feature is just
gross, and also very non-obvious, feeble and whatnot. And totally
prevents the new append/prepend options from being used with %install.

Take advantage of several newish features to make this happen: we need
expressions to properly handle the numeric %_enable_debug_packages value
from a macro, and if enabled, output the debuginfo template as a dynamic
.specpart.

Enable debuginfo on Linux by default in the platform configuration.
Notably noarch should not have debuginfo so it's disabled in the
platform configuration - since 96467dc
we can now actually rely on the platform configuration being valid,
so we can drop the "%ifnarch noarch" from the debug package check.
Further streamlining should be possible.

Note that the old %install hack MUST BE REMOVED from distros now.

As a nice bonus, this makes debuginfo work for packages that don't use
%setup. Add an explicit test for this in the "rpmbuild %caps" test.
specstep.spec needs to be made noarch here, otherwise it'll now try
to produce debuginfo and fail.

Co-authored-by: Florian Festi <[email protected]>

Fixes: #2204
Fixes: #1878
(cherry picked from commit 8535694)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
4 participants