Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use arcade infra for DEB and RPM build #45047

Merged

Conversation

NikolaMilosavljevic
Copy link
Member

@NikolaMilosavljevic NikolaMilosavljevic commented Nov 22, 2024

Resolves #43688

This enables SDK to use shared infrastructure for DEB and RPM build. It also uses the latest tooling implemented by @jkoritzinsky which enables us to run the build on containers without special tooling, i.e. FPM/dep-package-tool. I have updated the pipeline to use simple containers.

Previously, there was a simple test that ensured installation of SDK package, along with all of the dependencies. The test had limited value. I have created an issue (epic) to track all improvements around installer testing - dotnet/source-build#4758, as well as package installation - dotnet/runtime#109848

Manpages were never installed with SDK package even though there was some 'code' in the old infra. There was a bug in that code. New shared infrastructure supports manpages but due to a current issue, it does not install them to the correct location. I have created two issues to track relevant fixes:: dotnet/arcade#15243 and #44833

There are some minor differences in metadata contents. It, now, better matches other .NET packages, i.e. those produced by runtime. Tracked with #45126

Testing

Fully tested with internal build: https://dev.azure.com/dnceng/internal/_build/results?buildId=2586841&view=results

Copy link
Member

@jkoritzinsky jkoritzinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good. Just a few comments around things I'd like to get a little cleaner.

Also, we should be able to delete https://github.com/dotnet/sdk/blob/main/src/Tasks/sdk-tasks/BuildFPMToolPreReqs.cs and files under https://github.com/dotnet/sdk/tree/main/src/Installer/redist-installer/packaging/rpm now

Comment on lines 101 to 126
<Target Name="AddLinuxPackageInformation"
BeforeTargets="GetDebInstallerJsonProperties;GetRpmInstallerJsonProperties"
DependsOnTargets="CalculateLinuxNativeInstallerDependencyVersions">

<PropertyGroup>
<DebianPostinstTemplateFile>$(MSBuildThisFileDirectory)../redist-installer/packaging/deb/postinst</DebianPostinstTemplateFile>
<DebianPostinstFile>$(BaseIntermediateOutputPath)debian/postinst</DebianPostinstFile>
</PropertyGroup>

<!-- Generate postinst file -->
<ReplaceFileContents
InputFiles="$(DebianPostinstTemplateFile)"
DestinationFiles="$(DebianPostinstFile)"
ReplacementPatterns="%SDK_VERSION%"
ReplacementStrings="$(Version)" />

<ItemGroup>
<LinuxPackageDependency Include="dotnet-runtime-$(MicrosoftNETCoreAppMajorMinorVersion)" Version="$(MicrosoftNETCoreAppRuntimePackageVersionWithTilde)" />
<LinuxPackageDependency Include="dotnet-targeting-pack-$(MicrosoftNETCoreAppMajorMinorVersion)" Version="$(MicrosoftNETCoreAppRefPackageVersionWithTilde)" />
<LinuxPackageDependency Include="dotnet-apphost-pack-$(MicrosoftNETCoreAppMajorMinorVersion)" Version="$(MicrosoftNETCoreAppRuntimePackageVersionWithTilde)" />
<LinuxPackageDependency Include="netstandard-targeting-pack-$(NetStandardTargetingPackMajorMinorVersion)" Version="$(NetStandardTargetingPackPackageVersionWithTilde)" />
<LinuxPackageDependency Include="aspnetcore-runtime-$(AspNetCoreMajorMinorVersion)" Version="$(AspNetCoreRuntimeVersionWithTilde)" />
<LinuxPackageDependency Include="aspnetcore-targeting-pack-$(AspNetCoreMajorMinorVersion)" Version="$(AspNetCoreRefVersionWithTilde)" />
<LinuxPostInstallScript Include="$(DebianPostinstFile)" />
</ItemGroup>
</Target>
Copy link
Member

@jkoritzinsky jkoritzinsky Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move most of this target out of a target and move the "replace file contents" task usage into a target that runs before the GenerateInstallers target when BuildDebPackage is true?

(I'd like to reduce references to the GetDeb/RpmInstallerJsonProperties targets as much as we can)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these properties are similarly set in runtime, in the same target, i.e. https://github.com/dotnet/runtime/blob/main/src/installer/pkg/sfx/installers/dotnet-host.proj

I agree that task should be conditioned to run for DEB build only.

I'll check what else can be moved out of this target.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, many of them are due to how the old tooling worked and how it was easier to reuse the existing spot.

Other ones are mainly around getting the current installer package version and using that to declare dependencies on other packages.

I'm working on some changes in arcade to make moving the second group out of a target possible.

@NikolaMilosavljevic
Copy link
Member Author

Generally looks good. Just a few comments around things I'd like to get a little cleaner.

Also, we should be able to delete https://github.com/dotnet/sdk/blob/main/src/Tasks/sdk-tasks/BuildFPMToolPreReqs.cs and files under https://github.com/dotnet/sdk/tree/main/src/Installer/redist-installer/packaging/rpm now

Thanks. Yeah, those can be cleaned - will do that.

@NikolaMilosavljevic
Copy link
Member Author

Force pushed to update the branch. Resolved conflicts offline.

@NikolaMilosavljevic
Copy link
Member Author

Addressed few more comments. Removed unused task and conditioned the DEB postinst processing. We can do a follow-up cleanup if needed.


<Target Name="SetCustomPackagingProperties" AfterTargets="_GetInstallerProperties">
<PropertyGroup>
<PackageSummary>Microsoft .NET SDK $(PackageVersion)</PackageSummary>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you set <UseBrandingNameInLinuxPackageDescription>true</UseBrandingNameInLinuxPackageDescription> in the property group on the top of the project, you'll get a package summary like Microsoft .NET SDK - 10.0.101 instead of Microsoft .NET SDK 10.0.101 and (in combination with dotnet/arcade#15308) not need this custom target.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, it's worth waiting for arcade to flow to sdk, to be able to utilize that change and get rid of this target.

Or we can do this as a follow-up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either waiting or doing this as a follow-up cleanup.

@NikolaMilosavljevic NikolaMilosavljevic merged commit 10d457d into dotnet:main Dec 5, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Infrastructure untriaged Request triage from a team member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update sdk to use new Arcade .deb creation tooling
3 participants