[Bug]: Unable to install Linux shared libraries (with symlinks) into output folder

[emmenlau]

NuGet Product Used

dotnet.exe

Product Version

Version: 6.0.401

Worked before?

Never worked

Impact

I'm unable to use this version

Repro Steps & Context

I'm trying to build a package that includes a Linux shared library that is used via P/Invoke. The library resolved correctly when it is placed in the output bin folder together with the managed binaries.

Now I'd like to create a NuGet package. I've created a nuspec file for that. But for a task that should be rather simple, I've spent more than 20 hours on this problem now and fail to get the Linux shared library into the bin folder of a project that uses my NuGet package.

Things I've tried and how they failed:

1) Simply including all files in the package

When I try to add all files into the package using something like <file src="bin\**" target="lib\$TargetFramework$" />, then the managed dlls are correctly installed in the dependent project. The native libraries with extension *.so* are also included in the nupkg, but are missing from the output bin folder of the dependent project.

2) Explicitly adding all files into the package

When I try to add the native shared library files explicitly into the package using something like <file src="bin\*.so*" target="lib\$TargetFramework$" />, then it fails in the same way as above.

3) Store the native libraries in the runtimes directory

When I add the native shared library files in the runtimes directory, they are correctly installed in the dependent project, but in the same runtimes directory below the output directory bin. This makes the P/Invoke fail because they are not resolved from that directory.

4) Using contentFiles

I tried using contentFiles as outlined in https://devblogs.microsoft.com/nuget/nuget-contentfiles-demystified/, but that did not lead to a success.

5) Using a targets file to copy libraries

The currently best solution is to package the native shared library files in the build\native directory, and then use a dedicated *.targets file to copy them to the output folder. The *.targets file contains:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <NativeLibs Include="$(MSBuildThisFileDirectory)native\**" />
    <None Include="@(NativeLibs)">
      <Link>%(RecursiveDir)%(FileName)%(Extension)</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

This achieves almost what I need, except that it does not preserve symbolic links! Therefore, the common Linux shared library structure is broken. I need to have (where -> denotes a symbolic link):

bin/libMyProject.so -> bin/libMyProject.so.6.3.0
bin/libMyProject.so.6 -> bin/libMyProject.so.6.3.0
bin/libMyProject.so.6.3.0

but instead I get three copies of the library file:

bin/libMyProject.so
bin/libMyProject.so.6
bin/libMyProject.so.6.3.0

This breaks the P/Invoke dependency resolution, when libMyProject is a dependency of another library that is opened with P/Invoke.

Verbose Logs

No response

[erdembayar]

Thank you for filing this issue.
Do you want to have symbol links to prevent from duplicating .so binaries?

[emmenlau]

Thanks @erdembayar for the quick response.

I do not care about package size, so this is not my problem.

But when the libraries appear as duplicate files, P/Invoke fails, or better said, the libraries can not be loaded. I'm using a number of shared libraries including Qt libraries and some others, and they do not work when the symbolic links are broken.

I've tested and they do not load correctly if the symbolic links are missing, and also not when the symbolic links are replaced with the actual files.

[nkolev92]

@emmenlau

Sounds like you need the symlinks generated on the end user machine, as such we don't believe there's anything that needs to be done on the packaging side.

In your targets authoring, you may choose to create symlinks for the files, but at packaging time you really only want to have 1 file. If the current MSBuild capabilities do not allow you to create symlinks feel free to create an issue on the MSBuild side.

As far as, symlinks, within a package go, you can track #10734.

[emmenlau]

@nkolev92 no, I think there is a misunderstanding here. Please read my description again.

Here is the short summary, for reference: The Linux and MacOS shared libraries are treated as second class citizens by NuGet, compared to Windows dlls. When I use Windows dlls in the lib/{tfm} folder, they are automatically and correctly deployed to the build folder of the downstream project. The same does not work for Linux or MacOS shared libraries.

Missing support for symlinks makes the problem worse because even the common workarounds fail. But the correct solution would be that Linux and MacOS shared libraries are just supported in the same way as dlls.

[nkolev92]

Here is the short summary, for reference: The Linux and MacOS shared libraries are treated as second class citizens by NuGet, compared to Windows dlls.

Can you please elaborate on this?

dlls are managed, CLR, they have exact same support on both platforms. NuGet is a transport vehicle that handles compatibility and that's handled the same on all platforms.

The shared libraries are really native libraries and the support there is also equivalent for windows/linux.

But the correct solution would be that Linux and MacOS shared libraries are just supported in the same way as dlls.

What would that look like in your opinion?
How would package be authored and what should be happening at restore time?

[emmenlau]

What would that look like in your opinion?

What should just work is this:

• I create a package layout where I put in lib/{tfm}/
  • on Windows, a number of native shared libraries (extension dll)
  • on Linux, a number of native shared libraries (extension .so.X.Y.Z where X,Y,Z are version numbers, and symbolic links exist from .so.X.Y, .so.X and .so to .so.X.Y.Z)
  • on MacOS, a number of native shared libraries (extension .X.Y.Z.dylib where X,Y,Z are version numbers, and symbolic links exist from .X.Y.dylib, .X.dylib and .dylib to .X.Y.Z.dylib)
• I package a NuGet from this folder structure, always on the corresponding platform
• I install this NuGet in another project
• The native shared libraries are copied to the output folder (bin) of the downstream project when it is compiled

This already works for native shared libraries on Windows (dlls), but does not work on Linux or MacOS. On Linux and MacOS, it fails for two problems:

• No files are copied to the output folder (bin) of the downstream project when it is compiled
• NuGet does not support symbolic links (even though zip can support them)

Therefore I consider Linux and MacOS to be "second class citizens" compare to Windows, where things "just work".

[germgerm]

why was this closed ?
this is a valid concern - is there a work around ?

[emmenlau]

Dear @germgerm , I agree this was not fixed, and NuGet is treating Linux and MacOS as second class citizens, which is quite sad.

There is a rather complicated workaround, where you would package your shared libraries into a zip archive before NuGet packaging, then include the zip archive in the NuGet in a subfolder of build, and then use an msbuild task to extract the zip archive at install time. It is quite an effort to get right, and may be fragile, and whether it works depends on the way the user configures their NuGet dependencies. But at least for some use cases it can work.

[germgerm]

This was my resolution - based on the fact that all of our package are dependant upon one core package
Create an assembly level custom attribute
Have packages include the text file as an embedded resource and mark the assembly with our custom attribute
Have our config handler in the core package check all of our dlls in the bin folder for this custom attribute - if found, extract the embedded file from the assembly - actually we just get the embedded resource stream and never actually extract the file.
Currently we are only concerned with one json file ( appsettings.json) , but the scheme can handle additional items if required