MacOS Catalina fails to load libgdal.27.dylib

[CloudNiner]

Describe the bug

MacOS is unable to link the libgdal.27.dylib when loading GDALRasterSource from geotrellis-gdal package.

Complete error:

[error] Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/97/nzqwjmr53kzbnxbl6xmj1gx40000gp/T/nativeutils159976180077139/libgdalwarp_bindings.dylib: dlopen(/private/var/folders/97/nzqwjmr53kzbnxbl6xmj1gx40000gp/T/nativeutils159976180077139/libgdalwarp_bindings.dylib, 1): Library not loaded: @@HOMEBREW_PREFIX@@/opt/gdal/lib/libgdal.27.dylib
[error]   Referenced from: /private/var/folders/97/nzqwjmr53kzbnxbl6xmj1gx40000gp/T/nativeutils159976180077139/libgdalwarp_bindings.dylib
[error]   Reason: image not found
[error] 	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
[error] 	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
[error] 	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
[error] 	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
[error] 	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2407)
[error] 	at java.base/java.lang.Runtime.load0(Runtime.java:747)
[error] 	at java.base/java.lang.System.load(System.java:1857)
[error] 	at cz.adamh.utils.NativeUtils.loadLibraryFromJar(NativeUtils.java:109)
[error] 	at com.azavea.gdal.GDALWarp.init(GDALWarp.java:128)
[error] 	at com.azavea.gdal.GDALWarp.<clinit>(GDALWarp.java:142)
[error] 	at geotrellis.raster.gdal.GDALDataset$.apply(GDALDataset.scala:338)
[error] 	at geotrellis.raster.gdal.GDALRasterSource.dataset$lzycompute(GDALRasterSource.scala:53)
[error] 	at geotrellis.raster.gdal.GDALRasterSource.dataset(GDALRasterSource.scala:52)
[error] 	at geotrellis.raster.gdal.GDALRasterSource.gridExtent$lzycompute(GDALRasterSource.scala:93)
[error] 	at geotrellis.raster.gdal.GDALRasterSource.gridExtent(GDALRasterSource.scala:93)
[error] 	at geotrellis.raster.RasterMetadata.extent(RasterMetadata.scala:52)
[error] 	at geotrellis.raster.RasterMetadata.extent$(RasterMetadata.scala:52)
[error] 	at geotrellis.raster.RasterSource.extent(RasterSource.scala:43)
[error] 	at Main$.main(Main.scala:6)
[error] 	at Main.main(Main.scala)

To Reproduce

This sbt project reproduces the error when the environment conditions below are satisfied.

macos-gdal-link-failure.zip

Unzip then:

./sbt
run

Expected behavior

The project should successfully print the extent of whatever raster you provide and not throw a link error.

Environment

• GeoTrellis: 3.5.0
• OS: MacOS Catalina 10.15.7
• Scala version: 2.12.12
• JVM version: Failures on [email protected], [email protected], [email protected], [email protected], all installed with either sdkman or jabba
• GDAL version: 3.1.3 installed via brew install gdal or brew install osgeo-gdal
• SBT version: Failures on 1.13.3 and 1.14.0

Additional context

@pomadchin is able to load the dylib and view an extent when he used an old version of Java installed by the old Mac installer that places an entry into the MacOS System Preferences window, [email protected]. This is no longer the supported way to install Java on MacOS, instead you download the JRE/JDK you need directly from Oracle. Does Catalina do more fancy gatekeeper stuff that magically prevents some user-installed dylibs from loading? The error trace has refs to libgdalwarp_bindings.dylib moved or linked to from /private/var.

@pomadchin and I were unable to resolve this error by adding the following to build.sbt in the example project:

javaOptions ++= Seq("-Djava.library.path=/usr/local/lib")
fork := true

Here's my gdal dylibs:

➜  ~/src/gdal-rs-test ls -l /usr/local/lib | grep gdal
lrwxr-xr-x   1 andrew  admin    43 Oct 14 11:30 libgdal.27.dylib -> ../Cellar/gdal/3.1.3_3/lib/libgdal.27.dylib
lrwxr-xr-x   1 andrew  admin    36 Oct 14 11:30 libgdal.a -> ../Cellar/gdal/3.1.3_3/lib/libgdal.a
lrwxr-xr-x   1 andrew  admin    40 Oct 14 11:30 libgdal.dylib -> ../Cellar/gdal/3.1.3_3/lib/libgdal.dylib

This is likely some specific instance of dylib loading failure that would be resolved by #73, but I'm recording it here as a separate issue because it provides a complete example with specific versions of JVM, GDAL and SBT that were tested by @pomadchin and I.

[jamesmcclain]

I am still interested in this as a solution to Macintosh linkage questions (this in spite of the fact that your stack trace shows a failure in dlopen).