Error from Linux build using Gdal to open KML data

Jason_Giles · January 24, 2024, 11:30pm

I’m facing the following error when attempting to open a KMLFeatureLayer in our Linux service layer:

> Unhandled exception. System.TypeInitializationException: The type initializer for ‘ThinkGeo.Core.GdalConfiguration’ threw an exception. ││ —> System.TypeInitializationException: The type initializer for ‘OSGeo.GDAL.GdalPINVOKE’ threw an exception. ││ —> System.TypeInitializationException: The type initializer for ‘SWIGExceptionHelper’ threw an exception. ││ —> System.DllNotFoundException: Unable to load shared library ‘gdal_wrap’ or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libg ││ at OSGeo.GDAL.GdalPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_Gdal(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, Exc ││ at OSGeo.GDAL.GdalPINVOKE.SWIGExceptionHelper…cctor()

My issue is very similar to the following ThinkGeo post:
ThinkGeo.Dependency.Gdal version problems

The project builds, runs, and handles KML files when running locally under Windows.
Here is a list of the ThinkGeo packages I’m using:

  <PackageReference Include="ThinkGeo.Core" Version="13.1.0" />
  <PackageReference Include="ThinkGeo.Dependency.Gdal" Version="13.0.1" />
  <PackageReference Include="ThinkGeo.Dependency.MicrosoftVisualCRunTime140" Version="13.0.0" />
  <PackageReference Include="ThinkGeo.Gdal" Version="13.1.0" />
  <PackageReference Include="ThinkGeo.Kml" Version="13.1.0" />
  <PackageReference Include="ThinkGeo.UI.WebApi" Version="13.1.0" />
  <PackageReference Include="ThinkGeo.UnmanagedProj" Version="13.1.0" />

I’ve also included the ThinkGeo.Dependency.Gdal in my API project per the previously referenced post.

I’ve also added the undocumented call needed for setting up the Gdal before using it:
GdalConfiguration.ConfigureGdal();

However, the included packages do not properly install the .so files utilized in the Linux containers.
Why does the dependency package not include the required Linux files?

Thank you for your help,
Jason

Greg2 · January 25, 2024, 7:56pm

I work with Jason and we have narrowed down the potential issue with the ThinkGeo.Gdal package.

When building the project using “dotnet” with an output parameter (i.e. dotnet build -o buildTestt) then none of the runtimes “native” folders (i.e. win-x64\native or linux-x64\native) have any of the gdal files (libgdal, gdal_wrap, etc).

If you run it without an output folder (dotnet build) the native folders are populated with the expected gdal libs.

Here is a .NET project used to test this outcome:
TestThingGEO.zip (111.1 KB)

Ben · January 26, 2024, 8:01pm

Hi guys,

Thanks for reporting! This issue has been fixed in NuGet Gallery | ThinkGeo.Dependency.Gdal 13.1.0-beta1, Please updated your ThinkGeo.Gdal to the latest beta( NuGet Gallery | ThinkGeo.Gdal 13.3.0-beta017, which uses the latest ThinkGeo.Dependency.Gdal) and have another try.

Let us know if everything works fine for you, and we will create a production release for ThinkGeo.Dependency.Gdal.

Thanks,
Ben

Greg2 · January 30, 2024, 1:07am

The beta nuget packages does appear to have fixed the issue using "dotnet build’ with output parameter however, using “dotnet publish” is not including the needed gdal files. Running the following against the previously attached .NET project (and updated with the beta nuget packages) is missing the runtime native files.

dotnet publish “TestThingGEO.csproj” -c Release -o /app/publish

Ben · January 30, 2024, 6:12pm

Hi Greg,

This issue has been fixed in NuGet Gallery | ThinkGeo.Dependency.Gdal 13.1.0-beta2, please update your ThinkGeo.Gdal to the latest beta (018) and have another try. Again, thanks for reporting the issue!

Thanks,
Ben

Jason_Giles · February 14, 2024, 7:35pm

Hi there,
I’ve spent a some time testing the various libraries. While the changes you’ve supplied us with are moving in the right direction, I’m now hitting an actual error coming from the Unix version of the KML/Gdal package. I’ve attached an updated .NET project so you can see the error.

The error message is:
“System.NullReferenceException: Object reference not set to an instance of an object.
at ThinkGeo.Core.GdalFeatureSource.OpenCore()
at ThinkGeo.Core.FeatureSource.Open()”

The project runs correctly in the Windows environment but fails with that message in Docker. Initially, I thought the error might be a permissions error or that the supplied filename’s path was incorrect but as you can see in the test project, that doesn’t seem to be the case.

Please try the project and tell me if you see the same error.
Thank you,
JasonTestThinkGEO.zip (145.1 KB)

Ben · February 20, 2024, 5:33pm

Hi Jason,

It’s because expat.dll, which has the KML driver for GDAL, is only available in ThinkGeo’s GDAL package on Windows, but not available on Linux.

To support it on Linux, we need to rebuild the gdal for Linux and add the expat driver. This is usually difficult and will take some time.

So besides KML, do you need any other gdal features running on Linux? I’ll see if other drivers need to be added.

Thanks,
Ben

Jason_Giles · February 28, 2024, 5:10pm

Ben, that is the only driver we are using. I can’t comment on any others that may be missing.

Ben · March 4, 2024, 9:16pm

Hi Jason,

I’ve added to the todo list and we will work on it soon. And just wondering what Linux distribution you are using? want to make sure we cover your platforms.

Thanks,
Ben

Jason_Giles · March 5, 2024, 4:13pm

Ben, we are using Debian GNU/Linux 11 for Docker, and the nodes are running Ubuntu 20.04. Let me know if you need more info.

Thank you,
Jason

Jason_Giles · March 5, 2024, 4:27pm

Ben, one more tidbit of information around docker. The example project I included, TestThinkGEO.zip, is using the official base image for asp.net6, mcr.microsoft.com/dotnet/aspnet:6.0. This is what we use.
Thanks,
Jason

Ben · March 6, 2024, 4:45am

Good to know, thanks for the info!

Jason_Giles · March 20, 2024, 5:02pm

Any progress on this topic, Ben? Maybe a time frame?
Thanks,
-jason

Ben · March 20, 2024, 8:30pm

Hi Jason,

We are actively working on it. I will try to give you something by the end of this week.

Thanks,
Ben

Jason_Giles · March 20, 2024, 9:43pm

That’s great news, thank you for the update.
-jason

Ben · March 22, 2024, 2:56am

Hi Jason,

Please try the modified sample project below. Run without Debugging to avoid the ThinkGeo debug license check.
TestThinkGeoOnLinuxDocker_0321.zip (5.1 KB)

It should create and launch a docker image, and launch a web browser like following:

Thanks,
Ben

Jason_Giles · April 3, 2024, 2:49pm

Hi Ben,
Yes, the updated sample project ran successfully and gave me the same results you are showing here. I think this version has addressed the issue.

Thank you,
-jason

Ben · April 3, 2024, 7:53pm

Awesome! Let us know if you have more questions.

Jason_Giles · April 3, 2024, 9:18pm

Can you tell me when the fix will be applied to the production releases of those NuGet packages?
Many thanks,
-jason

Ben · April 4, 2024, 2:26pm

Hi Jason, it will be available in v13.3 released by May 1st.