Observations about WPF beta 13

[Ben]

Holy Moly! Haven’t seen such professional, informative and constructive feedback for years!!! We’ve logged those items in our tracking system and thanks for your support and understanding! Let us know when you need anything on the last 1% and thanks again for your support.

Ben

[Mikko_Viitala]

Hey @Ben

Thanks for asking. I might as well share that 1% here. I doubt those will be readily available by the end of the year when I take my leave but who knows when I start working with ThinkGeo products again.

First 0.25% is missing GeoPDF raster support.

Other 0.25% is poor 32-bit support (frequent crashes without warning when using large raster layers) but that’s on SkiaSharp.

The rest 0.50% is quite significant memory usage when multiple overlays are added to the MapView, no matter if these overlays are empty. Memory consumption seems to go down when overlay is hidden (which I didn’t consider at first since overlays were empty) but I still find creating overlays on-startup being better than creating overlays on-the-fly since creation takes some time.

I don’t have that many overlays and there’s only a few features to show on start up, still map consumes around 1.5Gb of (unmanaged) memory.

Thanks.

[Ben]

Hi Mikko,

Regarding GeoPdf, I think you meant Capital.pdf you provided in Problems opening GeoPDF layer and showing features from a WFS service - ThinkGeo UI for Desktop / WPF - ThinkGeo Discussion Forums. ThinkGeo is using GDAL for GeoPDF, and we found this particular file cannot be correctly opened in QGIS 3.16.11 on Windows (GDAL Version 3.3.1), but it can be opened in the same version (QGIS 3.16.11) on MacOS. I have a feeling GDAL has an issue on Windows dealing with GeoPDF. Do you think I can share your Capital.pdf to GDAL team to help them locate this issue? We can integrate the latest GDAL in after they fix it.

We saw that 32-bit collapse issue from SkiaSharp as well. We contacted SkiaSharp team and upgraded to the latest version (from 1.* to 2.*), some of the issues were fixed but this one still exists. Skia is a good cross platform rendering engine we don’t want to give up, and I thought 32-bit system is sunsetting, and a 32-bit app is easily to have memory related issue anyway and users should be aware of when loading large data, so even this problem is still bad I think most likely it will go away in the future when everyone uses X64.

The memory issue should be something new. We reviewed the memory usage in the early days of V12 and it looked good, it might be some recent changes that introduced some new memory issues. I am logging it in and will have a look.

Again thanks for your input, we’ll work on this last 1% and keep you posted.

Thanks!
Ben

[Mikko_Viitala]

Hey,

It’s getting quite late in Finland but I’ll provide you with another PDF tomorrow. I find some file that opens fine using ThinkGeo PDF layer and also in QGIS.

If I have few minutes to spare I’ll wrap up an example project regarding memory consumption, too. I also considered it to be fine when initially testing beta 13 but can’t say if it was due to some issue that was not present at the time or just my lazy evaluation on that regard (not putting enough load on the component).

What comes to 32-bit, yes, I agree. Those machines will be phased out eventually but in use cases similar to ours lifespan is expected to be quite long (XP-SP3-was-dropped-last-year long). But we’ll manage with this since problem is known so we can limit file sizes and make sure it comes up in training.

Thanks again, I’ll be back.

[Mikko_Viitala]

@Ben

Attached is a better PDF to work on, something I found from the internetz if I remember correctly.

Here’s how it looks in QGIS 3.18.3 Zürich when opened as vector and as raster on top of WMS service.

Then that same file opened using latest ThinkGeo beta, again on top of some WMS service. The latter image shows the said GeoPDF after it has been transformed into TIF file using gdal_translate.exe from OSGeo4W64.

Conversion used here is

from
"+proj=aea +lat_1=20 +lat_2=60 +lat_0=40 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs"

to
"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"

Here’s the file itself, zipped because of size limitation.
US_County_Populations.zip (2.5 MB)

And then the memory usage thing. Here’s example code using the latest binaries (WPF, .NET Framework 4.8).

If I only add an OpenStreetMapOverlay to the map and pan around, memory usage is around 100Mb…200Mb quite consistently, sometimes spiking to 300Mb for short periods of time.

mapView.MapUnit = GeographyUnit.Meter;
var osmOverlay = new OpenStreetMapOverlay();
mapView.Overlays.Add(osmOverlay);
mapView.Refresh();

If I add ten empty overlays and pan around, I easily hit 1.0Gb…1.2Gb, spiking up to 2Gb in memory usage.

mapView.MapUnit = GeographyUnit.Meter;
var osmOverlay = new OpenStreetMapOverlay();
mapView.Overlays.Add(osmOverlay);

for (int index = 0; index < 10; index++)
{
    var layer = new InMemoryFeatureLayer();
    var overlay = new LayerOverlay(new List<Layer> { layer });
    mapView.Overlays.Add(overlay);
}

mapView.Refresh();

[Ben]

Hi Mikko,

Thanks for the detailed info!

GDAL can open a GeoPDF in either Vector mode or Raster mode. GeoPdfFeatureLayer behind the scenes is opening a pdf using GDAL’s vector mode, in fact you can directly open a GeoPDF using GDAL’s raster mode like this: (with ThinkGeo.Gdal nuget package)

        GeoImage bitmap = new GeoImage(1024, 512);
        GdalRasterLayer layer = new GdalRasterLayer(@"GeoPdf\US_County_Populations.pdf");

        layer.Open();
        RectangleShape extent = layer.GetBoundingBox();
        GeoCanvas canvas = GeoCanvas.CreateDefaultGeoCanvas();
        canvas.BeginDrawing(bitmap, extent, GeographyUnit.Meter);
        layer.Draw(canvas, new Collection<SimpleCandidate>());
        canvas.EndDrawing();

        bitmap.Save($"GeoPdfTest.png"); 

Here below is what the result image look like, you can reproject it just like how you did it on a tiff layer. so you see there’s no need to convert a geoPdf to a tiff.

So can I say you don’t have issues in US_County_Populations.pdf? Capital.pdf (which cannot be opened correctly in QGIS Windows or ThinkGeo but can be opened in QGIS MacOS) is the only issue you have regarding GdoPDF?

Thanks,
Ben

[Mikko_Viitala]

Hey,

I have issues with some PDF files, that Capital.PDF included. I can open it as a raster in QGIS 3.18.3 Zürich but when trying to open as vector, I’ll receive an error.

I have just assumed that there’s something wrong with the file itself. But if you think it should open, also as vector source, then yes, maybe someone can take a look at it further.

And thanks for that piece of code on how to open PDF as raster. I tried it out and it worked perfectly.

Thanks a bunch.

[Ben]

Hi Mikko,

After a closer look, I think that Capital.pdf simply doesn’t include any vector data and we can only open it in raster mode, which reminds me we should probobly add a GeoPdfRasterLayer class so make it easier to be discovered for users.

We’ll check out the memory issue you mentioned and help you complete the last 1%.

Thanks,
Ben

[Mikko_Viitala]

Hey,

Anything new on this? I actually noticed that on older Win7 machine the memory consumption was OK. Results here are from my own laptop running Win10 and 48Gb RAM. So there’s that…

[Ben]

Hi Mikko,

I was pulled over to something else and one of my colleagues is working on it. We recreated it on Win 10 and Win 11. Thanks for the info about Win7, that’s interesting.

This is something we gonna fix in the next major release anyway. Things are going slow this couple weeks because of the holidays but I think we can have it fixed by sometime next month.

I’ll keep you posted and Happy Holidays!

Thanks,
Ben

[Mikko_Viitala]

Hey @Ben,

Happy holidays to you and your team, too.
One other thing I failed to mention is that on Win10 we used 64bit build and on Win7 32bit.

Br,
Mikko

[Ben]

Good to know, Thanks!

[Ben]

Hi Mikko,

The memory issue has been resolved in ThinkGeo.UI.WPF-Beta377, please have a try and let me know what you think. We will put it into the full 12.* version if everything goes well.

Thanks,
Ben

[Mikko_Viitala]

Hey Ben,

I tried the example I posted above with both the latest WPF & Core betas and also with WPF 370 & Core 309.

Based on my subjective view and extra fast map interactions I can’t see much of a difference in RAM usage with my PC setup. Then again I feel like the tiles loaded in smoother and fast panning didn’t freeze the map too much.

So, I didn’t encounter any problems, component seems to be in better state but it’s really hard to test it objectively.

Thanks.

[Ben]

Hi Mikko,

Here below is my test code:

    private void Map_Loaded(object sender, RoutedEventArgs e)
    {
        wpfMap1.MapUnit = GeographyUnit.Meter;

        OpenStreetMapOverlay baseOverlay = new OpenStreetMapOverlay(@"E:\worldstreets-11-11-2021.mbtiles");
        wpfMap1.CurrentExtent = new RectangleShape(-200000000, 200000000, 200000000, -200000000);
        wpfMap1.Overlays.Add(baseOverlay);

        for (int index = 0; index < 10; index++)
        {
            var layer = new InMemoryFeatureLayer();
            var overlay = new LayerOverlay();
            overlay.Layers.Add(layer);
            wpfMap1.Overlays.Add(overlay);
        }

        wpfMap1.Refresh();
    }

Run it in Visual Studio 2022 and keep zooming in/out, and here’s the memory usage I see with Beta-377:

And here is the memory usage with Beta-376:

However, I didn’t see the memory issue in Runtime even with Beta-376. Here is the Memory Usage from Task Manager after doing the same zooming in/out for the same period of time

I tested with both .NET 6.0 and .NET Framework 4.8 and the results are the same. Here below are the test projects, please have a try. They are using 376 and you should see the mem differences after upgrading to 377.
MemoryTest.zip (12.6 KB)

Please send me a demo which recreates your issue. I have a feeling we might not talking about the same memory issue here.

Thanks,
Ben

[Mikko_Viitala]

Hey Ben,

I’m using VS 2017 and only difference in my testing was that I used just var osmOverlay = new OpenStreetMapOverlay() and no initial extent set.

No matter what I do I can’t see big (or any?) difference in memory usage but 377 is still more responsive.

You see much better memory management, I see more responsive map… should we just leave it there or… ?

[Ben]

Hi Mikko,

I installed VS 2017 and didn’t see the issue with 376, I’m pretty sure it’s an issue in VS 2022.

What we did in 377 is to avoid creating new writeableBitmaps if possible, use the old ones instead. That must be where your responsiveness come from, even it doesn’t bring many improvements for you on the memory.

I created a ticket about this on visualstudio.com, I hope Microsoft can fix it soon:
https://developercommunity.visualstudio.com/t/WriteableBitmap-cannot-be-GC-collected-w/1626056?entry=myfeedback

Anyway, I think this means your last 0.5% issues have been fixed. Congratulations! and let me know if you have more issues.

Thanks,
Ben

[Mikko_Viitala]

Yes, everything seems to be good. Funny feeling when I can’t go about complaining about something but I might just get used to it Thanks.

[Mikko_Viitala]

And my moment came before long!

Although speed is fine and everything is working like a charm most of the time, I managed to have e.g. my adornment layer with scale bar to be duplicated across the map after few map view resizes. Those adornments stay where they are (see picture) even though tiles underneath change when panning/zooming.

Adornment should be only on bottom right.

Resizing and moving window from monitor to another might also cause infamous “can’t read memory, this is indication that other memory is corrupt” or something along those lines.

[Ben]

Hi Mikko,

haha, indeed. and I’m sure more issues are coming up, which are the opportunities to improve and grow.

The scale line adornment layer issue is specific and a bit off this topic. I created a new post on behalf of you which will make it easier for other guys to find. Let’s track the issue there.

ScaleBarAdornmentLayer doesn’t work - ThinkGeo UI for Desktop / WPF - ThinkGeo Discussion Forums

Thanks,
Ben