Observations about WPF beta 13

[Mikko_Viitala]

Hey,

I have a few general observations about different web maps, tile cache and such in beta 13 that I’d like to share with your development & support team as my role in my current project is coming to an end.

Some of them are just annoyances that one just needs to know and some I’d categorize as actual bugs.

That being said, first time I ever used ThinkGeo offering was back in 2012, with MapSuite Desktop 6.0. Now I have been working with beta 13 about a year. It’s good to see how far you have come with your products and in general I’m more than happy with them. There’s more features I would ever need and current beta has solved 99% of my problems. Not to mention your fast & excellent support here on the forums and with tickets. Same applies to sales, hassle free and fast.

But enough with the sweet talk! Here’s The List

Tile cache

You’ll create overlay’s TileCache using new FileRasterTileCache(string path, string id) so e.g. new FileRasterTileCache(@"c:\temp", "layer")

• If you want to get the path to cache afterwards you need to combine CacheDirectory and CacheId yourself. It would be great to have a new get property that provides this information directly, say CachePath.

• Also, because TileCache works little differently across overlay implementation, getting cache path would be great. Now if you apply mentioned TileCache to OpenStreetMapOverlay you’ll get the exact same path and id you provided in the constructor. But if you apply it to ThinkGeoCloudRasterMapsOverlay then id is changed to e.g. layer\Light_3857 behind the scenes (because of cloud based styles) when you actually wanted it to be just layer.

WMS

• If internet connectivity is lost, then using WmsRasterLayer with LayerOverlay stores error tiles into the tile cache, effectively preventing you from ever getting the actual tile from the service again when connectivity is restored. WmsOverlay works as it’s supposed to in this regard.

• You can add Credentials to WmsOverlay but those credentials are never applied to outgoing web requests. One needs to catch every request and apply request header by hand.

• If you add “custom” parameters to Parameters array of WmsOverlay, implementation breaks. One would expect that anything you add there would be then applied to outgoing request.

• If your base server uri contains any extra parameters, those are dropped or implementation break. Say, end point needs some parameter to work correctly then you need to jump hoops to make that happen (e.g. http://www.someservice.com?parameter=123). This relates to bullet above in a sense that a) query parameters could be added to Parameters array from the WmsOverlay constructor or b) one could add these extra parameters to Parameters array by hand

• You can’t change VERSION of outgoing request w/o catching individual request and modifying it on the fly. Now version is always fixed as 1.0.0 and some services do not like that. I know services don’t always provide response using the version they advertise they do, but this could maybe be a new property on WmsOverlay, say Version ?

• WmsOverlay does not obey max width and -height provided in service capabilities. This can lead to situations where no response is received since requested size exceeds the limits.

WMTS

• Things mentioned above apply here for the most part
• One would expect that WMTS outperforms it’s WMS counterpart from the same service. In reality WMTS is really, really slow. It is usable but one can’t really say it’s a smooth experience. I find this odd.

WFS

• One would expect to find WfsOverlay, similar to WmsOverlay and WmtsTiledOverlay but no such class exists. This makes implementation of multiple web based services a little quirky as they don’t follow the same pattern.

• Credentials property does not exist on WfsLayer

• Feature caching is hidden pretty deep and I’m sure many developers can miss it easily (e.g. layer.FeatureSource.GeoCache.IsActive). It’s also unclear how one should properly work with feature cache in combination with tile cache set for the overlay.

• Sometimes when projection converter is applied to WFS layer nothing is shown on the map if layer was opened right after creation, before applying the projection converter. Sadly I couldn’t reproduce this easily in a simple application but it gave me few new gray hairs.

UnmanagedProjectionConverter

• Sometimes due to not-any-specific-condition creation of UnmanagedProjectionConverter fails when using SRIDs in the constructor, e.g. new UnmanagedProjectionConverter(4326, 3857). Using projection strings never fail, e.g. new UnmanagedProjectionConverter("+proj=longlat +datum=WGS84 +no_defs", "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs")

Layer file path

• For consistency property pointing to the “main file” of any given layer could be the same across implementations. Now we have e.g. PathName in some vector based layers but in TAB it’s TabPathFilename and for rasters we have ImagePathFilename. PathName would work nicely for each.

[Ben]

Holy Moly! Haven’t seen such professional, informative and constructive feedback for years!!! We’ve logged those items in our tracking system and thanks for your support and understanding! Let us know when you need anything on the last 1% and thanks again for your support.

Ben

[Mikko_Viitala]

Hey @Ben

Thanks for asking. I might as well share that 1% here. I doubt those will be readily available by the end of the year when I take my leave but who knows when I start working with ThinkGeo products again.

First 0.25% is missing GeoPDF raster support.

Other 0.25% is poor 32-bit support (frequent crashes without warning when using large raster layers) but that’s on SkiaSharp.

The rest 0.50% is quite significant memory usage when multiple overlays are added to the MapView, no matter if these overlays are empty. Memory consumption seems to go down when overlay is hidden (which I didn’t consider at first since overlays were empty) but I still find creating overlays on-startup being better than creating overlays on-the-fly since creation takes some time.

I don’t have that many overlays and there’s only a few features to show on start up, still map consumes around 1.5Gb of (unmanaged) memory.

Thanks.

[Ben]

Hi Mikko,

Regarding GeoPdf, I think you meant Capital.pdf you provided in Problems opening GeoPDF layer and showing features from a WFS service - ThinkGeo UI for Desktop / WPF - ThinkGeo Discussion Forums. ThinkGeo is using GDAL for GeoPDF, and we found this particular file cannot be correctly opened in QGIS 3.16.11 on Windows (GDAL Version 3.3.1), but it can be opened in the same version (QGIS 3.16.11) on MacOS. I have a feeling GDAL has an issue on Windows dealing with GeoPDF. Do you think I can share your Capital.pdf to GDAL team to help them locate this issue? We can integrate the latest GDAL in after they fix it.

We saw that 32-bit collapse issue from SkiaSharp as well. We contacted SkiaSharp team and upgraded to the latest version (from 1.* to 2.*), some of the issues were fixed but this one still exists. Skia is a good cross platform rendering engine we don’t want to give up, and I thought 32-bit system is sunsetting, and a 32-bit app is easily to have memory related issue anyway and users should be aware of when loading large data, so even this problem is still bad I think most likely it will go away in the future when everyone uses X64.

The memory issue should be something new. We reviewed the memory usage in the early days of V12 and it looked good, it might be some recent changes that introduced some new memory issues. I am logging it in and will have a look.

Again thanks for your input, we’ll work on this last 1% and keep you posted.

Thanks!
Ben

[Mikko_Viitala]

Hey,

It’s getting quite late in Finland but I’ll provide you with another PDF tomorrow. I find some file that opens fine using ThinkGeo PDF layer and also in QGIS.

If I have few minutes to spare I’ll wrap up an example project regarding memory consumption, too. I also considered it to be fine when initially testing beta 13 but can’t say if it was due to some issue that was not present at the time or just my lazy evaluation on that regard (not putting enough load on the component).

What comes to 32-bit, yes, I agree. Those machines will be phased out eventually but in use cases similar to ours lifespan is expected to be quite long (XP-SP3-was-dropped-last-year long). But we’ll manage with this since problem is known so we can limit file sizes and make sure it comes up in training.

Thanks again, I’ll be back.

[Mikko_Viitala]

@Ben

Attached is a better PDF to work on, something I found from the internetz if I remember correctly.

Here’s how it looks in QGIS 3.18.3 Zürich when opened as vector and as raster on top of WMS service.

Then that same file opened using latest ThinkGeo beta, again on top of some WMS service. The latter image shows the said GeoPDF after it has been transformed into TIF file using gdal_translate.exe from OSGeo4W64.

Conversion used here is

from
"+proj=aea +lat_1=20 +lat_2=60 +lat_0=40 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs"

to
"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"

Here’s the file itself, zipped because of size limitation.
US_County_Populations.zip (2.5 MB)

And then the memory usage thing. Here’s example code using the latest binaries (WPF, .NET Framework 4.8).

If I only add an OpenStreetMapOverlay to the map and pan around, memory usage is around 100Mb…200Mb quite consistently, sometimes spiking to 300Mb for short periods of time.

mapView.MapUnit = GeographyUnit.Meter;
var osmOverlay = new OpenStreetMapOverlay();
mapView.Overlays.Add(osmOverlay);
mapView.Refresh();

If I add ten empty overlays and pan around, I easily hit 1.0Gb…1.2Gb, spiking up to 2Gb in memory usage.

mapView.MapUnit = GeographyUnit.Meter;
var osmOverlay = new OpenStreetMapOverlay();
mapView.Overlays.Add(osmOverlay);

for (int index = 0; index < 10; index++)
{
    var layer = new InMemoryFeatureLayer();
    var overlay = new LayerOverlay(new List<Layer> { layer });
    mapView.Overlays.Add(overlay);
}

mapView.Refresh();

[Ben]

Hi Mikko,

Thanks for the detailed info!

GDAL can open a GeoPDF in either Vector mode or Raster mode. GeoPdfFeatureLayer behind the scenes is opening a pdf using GDAL’s vector mode, in fact you can directly open a GeoPDF using GDAL’s raster mode like this: (with ThinkGeo.Gdal nuget package)

        GeoImage bitmap = new GeoImage(1024, 512);
        GdalRasterLayer layer = new GdalRasterLayer(@"GeoPdf\US_County_Populations.pdf");

        layer.Open();
        RectangleShape extent = layer.GetBoundingBox();
        GeoCanvas canvas = GeoCanvas.CreateDefaultGeoCanvas();
        canvas.BeginDrawing(bitmap, extent, GeographyUnit.Meter);
        layer.Draw(canvas, new Collection<SimpleCandidate>());
        canvas.EndDrawing();

        bitmap.Save($"GeoPdfTest.png"); 

Here below is what the result image look like, you can reproject it just like how you did it on a tiff layer. so you see there’s no need to convert a geoPdf to a tiff.

So can I say you don’t have issues in US_County_Populations.pdf? Capital.pdf (which cannot be opened correctly in QGIS Windows or ThinkGeo but can be opened in QGIS MacOS) is the only issue you have regarding GdoPDF?

Thanks,
Ben

[Mikko_Viitala]

Hey,

I have issues with some PDF files, that Capital.PDF included. I can open it as a raster in QGIS 3.18.3 Zürich but when trying to open as vector, I’ll receive an error.

I have just assumed that there’s something wrong with the file itself. But if you think it should open, also as vector source, then yes, maybe someone can take a look at it further.

And thanks for that piece of code on how to open PDF as raster. I tried it out and it worked perfectly.

Thanks a bunch.

[Ben]

Hi Mikko,

After a closer look, I think that Capital.pdf simply doesn’t include any vector data and we can only open it in raster mode, which reminds me we should probobly add a GeoPdfRasterLayer class so make it easier to be discovered for users.

We’ll check out the memory issue you mentioned and help you complete the last 1%.

Thanks,
Ben

[Mikko_Viitala]

Hey,

Anything new on this? I actually noticed that on older Win7 machine the memory consumption was OK. Results here are from my own laptop running Win10 and 48Gb RAM. So there’s that…

[Ben]

Hi Mikko,

I was pulled over to something else and one of my colleagues is working on it. We recreated it on Win 10 and Win 11. Thanks for the info about Win7, that’s interesting.

This is something we gonna fix in the next major release anyway. Things are going slow this couple weeks because of the holidays but I think we can have it fixed by sometime next month.

I’ll keep you posted and Happy Holidays!

Thanks,
Ben

[Mikko_Viitala]

Hey @Ben,

Happy holidays to you and your team, too.
One other thing I failed to mention is that on Win10 we used 64bit build and on Win7 32bit.

Br,
Mikko

[Ben]

Good to know, Thanks!

[Ben]

Hi Mikko,

The memory issue has been resolved in ThinkGeo.UI.WPF-Beta377, please have a try and let me know what you think. We will put it into the full 12.* version if everything goes well.

Thanks,
Ben

[Mikko_Viitala]

Hey Ben,

I tried the example I posted above with both the latest WPF & Core betas and also with WPF 370 & Core 309.

Based on my subjective view and extra fast map interactions I can’t see much of a difference in RAM usage with my PC setup. Then again I feel like the tiles loaded in smoother and fast panning didn’t freeze the map too much.

So, I didn’t encounter any problems, component seems to be in better state but it’s really hard to test it objectively.

Thanks.

[Ben]

Hi Mikko,

Here below is my test code:

    private void Map_Loaded(object sender, RoutedEventArgs e)
    {
        wpfMap1.MapUnit = GeographyUnit.Meter;

        OpenStreetMapOverlay baseOverlay = new OpenStreetMapOverlay(@"E:\worldstreets-11-11-2021.mbtiles");
        wpfMap1.CurrentExtent = new RectangleShape(-200000000, 200000000, 200000000, -200000000);
        wpfMap1.Overlays.Add(baseOverlay);

        for (int index = 0; index < 10; index++)
        {
            var layer = new InMemoryFeatureLayer();
            var overlay = new LayerOverlay();
            overlay.Layers.Add(layer);
            wpfMap1.Overlays.Add(overlay);
        }

        wpfMap1.Refresh();
    }

Run it in Visual Studio 2022 and keep zooming in/out, and here’s the memory usage I see with Beta-377:

And here is the memory usage with Beta-376:

However, I didn’t see the memory issue in Runtime even with Beta-376. Here is the Memory Usage from Task Manager after doing the same zooming in/out for the same period of time

I tested with both .NET 6.0 and .NET Framework 4.8 and the results are the same. Here below are the test projects, please have a try. They are using 376 and you should see the mem differences after upgrading to 377.
MemoryTest.zip (12.6 KB)

Please send me a demo which recreates your issue. I have a feeling we might not talking about the same memory issue here.

Thanks,
Ben

[Mikko_Viitala]

Hey Ben,

I’m using VS 2017 and only difference in my testing was that I used just var osmOverlay = new OpenStreetMapOverlay() and no initial extent set.

No matter what I do I can’t see big (or any?) difference in memory usage but 377 is still more responsive.

You see much better memory management, I see more responsive map… should we just leave it there or… ?

[Ben]

Hi Mikko,

I installed VS 2017 and didn’t see the issue with 376, I’m pretty sure it’s an issue in VS 2022.

What we did in 377 is to avoid creating new writeableBitmaps if possible, use the old ones instead. That must be where your responsiveness come from, even it doesn’t bring many improvements for you on the memory.

I created a ticket about this on visualstudio.com, I hope Microsoft can fix it soon:
https://developercommunity.visualstudio.com/t/WriteableBitmap-cannot-be-GC-collected-w/1626056?entry=myfeedback

Anyway, I think this means your last 0.5% issues have been fixed. Congratulations! and let me know if you have more issues.

Thanks,
Ben

[Mikko_Viitala]

Yes, everything seems to be good. Funny feeling when I can’t go about complaining about something but I might just get used to it Thanks.

[Mikko_Viitala]

And my moment came before long!

Although speed is fine and everything is working like a charm most of the time, I managed to have e.g. my adornment layer with scale bar to be duplicated across the map after few map view resizes. Those adornments stay where they are (see picture) even though tiles underneath change when panning/zooming.

Adornment should be only on bottom right.

Resizing and moving window from monitor to another might also cause infamous “can’t read memory, this is indication that other memory is corrupt” or something along those lines.