ThinkGeo.com    |     Documentation    |     Premium Support

Value filter styling is slow

#1

Hi,
We have a scenario where a large(ish) number of features can be styled based on the value of a column in an external data source, such as an Access DB or an Excel spreadsheet. What we have noticed is that this has got a lot slower in MapSuite 13 compared to 10.

We have worked quite a bit on optimising the CustomColumnFetch on our custom SQL Lite layer type and got some improvement. In so doing we noticed there is a very large increase in the number of calls to CustomColumnFetch between the versions and that is a significant driver in the drop off in performance.

Do you know why there is a big increase? and is there anything we could do about it?

Regards,
Jonathan

#2

Hi Jonathan,

Thanks for your post. Is it possible for you to send us a pared-down sample project showing how you’re using the CustomColumnFetch and we can take a look. If you don’t want to attach the sample here, you can email it to support@thinkgeo.com.

Thanks,
John

#3

Hi John,

I was going to give it a go, but it was difficult making a cutdown scenario as we have a lot of custom code. However we have done a lot of work on the method’s efficiency and better use of a dictionary style cache and that has improved things a lot.

We do not call this method ourselves directly, but it is associated with a layer by
layer.CustomColumnFetch += Pear_CustomColumnFetch( … );

And something significant about the difference between MapSuite 10 and 13 is that in 10, on a specific map change from zoomed in to full extent it gets called by ThinkGeo’s code approx. 12K times, but in the same scenario in 13 it get called approx. 104.5K times.

So it would be interesting to know what has changed to cause such a big increase?

Regards,
Jonathan

#4

Hi Jonathan,

I know a lot changed in our architecture and threading model between ThinkGeo 10 and 12. I will do some digging and see if I can come up with any conclusive reason and get back to you.

How many features and columns are in the layer that’s giving you trouble?

Thanks,
John

#5

Hi John,

The external data source has 4 columns, including the ID that links to the normal layer and the column that is used for the value filter style. The map layer itself (our own SQLite type) has just 2 columns, the linking column and a blank column.

All together there are 11,750 features on the layer, all of which have a matching record in the external data source (excel).

Regards,
Jonathan

#6

Hi Jonathan,

I just did a sample test using a ShapefileFeatureLayer with 73 features. The CustomColumnFetch gets called 73 times at full extend and as I zoom in, the CustomColumnFetch is called much less (once for every feature in the current extent).

Are you seeing the same strange behavior if you try a simple sample? Are you able to send us a sample project that demonstrates the issue?

Thanks,
John

#7

Hi John,

I think I have got to the nub of the problem. The setting DrawingMarginInPixcels seems to be key, and we had it set to a significantly larger than default value as we had problems with only partial labels being drawn when they crossed tile boundaries. That triggers a lot of extra calls to the custom column fetch.

I will send you a test project shortly.

All the best,
Jonathan

#8

Hi Jonathan,

Yep, you are right, it’s related to MultiTile, and DrawingMarginInPixcels will make the result different.

  1. The CustomColumnFetch will be raised exactly once per feature If the overlay.TileType is set to SingleTile.
    2, If the overlay.TileType is set to multiTile, that means some features will be shared by multiple tiles and the CustomColumnFetch in thoses features will be called multiple times. The number of the times that a feature’s event is being raised equals to the number of tiles that owns the feature.
  2. As a result, the greater DrawingMarginInPixcels is, the more features will be shared between different tiles so the event will be called more times. (The greater the “max” will be in your demo)

Let us know if you have any more questions.

Thanks,
Ben

#9

Thanks Ben, that is interesting but makes it difficult to find an effective compromise. If I use single tile then that probably is the most efficient but you loose the improved UI experience where you can see that the map is being redrawn and allows some actions, like panning. But if you go multi tile then with a small value for the margin we get a lot of text/labels not drawn fully but get too much slow down when it is set large.

Are tiles drawn on separate threads? if not could there be scope for caching so that only the first tile gets the value and subsequent tiles make use of it?

#10

Hi Jonathan,

You will love this. Please upgrade to the latest beta version (ThinkGeo.Core Beta040) and use overlay.TileType = TileType.PreloadDataMultiTile; It will allow you to render in multi tiles while only loading the data once . This type had issues in the previous version so please make sure uploading all the ThinkGeo packages to the latest.

You will see “max” in the demo greater than 1 when zooming the map before it finished rendering, because one feature might be rendered twice before clearing up the counter. Other than that, each feature’s fetch event will only be called once.

We will send you an updated demo from support@thinkgeo.com right away.

Thanks,
Ben

#11

Email just sent out.

#12

Hi Ben,
Thanks for the updated code. The example works well and the PreloadDataMultiTile was very good, however I am having trouble transferring it to the full project, but I have not pinpointed why yet.

In the main code a tab shows a data grid of data from the external source and this loads in a separate thread to the main application. It looks like the tiles get drawn and then disappear when the data finishes loading (doesn’t happen in single or multi tile). I know the styling cannot be worked out until after the data is available.

I will try and work out what triggers the clearance.

Actually single title does seem to work best in many instances due to labels / text only being partially displayed when zoomed in under multi tile.

Thanks,
Jonathan

#13

Hi Jonathan,

In MultiTile, each tile handles data loading and map rendering “independently” while in PreloadData mode, the data isloaded in one thread but rendered in different threads, it could have some issues if other threads are involved in the application. It will be great If you can send me a sample to recreate this issue, I’m sure we can make it right.

The partially labels issue can be fixed by DrawingMarginInPixcels, or another way is to put the label layer in a SingleTile layerOverlay while put the other layers to another MultiTile Overlay, In this way you don’t need to worry about DrawingMarginInPixels.

Thanks,
Ben

#14

Hi Ben,

Putting the labels on a specific overlay could work nicely I think. I will give it a try.

In the meantime should I be switching this to a ticket? It has escalated quite a bit since the initial question.

Regards,
Jonathan

#15

That might be better :grinning: