ThinkGeo.com    |     Documentation    |     Premium Support

VERY LARGE vector data set: what is the best format for performance?

#1

I have been using MapSuite WPF Desktop for some time now but would like some advice on what is the best way to handle large amounts of vector data. We have tried MsSqlFeatureLayer but customers have complained that getting their data into this format takes a long, long time (sometimes several days :rage: ). The data in question is Ordnance Survey MasterMap vector data for Great Britain (see here for more information on MasterMap). Typically we have been using SQL Express to store the data but we are hitting the 10GB database limit quite often. We are hoping to use ESRI Shape files but we are concerned about the large number of SHP files that need to be referenced, e.g. 900+, and the need to generate a ThinkGeo index file for each of them.

Is there a way to handle large numbers of shape files in a layer in a similar way to the magnificent MultiGeoRasterLayer which we have used to great effect?

Thanks in advance to any responders :+1:

#2

Hi John,

The MultiGeoRasterLayer can works is because for raster layer, you don’t need to set style for each file, just need to add it into map. But for vector layer, you have to set style for different types data, for example, some data is area, the other is line and the rest is point style.

So if you want to load a group of vector shape file, you have to know what’s the style of them should be.

If you know all the files use the same style, for example, all points want to be rendered like a same circle, so you can write a custom function which works like MultiGeoRasterLayer.

It’s simple and the logic like this:

  1. Get the file folder, pick all files which is .shp
  2. Build index with parameter DoNotRebuild
  3. Set DefaultAreaStyle, DefaultLineStyle and DefaultPointStyle for each layer, and apply the style from zoomlevel 01 to zoomlevel 20
  4. Add the layer to your overlay
  5. Expand current extent for showing the new layer

Then you can load many files by the simple function and render them.

Open cache can make render speed up.

Regards,

Ethan

#3

Hi Ethan

Thanks for your response :+1:

I was thinking of using the class ThinkGeo.MapSuite.Layers.MultipleShapeFileFeatureLayer which I found after looking through the API documentation. Would you recommend this for many SHP files or do you think I should develop a custom class, like MultiGeoRasterLayer, as you suggested?

Thanks
John

#4

Hi John,

You’re right, I forget the MultipleShapeFileFeatureLayer class because I hadn’t heard it for long time, this layer is mainly used in our WorldMapKit before, it can handle a group of layers which have nearly the same name for example: “xxx1.shp” “xxx2.shp” “xxx3.shp”.

I viewed it’s code and found it should also works for your scenario. You can just choose this constructor: public MultipleShapeFileFeatureLayer(IEnumerable shapeFiles)

Please try to use it, this layer read the files by deeper API so it should get better performance. If you met problem when use it then you can try my solution to build a custom class, it’s also easy to build.

Regards,

Ethan