ThinkGeo.com    |     Blog    |     Wiki    |     Support

WMK data extractor not take mbtiles input

Hi there,

The new WMK data we received is in mbtiles format. But WMK data extractor can not take sqlite format.

I tried to rename the input file from worldstreets.mbtiles to workdstreets.mbtiles.sqlite but it generates an exception.

Please take a look,

Liang

Hi Liang,

The old extractor don’t support new data format, our developer is working on it, after complete it we will upload it on GitHub.

Regards,

Ethan

Hi Liang,

Here is our new sample https://github.com/ThinkGeo/MBTilesExtractorSample-ForWpf

Please view it and wish that’s helpful.

Regards,

Ethan

Hi Ethan,

The sample gave me SQL exceptions unless I changed the sql strings:
string querySql = $“SELECT * FROM {sourceDB.Map.TableName} WHERE " + ConvetToSqlString(tileRange) + $” LIMIT {offset},{offset + 1000}";
var entries = sourceDB.Map.Query(querySql);
targetDB.Map.Insert(entries);

                querySql = $"SELECT images.tile_data as tile_data, images.tile_id as tile_id FROM {sourceDB.Images.TableName} WHERE images.tile_id IN ( SELECT {Map.TileIdColumnName} FROM {sourceDB.Map.TableName} WHERE " + ConvetToSqlString(tileRange) + ")";

The map we got from Thinkgeo is a world streets map of size 59GB. How do I modify the example to use it as source?

Thank you.

Liang

Thanks Liang for pointing out the issue! We’ve fixed the issue on GitHub.

You can change the sample code and launch the global mbtiles instead of the Frisco one in the sample, zoom to the area you interested and then draw a rectangle to start clipping. Drawing the mbtiles will not take much time, of course extracting will take a while depending the size you cut it.

Ben

Hi Ben,

Do I need to change other settings? I tried replace local DB with worldstreets.mbtiles but got exceptions.

//ThinkGeoMBTilesFeatureLayer thinkGeoMBTilesFeatureLayer = new ThinkGeoMBTilesFeatureLayer(“Data/tiles_Frisco.mbtiles”, new Uri(“Data/thinkgeo-world-streets-light.json”, UriKind.Relative));
ThinkGeoMBTilesFeatureLayer thinkGeoMBTilesFeatureLayer = new ThinkGeoMBTilesFeatureLayer(“Data/worldstreets.mbtiles”, new Uri(“Data/thinkgeo-world-streets-light.json”, UriKind.Relative));

Thanks,

Liang

Liang,

It launches the map correctly for me. Can you check the size of the map file and make sure it’s not corrupted, it should be 60,474,036,224 bytes. Also, I see the same issue once I input the wrong mbtiles file name, please make sure the name is correct.

Ben

Hi Ben,

My map file size is correct at 56.3 GB (60,474,036,224 bytes).

I didn’t change MbtilesDatabase sourceDB = MbtilesDatabase.OpenDatabase(“Data/tiles_Frisco.mbtiles”);
to worldstreets.mbtiles.

After changing it the window shows up. But it takes forever to show a whole map of US for I want to generate a map of Illinois. When I checked Extract Vector Tiles, it hit an exception.

Thanks,

Liang

I can zoom out to the US without a problem. Once you draw a rectangle and start the clipping, the map will display a bit weird though. Can you start from clipping a small area for test, or you prefer us to generate IL for you?

Ben,

What could be the reason that I could hardly see US? Does the program need connection to your server to display map?

Select a small area near Frisco game the same exception “{“SQL logic error or missing database\r\nno such table: map”}”

We would like to have the tool working so that we can generate customerized map data.

Thanks,

Liang

Liang,

It works fine on my side. It doesn’t connect to the server, everything is offline.

Can you shoot an email to sales@thinkgeo.com and let us know a good time for a meeting? Let’s meet and get to the bottom of it.

Thanks,
Ben

Hi Ben,

Thank you for showing me how to extract map properly today.

I tried to extract Illionois map this afternoon. I think something went wrong. After 30 minutes it was still running, the file size grew to more than 1.5 GB. It seems there is an infinite loop somewhere.

Can you please try to see if you can reproduce?

Thanks,

Liang

Hi Liang,

We fixed the infinite loop issue. Please fetch the newest code from https://github.com/ThinkGeo/MBTilesExtractorSample-ForWpf.

Thanks,
Rex

Hi Rex,

I downloaded the new code and was able to generate Illinois maps quickly within 5 minutes. The map is around 280 to 332 MB.

But when I tried to use the map, i.e., set DB to the newly generated map, with CurrentExtent as (-10970065.1793334, 5536525.3190895, -8621919.68345808, 4292742.00174302), the map did not show as expected.

The Illinois map I generated is at https://www.dropbox.com/s/s300bg8oej6gian/Illinois.mbtiles?dl=0

Can you please take a look?

Thanks,

Liang

Hi Liang,

Sorry now we cannot access DropBox or GoogleDrive succeed, could you please describe what’s the problem you met? You mentioned the result is not shows as expected, I think some screen shots and description about it should be helpful.

And we can build that in our side, so if we know your setting we can did the same test like yours.

Regards,

Ethan

Hi Ethan,

I used world map as source, zoomed on and generated map for Illinois

Illinois3.mbtiles was generated in a few minutes. It was 233MB.

Now I changed the program to use Illinois3.mbtiles as source, changed current extent to (-10970065.1793334, 5536525.3190895, -8621919.68345808, 4292742.00174302).
It took very long o load. When I zoomed in, nothing appeared.

Please take a look,

Liang

Hi Liang,

Thanks to let us know it.

The reason of that is when the sample build the new mbtils, it forget build index.

You can add this code into the logic when create new mbtiles file, or you can just run it once to build index for your existing mbtiles and it should also works.

            MbtilesDatabase sourceDB = MbtilesDatabase.OpenDatabase(@"D:\Illinois3.mbtiles");
        string querySql = "CREATE INDEX tile_id_index ON images(tile_id)";
        sourceDB.Images.Query(querySql);

Here is my test result, it’s fast after build index.

Regards,

Ethan

Hi Ethan,

I added the 3 lines at the end of function
private void ExtractTiles(RectangleShape bbox, string targetFilePath)
{

sourceDB = MbtilesDatabase.OpenDatabase(targetFilePath);
string querySql1 = “CREATE INDEX tile_id_index ON images(tile_id)”;
sourceDB.Images.Query(querySql1);
}

Is this correct? It does seem to work.

Thanks,

Liang

Hi Liang,

We will add this line to our sample later.

For now you just need to run it once, and then the index will be built.

I am glad to hear that works.

Regards,

Ethan