ThinkGeo.com    |     Documentation    |     Premium Support

WMK data extractor not take mbtiles input

#1

Hi there,

The new WMK data we received is in mbtiles format. But WMK data extractor can not take sqlite format.

I tried to rename the input file from worldstreets.mbtiles to workdstreets.mbtiles.sqlite but it generates an exception.

image.png1712×971 165 KB

Please take a look,

Liang

#2

Hi Liang,

The old extractor don’t support new data format, our developer is working on it, after complete it we will upload it on GitHub.

Regards,

Ethan

#3

Hi Liang,

Here is our new sample https://github.com/ThinkGeo/MBTilesExtractorSample-ForWpf

Please view it and wish that’s helpful.

Regards,

Ethan

#4

Hi Ethan,

The sample gave me SQL exceptions unless I changed the sql strings:
string querySql = $“SELECT * FROM {sourceDB.Map.TableName} WHERE " + ConvetToSqlString(tileRange) + $” LIMIT {offset},{offset + 1000}";
var entries = sourceDB.Map.Query(querySql);
targetDB.Map.Insert(entries);

                querySql = $"SELECT images.tile_data as tile_data, images.tile_id as tile_id FROM {sourceDB.Images.TableName} WHERE images.tile_id IN ( SELECT {Map.TileIdColumnName} FROM {sourceDB.Map.TableName} WHERE " + ConvetToSqlString(tileRange) + ")";

The map we got from Thinkgeo is a world streets map of size 59GB. How do I modify the example to use it as source?

Thank you.

Liang

#5

Thanks Liang for pointing out the issue! We’ve fixed the issue on GitHub.

You can change the sample code and launch the global mbtiles instead of the Frisco one in the sample, zoom to the area you interested and then draw a rectangle to start clipping. Drawing the mbtiles will not take much time, of course extracting will take a while depending the size you cut it.

Ben

#6

Hi Ben,

Do I need to change other settings? I tried replace local DB with worldstreets.mbtiles but got exceptions.

//ThinkGeoMBTilesFeatureLayer thinkGeoMBTilesFeatureLayer = new ThinkGeoMBTilesFeatureLayer(“Data/tiles_Frisco.mbtiles”, new Uri(“Data/thinkgeo-world-streets-light.json”, UriKind.Relative));
ThinkGeoMBTilesFeatureLayer thinkGeoMBTilesFeatureLayer = new ThinkGeoMBTilesFeatureLayer(“Data/worldstreets.mbtiles”, new Uri(“Data/thinkgeo-world-streets-light.json”, UriKind.Relative));

image.png1420×804 66.5 KB

Thanks,

Liang

#7

Liang,

It launches the map correctly for me. Can you check the size of the map file and make sure it’s not corrupted, it should be 60,474,036,224 bytes. Also, I see the same issue once I input the wrong mbtiles file name, please make sure the name is correct.

Ben

#8

Hi Ben,

My map file size is correct at 56.3 GB (60,474,036,224 bytes).

I didn’t change MbtilesDatabase sourceDB = MbtilesDatabase.OpenDatabase(“Data/tiles_Frisco.mbtiles”);
to worldstreets.mbtiles.

After changing it the window shows up. But it takes forever to show a whole map of US for I want to generate a map of Illinois. When I checked Extract Vector Tiles, it hit an exception.

image.png1010×761 112 KB

Thanks,

Liang

#9

I can zoom out to the US without a problem. Once you draw a rectangle and start the clipping, the map will display a bit weird though. Can you start from clipping a small area for test, or you prefer us to generate IL for you?

#10

Ben,

What could be the reason that I could hardly see US? Does the program need connection to your server to display map?

Select a small area near Frisco game the same exception “{“SQL logic error or missing database\r\nno such table: map”}”

We would like to have the tool working so that we can generate customerized map data.

Thanks,

Liang

#11

Liang,

It works fine on my side. It doesn’t connect to the server, everything is offline.

Can you shoot an email to sales@thinkgeo.com and let us know a good time for a meeting? Let’s meet and get to the bottom of it.

Thanks,
Ben

#12

Hi Ben,

Thank you for showing me how to extract map properly today.

I tried to extract Illionois map this afternoon. I think something went wrong. After 30 minutes it was still running, the file size grew to more than 1.5 GB. It seems there is an infinite loop somewhere.

Can you please try to see if you can reproduce?

Thanks,

Liang

#13

Hi Liang,

We fixed the infinite loop issue. Please fetch the newest code from https://github.com/ThinkGeo/MBTilesExtractorSample-ForWpf.

Thanks,
Rex

#14

Hi Rex,

I downloaded the new code and was able to generate Illinois maps quickly within 5 minutes. The map is around 280 to 332 MB.

But when I tried to use the map, i.e., set DB to the newly generated map, with CurrentExtent as (-10970065.1793334, 5536525.3190895, -8621919.68345808, 4292742.00174302), the map did not show as expected.

The Illinois map I generated is at https://www.dropbox.com/s/s300bg8oej6gian/Illinois.mbtiles?dl=0

Can you please take a look?

Thanks,

Liang

#15

Hi Liang,

Sorry now we cannot access DropBox or GoogleDrive succeed, could you please describe what’s the problem you met? You mentioned the result is not shows as expected, I think some screen shots and description about it should be helpful.

And we can build that in our side, so if we know your setting we can did the same test like yours.

Regards,

Ethan

#16

Hi Ethan,

I used world map as source, zoomed on and generated map for Illinois

image.png1010×761 237 KB

Illinois3.mbtiles was generated in a few minutes. It was 233MB.

image.png1010×761 325 KB

Now I changed the program to use Illinois3.mbtiles as source, changed current extent to (-10970065.1793334, 5536525.3190895, -8621919.68345808, 4292742.00174302).
It took very long o load. When I zoomed in, nothing appeared.

image.png1920×1040 29.2 KB

Please take a look,

Liang

#17

Hi Liang,

Thanks to let us know it.

The reason of that is when the sample build the new mbtils, it forget build index.

You can add this code into the logic when create new mbtiles file, or you can just run it once to build index for your existing mbtiles and it should also works.

            MbtilesDatabase sourceDB = MbtilesDatabase.OpenDatabase(@"D:\Illinois3.mbtiles");
        string querySql = "CREATE INDEX tile_id_index ON images(tile_id)";
        sourceDB.Images.Query(querySql);

Here is my test result, it’s fast after build index.

image.png691×49 7.68 KB

image.png1886×1033 476 KB

Regards,

Ethan

#18

Hi Ethan,

I added the 3 lines at the end of function
private void ExtractTiles(RectangleShape bbox, string targetFilePath)
{
…
sourceDB = MbtilesDatabase.OpenDatabase(targetFilePath);
string querySql1 = “CREATE INDEX tile_id_index ON images(tile_id)”;
sourceDB.Images.Query(querySql1);
}

Is this correct? It does seem to work.

Thanks,

Liang

#19

Hi Liang,

We will add this line to our sample later.

For now you just need to run it once, and then the index will be built.

I am glad to hear that works.

Regards,

Ethan