WMS Server Image Caching

[Dennis]

MapSuite Team,

I'd like to determine if caching of a WmsRasterLayer within the WMS Server would be beneficial.

Does MapSuite handle WMS Cache Headers? One issue about caching is that the cache has to age out. It cannot remain indefinitely, but rather age out after a pre-determined time. The third-party WMS Provider can install new imagery at any time and we don't have to be aware or do anything special. If caching does not automatically age out the system would use old cached images instead of new imagery.

What information can you provide about caching and a code snippet of how to enable it on the server?

Thanks for your assistance.
Dennis
OriStar Mapping, Inc.

[Ethan]

Hi Dennis,

You can set the "ExpirationTime" for FileBitMapTileCache, which can check the cache created time, if it's expired, it will be updated when access it again.

The other solution is you can write a windows service to handle the cached images, the logic is the same.

Your service run a fixed interval of time, it will loop all files in cached folder, check the created time of image file and currently time, if it's longer than your set, just delete it.

Wish that's helpful.

Regards,

Ethan

[Dennis]

hi Ethan,

Thanks for the information.

I've done some initial development for caching and have found that it does negate requests to the third-party WMS Server and does improve response time.

However, the ExpirationTime and refresh is not behaving as I had expected and as I believe you had stated in your post. I started by setting ExpirationTime to 5 minutes. The first time a request was made the cache tiles were created. Within that initial 5 minute time span each time I went back to an extent that was cached it was the cached tiles that were used, which is correct. However, once the 5 minutes had expired the cached tiles are refreshed with a request to the server, but the 5 minute expiration no longer works. Once the tiles are refreshed they continue to be refreshed each time that extent is requested even though they had been refreshed less than 5 minutes ago.

Operating like this is like there is no caching at all because after the initial ExpirationTime is complete all extents make a request to the third-party WMS Server and refresh the cache.

This can't possibly be the way it was intended to work. From my perspective the ExpirationTime should reset after each refresh. Below is a screen capture of the times of one of the tiles. I believe what is happening here is that MapSuite is looking at Created Time instead of Modified Time. MapSuite should either reset Created Time after a refresh or it should be checking Modified Time.

Does MapSuite have some other design intention for how it is operating?

Thanks,
Dennis

[Ethan]

Hi Dennis,

Our developer checked the code, it looks our logic is in the GetTileCore function, check whether the tile image is expired, if it's expired, it will delete it.

So please check your logic to see whether you override the GetTileCore to cover this delete logic.

Regards,

Ethan

[Dennis]

hi Ethan,

I've not implemented an override of GetTileCore function.

What I stated in my original post is happening. During testing I have the map displayed and Windows Explorer displayed and opened to the directory where the tiles are stored. When the map is moved to that extent I can see all the tiles in said directory being removed and then added back. However, the Created Date on the refreshed tiles remains the same as the original set of tiles as evidenced in the screen capture in my original post.

The issue is that MapSuite is not setting the Creation Date to the current date/time during the refresh process. Instead, it leaves the Creation Date equal to the date/time of when the tile was originally created. This results in the tiles being refreshed for every movement to said extent as I stated in my original post.

Would you please ask the developer to check how Creation Date is set?

Thanks,
Dennis
OriStar Mapping, Inc.

[Ethan]

Hi Dennis,

I get reply from our developer, it looks we delete the expired tile image in GetTileCore, and saved the tile image in SaveTileCore if it's not exist.

We don't have the logic to modify the tile image.

And I want to double check whether your cache is handled by the WmsServiceEdition and FileBitmapTileCache.

For make sure that, I think maybe you can try to set the TileAccessMode equal ReadOnly, and watch whether the tile image get deleted succeed when it's expired and GetTileCore get called again.

Because each time when the server get a request, it will go follow this steps, so I think the behavior should be the same each time it get expired.

Regards,

Ethan

[Dennis]

hi Ethan,

The project uses WmsServerEdition V9.0.0.856. Below is the code to incorporate caching.

I've performed the following test scenario...

1. The existing cache directory was manually deleted. The next step was to set TileAccessMode=TileAccessMode.ReadOnly. Under this scenario the cache is actually not even created.

2. Set TileAccessMode=TileAccessMode.ReadAddDelete. Under this scenario the cache is created. The expiration time of 5 minutes was allowed to expire.

3. Set TileAccessMode=TileAccessMode.ReadOnly. Under this scenario when the extent is changed to correspond to the cache the cache tiles were all deleted.

It is certainly odd to me that under Step 3 the tiles were deleted even though ReadOnly access is specified.

There is clearly an issue here without a doubt.

I would implore you to ask the developer to check Creation Date and how is it set. This is where the problem is. Or like I stated in one of my previous posts MapSuite needs to be checking Modified Date instead of Creation Date.

Regards,
Dennis

OriStar Mapping, Inc.

        string TheCachedTileDirectory;
        TimeSpan TheTimeSpan;
        FileBitmapTileCache TheTileCache;

        TheTimeSpan = new TimeSpan(0, 5, 0);

        TheTileCache                = new FileBitmapTileCache();

        TheCachedTileDirectory = string.Format("{0}\\{1}", "C:\\MapCachedTilesWms", "TheWmsServerCache");
        TheTileCache.CacheDirectory = TheCachedTileDirectory;
        TheTileCache.CacheId        = "MyLayerName";
        TheTileCache.ExpirationTime = TheTimeSpan;
        TheTileCache.TileAccessMode = TileAccessMode.ReadAddDelete;

        oWmsLayerActive.MapConfiguration.TileCache = TheTileCache;

[Ethan]

Hi Dennis,

In fact the expired logic is simple, I viewed the code also about it.

1. The TileAccessMode only handle the SaveTileCore, if it's "ReadOnly" the SaveTileCore won't delete or modify the tile image.

2. The TileAccessMode cannot constrain the behavior of "ExpirationTime", if the tile image is expired, and map tried to read the tile image, the expired image will get deleted. About whether this API is reasonable, I will asked our developer discuss about it, let's just ignore it in this question.

3. For the tile images, the Modified Date should be the same like Creation Date, because our logic won't modify it, the modify is always the "create new after delete old". So your scenario should have some special logic to block the delete process, that's why your Modified Date is different to Creation Date.

And I asked you test "ReadOnly" is want to see whether the first "Delete" step works, it looks the delete after expired works here, so that's strange.

Today you mentioned you are using 9.0.0.x, could you please did a quickly test for 9.0.x.0 to see whether it's the same behavior that the expired time don't works again after it's deleted the first time?

Our developer also need to do some deeper research about the logic.

If I missed any information please let me know.

Regards,

Ethan

[Dennis]

hi Ethan,

Regarding your Item #1 -- As I stated in my previous post the tiles get deleted even though ReadOnly is specified.

Regarding your Item #3 -- The code I previously posted is the only code for caching that my application has. I am totally convinced there is an issue in MapSuite with not properly setting Creation Date as I have explained previously.

At first opportunity I will download the latest WMS Server build and test.

Please let me know what your developer finds after doing deeper research.

Regards,
Dennis

[Ethan]

Hi Dennis,

Any update I will let you know.

Regards,

Ethan

[Ethan]

Hi Dennis,

Our developer did further research, we double check the logic and build sample try to reproduce it about the ExpirationTime don't works after the first time. It looks everything goes well, so we think that's maybe related with your other code implementation.

WmsSample.zip (511.4 KB)

If you have more information about it please let us know.

Regards,

Ethan

[Dennis]

hi Ethan,

The code that I previously posted is the only code I have for caching. Is this code correct? It's the same code that's in the example project you provided.

Are your tests under Windows 7 or Windows 10? I develop under Windows 10.

I am 100% certain this is not an issue within my application.

I will upgrade to the latest WMS Server deployment build and see if this bug has been fixed since version V9.0.0.856.

Regards,
Dennis
OriStar Mapping, Inc.

[Ethan]

Hi Dennis,

We tested that in win10, and we test it with latest dll and the specified dll 9.0.0.856.

Do you think we missed anything? Have you tried to run our sample to see whether it also can reproduce this problem in your machine?

Regards,

Ethan

[Dennis]

hi Ethan,

I've updated to WmsServerEdition V9.0.0.1096 and still have this issue.

I repeat - MapSuite is not updating the Creation Date on the cached images when those cached tiles are refreshed. This is the issue.

I have not yet tried to run your sample, but I will.

The only difference that I can see between your sample and my code is that your sample uses WmsOverlay where my application uses WmsRasterLayer.

Regards,
Dennis

[Ethan]

Hi Dennis,

It looks the cache is in your transfer sever (WmsServiceEdition), and for this edition it should only contains the WmsRasterLayer but without the WmsOverlay.

Maybe we confuse something?

Regards,

Ethan

[Dennis]

Ethan,

I was merely pointing out that your client example uses WmsOverlay whereas my client uses WmsRasterLayer as a possible problem area.

In my posts I have mentioned Creation Date not being updated upon tile refresh. Would you please respond concerning Creation Date?

I setup a test where tiles were cached for two different extents. The expiration time of five minutes then elapsed. I then used Windows Explorer to manually delete the cached tiles on one of the extents. I then positioned the map client to said extent and the tiles were refreshed with the current date as the Creation Date.

When MapSuite does a tile refresh, on an existing set of tiles, the Accessed & Modified dates are updated correctly, but the Creation Date stays as the original date. Please speak to the fact that MapSuite, upon tile cache delete & refresh, does not update the Creation Date.

I just did some research and found this article.

After deleting file and re-creating file, not change creation date in windows

Based on this article it would seem to me that MapSuite needs to be specifically setting the Creation Date after a tile refresh. Or, as previously suggested, use Modified Date to determine expiration/refresh interval.

Here's what Microsoft has to say about replacing files...
https://msdn.microsoft.com/en-us/library/system.io.file.setcreationtime(v=vs.110).aspx

Your thoughts?

Regards,
Dennis
OriStar Mapping, Inc.

[Ethan]

Hi Dennis,

Thanks for your thinking.

I read the topic and find some related topic for example: https://blogs.msdn.microsoft.com/oldnewthing/20050715-14/?p=34923

And I think we hadn't reproduced it is because the OS version reason.

Our developer discuss about the solution and decide to modified the logic to check the modify time of tile image, please get that when NuGet update the package: https://www.nuget.org/packages/ThinkGeo.MapSuite/11.0.0-beta069

Please let us know whether it solved this problem in your end.

Regards,

Ethan

[Dennis]

Ethan,

I'm using WmsServerEdition V9.0.0.1096. How does nuget work with V9? I had expected to download a new V9 using V9 Product Center.

Dennis

[Ethan]

Hi Dennis,

Please latest package which will be updated today or tomorrow.

And let us know whether it works for you.

Regards

Ethan

[Dennis]

hi Ethan,

My application is now compiled with WmsServerEdition V9.0.0.1103 and cache expiration/refresh is now working.

Thanks for your support!

Dennis
OriStar Mapping, Inc.