I am building some rather large multipolygon shapes and it looks like I hit a limit on how many polygons can be contained. Can you advice if this is expected or a bug? Here’s a snipit that produces the behavior. It does not fail at the same location every time, so I would like to know how I can anticipate the limit so I can break the shapes up accordingly.
MultipolygonShape mpg = new MultipolygonShape();
for (int i = 0; i < 27000000; i++)
{
RingShape ring = new RingShape();
ring.Vertices.Add(new Vertex(0, 0));
ring.Vertices.Add(new Vertex(0, 5));
ring.Vertices.Add(new Vertex(5, 5));
ring.Vertices.Add(new Vertex(5, 0));
PolygonShape pg = new PolygonShape(ring);
mpg.Polygons.Add(pg);
}
I am adding this layer to an InMemoryFeatureLayer and then using the Draw method to save to a bitmap image.
I got around the polygon memory issue by adding additional features that were subsets of my original polygon set, but when calling the draw command, I get the following error:
I tested it on my computer whose memory size is 16GB, the limit are the following, if your application is designed to run on the computers with 16GB memory, I think it’s safe to use 16,000,000 as the limit. If you want to run your application to the low-memory machines, you need to reduce it to a small number, like 100,000 or 5,000 to make sure it won’t break.
The reason why it didn’t fail at the same location every time is that the memory was being used dynamically, other apps would occupy memory also.
16510612
16258894
16426681
16594466
16678362
16262379
16346276
16597952
16597950
16263539
Regarding the “Arithmetic operation resulted in an overflow.” error, could you please post the code snippet or the sample to help us reproduce it?
Here’s the code where the arithmetic error happens. Specifically at the Draw function.
Regarding the number of objects, I find it odd that I don’t experience any peaks in memory usage prior to the MultipolygonShape error. I have 64 Gb of Ram on my machine and the failures were very near to your reported 16M with no other “sizeable” programs running. I think it is something else.
using (Bitmap bitmap = new Bitmap(w, h))
{
PlatformGeoCanvas canvas = new PlatformGeoCanvas();
canvas.BeginDrawing(bitmap, rect, GeographyUnit.Meter);
imfl.Draw(canvas, new Collection<SimpleCandidate>());
canvas.EndDrawing();
bitmap.Save(filename, System.Drawing.Imaging.ImageFormat.Png);
}
Here is the full code minus the part about adding the polygons to the allBins object since that is a data specific process. Essentially, the allBins object has a double value which is used to color code the individual polygons with a class break style. The polygons are all rectangles that create a map. So, you could generically make some code that took a rectangle shape and translated it in rows and columns and randomly assign to values between 0 and 10. The rows times the columns should be greater than 16M. In my specific case, I had 27M cells.
Please look at the memory usage on your machine when you add polygons to the InMemoryFeatureLayer and see if the memory is actually being totally consumed. As I said, mine is not and the maxpg segmentation gets me past the out of memory error…
ConcurrentDictionary<double, MultipolygonShape> allBins = new ConcurrentDictionary<double, MultipolygonShape>();
int maxpg = 15000000;
foreach (var pg in allBins)
{
if (pg.Value.Polygons.Count == 0) continue;
if (pg.Value.Polygons.Count > maxpg)
{
int fc = Convert.ToInt32(Math.Ceiling(pg.Value.Polygons.Count / (float)maxpg));
for (int i = 0; i < fc; i++)
{
List<PolygonShape> pgs = pg.Value.Polygons.Skip(i * maxpg).Take(maxpg).ToList();
MultipolygonShape mpg = new MultipolygonShape(pgs);
imfl.InternalFeatures.Add(new Feature(mpg, new List<string>() { offsetName + ":" + pg.Key }));
}
}
else
imfl.InternalFeatures.Add(new Feature(pg.Value, new List<string>() { offsetName + ":" + pg.Key }));
}
Debug.Print(string.Format("Adding bins to imfl took {0:0.0} s", s.Elapsed.Seconds));
s.Restart();
imfl.Open();
RectangleShape rect = imfl.GetBoundingBox();
// Have to set map size first otherwise it is taking forever if the data are already loaded
double mapX = rect.LowerRightPoint.X - rect.LowerLeftPoint.X;
double minBin = model.GridDef.BinX;
if (model.GridDef.BinY < model.GridDef.BinX)
minBin = model.GridDef.BinY;
double ratio = 16 / 9f;
ratio = rect.Width / rect.Height;
int w = 5120;
int h = Convert.ToInt32(w / ratio);
double pixelsPerBin = minBin * w / mapX;
while (pixelsPerBin < 4)
{
w += 100;
h = Convert.ToInt32(w / ratio);
double imageSize = w * h * 4f;
if (imageSize >= 2.147483648e9)
{
w -= 100;
h = Convert.ToInt32(w / ratio);
break;
}
pixelsPerBin = minBin * w / mapX;
}
// Bitmap can't exceed 65535 x 65535
if (h >= 65535)
{
h = 65535;
w = Convert.ToInt32(h * ratio);
}
Debug.Print(string.Format("Building map took {0:0.0} s", s.Elapsed.Seconds));
s.Restart();
using (Bitmap bitmap = new Bitmap(w, h))
{
PlatformGeoCanvas canvas = new PlatformGeoCanvas();
canvas.BeginDrawing(bitmap, rect, GeographyUnit.Meter);
imfl.Draw(canvas, new Collection<SimpleCandidate>());
canvas.EndDrawing();
bitmap.Save(filename, System.Drawing.Imaging.ImageFormat.Png);
}
Here’s the sample I created according to your code, but regarding the adding logic, I’m not sure if I understand you correctly, Could you please take a look at the sample, correct the code if it’s wrong, and send it to us?
This issue is recreated and fixed in the latest beta version ThinkGeo.MapSuite 11.0.0-beta154 . The cause of this issue is some intermediate variable exceeds the int limit because of the large size of the polygon.
Please get that version and have a try (you don’t need to update the other packages), Let us know if everything goes well and we will move it over to the release version 10...
Please let me know when you have it rolled out into v10. Will that be imminent or some days in the future. I have been holding a release based on this issue.
