ThinkGeo.com    |     Documentation    |     Premium Support

Advice needed about performance!

#1

Hello,


 


I need to colorize the US zip codes based on sales reps assigned per zip code.


I have my sales reps table about 42 entries and I have the shapfile (ZIP5) from the US census.


The code I am using is provided.  


My issue is that kind of process is slow very slow......  The ZIP shapfile is on a RAM drive.


My server is a physical one with 24 cores and 80GB of memory.  I can see the process running and when I am computing


the zip unions the server is only busy at 3/4%......  I would like to see my server busy at 100%!


Do I have miss something?


thanks a lot.


 


 InMemoryFeatureLayer inMemoryFeatureLayer_Zips_Colorization_union = new InMemoryFeatureLayer();

                inMemoryFeatureLayer_Zips_Colorization_union.Name = "inMemoryFeatureLayer_Zips_Colorization";

                inMemoryFeatureLayer_Zips_Colorization_union.IsVisible = true;

                inMemoryFeatureLayer_Zips_Colorization.ZoomLevelSet.ZoomLevel01.CustomStyles.Add(valueStyleRep);

                inMemoryFeatureLayer_Zips_Colorization_union.ZoomLevelSet.ZoomLevel01.ApplyUntilZoomLevel = ApplyUntilZoomLevel.Level20;



                foreach (DictionaryEntry s in _repNameList)

                {

                    inMemoryFeatureLayer_Zips_Colorization.FeatureSource.Open();

                    Collection<Feature> selectedZipFeatures = inMemoryFeatureLayer_Zips_Colorization.FeatureSource.GetFeaturesByColumnValue(RequestedColumns_RepPerZip[0].ColumnName.ToString(), s.Value.ToString());

                    inMemoryFeatureLayer_Zips_Colorization.FeatureSource.Close();

                    Collection<AreaBaseShape> areaBaseShapes = new Collection<AreaBaseShape>();

                    foreach (Feature feature in selectedZipFeatures)

                    {

                        areaBaseShapes.Add((AreaBaseShape)feature.GetShape());

                    }

                    MultipolygonShape unionMultipolygonShape = MultipolygonShape.Union(areaBaseShapes);

                    inMemoryFeatureLayer_Zips_Colorization_union.InternalFeatures.Add(new Feature(unionMultipolygonShape));

                }

                geoTools.Serialize_InMemoryFeatureLayer(inMemoryFeatureLayer_Zips_Colorization_union, SERVER_RAM_DRIVE_CACHE_PATH + "inMemoryFeatureLayer_Zips_Colorization_union.bin");

            }


 



#2

better code view 
  
  

    InMemoryFeatureLayer inMemoryFeatureLayer_Zips_Colorization_union = new InMemoryFeatureLayer();
                inMemoryFeatureLayer_Zips_Colorization_union.Name = "inMemoryFeatureLayer_Zips_Colorization";
                inMemoryFeatureLayer_Zips_Colorization_union.IsVisible = true;
                inMemoryFeatureLayer_Zips_Colorization.ZoomLevelSet.ZoomLevel01.CustomStyles.Add(valueStyleRep);
                inMemoryFeatureLayer_Zips_Colorization_union.ZoomLevelSet.ZoomLevel01.ApplyUntilZoomLevel = ApplyUntilZoomLevel.Level20;

                foreach (DictionaryEntry s in _repNameList)
                {
                    inMemoryFeatureLayer_Zips_Colorization.FeatureSource.Open();
                    Collection<Feature> selectedZipFeatures = inMemoryFeatureLayer_Zips_Colorization.FeatureSource.GetFeaturesByColumnValue(RequestedColumns_RepPerZip[0].ColumnName.ToString(), s.Value.ToString());
                    inMemoryFeatureLayer_Zips_Colorization.FeatureSource.Close();
                    Collection<AreaBaseShape> areaBaseShapes = new Collection<AreaBaseShape>();
                    foreach (Feature feature in selectedZipFeatures)
                    {
                        areaBaseShapes.Add((AreaBaseShape)feature.GetShape());
                    }
                    MultipolygonShape unionMultipolygonShape = MultipolygonShape.Union(areaBaseShapes);
                    inMemoryFeatureLayer_Zips_Colorization_union.InternalFeatures.Add(new Feature(unionMultipolygonShape));
                }
                geoTools.Serialize_InMemoryFeatureLayer(inMemoryFeatureLayer_Zips_Colorization_union, SERVER_RAM_DRIVE_CACHE_PATH + "inMemoryFeatureLayer_Zips_Colorization_union.bin");



#3

Jean-marie, 
  
 It looks very strange, your hardware so awesome and you just using InMemoryFeatureLayer, that should do everything in memory and so fast. 
  
 Could you add some StopWatch for make sure which API takes most time? 
  
 Sorry I don’t have your environment and data so I cannot test that local. 
  
 Regards, 
  
 Don

#4

Much better when using the TPL from MS!!! 
  
 
  public InMemoryFeatureLayer unionizedShapes(Collection<FeatureSourceColumn> RequestedColumns, InMemoryFeatureLayer inMemoryFeatureLayer_in, SortedList nameList, string colKey)
        {
            InMemoryFeatureLayer inMemoryFeatureLayer_out = new InMemoryFeatureLayer();

            Parallel.ForEach(nameList.Cast<DictionaryEntry>(), s =>
            {              
                inMemoryFeatureLayer_in.FeatureSource.Open();
                Collection<Feature> selectedFeatures = inMemoryFeatureLayer_in.FeatureSource.GetFeaturesByColumnValue(RequestedColumns[0].ColumnName.ToString(), s.Value.ToString());
                inMemoryFeatureLayer_in.FeatureSource.Close();
                Collection<AreaBaseShape> areaBaseShapes; areaBaseShapes = new Collection<AreaBaseShape>();
                if (selectedFeatures.Count > 0)
                {
                    foreach (Feature feature in selectedFeatures)
                    {
                        areaBaseShapes.Add((AreaBaseShape)feature.GetShape());
                    }
                    MultipolygonShape unionMultipolygonShape = PolygonShape.Union(areaBaseShapes);
                    if (unionMultipolygonShape.Polygons.Count > 0)
                    {
                        Feature f = new Feature(unionMultipolygonShape);
                        f.ColumnValues.Add(colKey, selectedFeatures[0].ColumnValues[colKey].ToString());
                        inMemoryFeatureLayer_out.InternalFeatures.Add(f);
                    }
                }  
            });
            return (inMemoryFeatureLayer_out);
        }
 

#5

Jean-marie,  
  
 So it looks the Task Parallel Library should be helpful for union operation. Thanks for your share! 
  
 Regards, 
  
 Don