I have a single custom data source that I want to have associated with three layers. Each layer should display only a filtered subset of the available features. I've reviwed the example of filtering the features collection within a layer, but is there a way to apply that filter so that the data source will not actually generate the unnecessary features? In other words... is there some mechanism for a layer (a custom layer) to pass a filter into its underlying datasource. Seems like there needs to be some parameter passed into the data source functions? Or is there a method on the layer that we can implement where we make our own call to the data source?
Filtering data in a custom data source
Ted,
We don’t generate the unnecessary features in the current Desktop Edition (as well as Service and Web) . That means it will not have too much performance penalty when multi layers share one data source. For example, I have one feature source which includes Big Cities and Small Cities. I created two layer, bigCityLayer and smallCityLayer, using this same data source. I add value style for each layer. In bigCityLayer, I render the layer only if the field CityType equals to “Big”, and for smallCityLayer, I render it only if that field is “Small”. In this way, Small City Features will never be read from the data source in BigCityLayer, and Big City Features will never be read in smallCityLayer. It checks every feature if it is a big or small, but it will not read the data out unless that feature is qualified.
I’m not very sure about your scenario, if it’s similar as the one above, I don’t think you will have too much performance issue on the filter part. To make it more efficient, you can split the common data source to 2(for the above sample), one specific for Big Cities and the other is just for small cities, that will save the times for checking the type of every features.
If you want, you can also create your own class by inheriting the ShapeFileFeatureSource class for example, add properties for the filtering and override the key methods letting them work with the filtering conditions. This is a bit difficult though and I don’t think it will help much for this scenario,
Hope that helps, let me know if you have any issues.
Thanks,
Ben
I think I understand your first example. If I have a column called FarmName, and I have 50 records in the datasource with 5 distinct values for FarmName, and 6 of those records have "Hoaglin" as their value. I understand I can add a ValueType style that only has a single element... for FarmName = "Hoaglin". And I see that only that set of features would get drawn. But I'm not sure that I understand how I would go about filtering for those 6 records in my custom datasource, such that the feature collection that I return to you would have just 6 members, rather than 50. I know that 6 vs 50 is not a big deal, but it's the principle, right :)
I am interested in further discussion/example of your suggestion for filtering methods in the custom data source. That was my original intent, but I didn't see where you ever ask my datasource for a feature collection based upon a tabular filter.
After asking the original question, I decided that the answer would probably be for me to have my custom data sources built to proxy to an alternative master datasource. Then each layer would have its own data source, as you suggested, and that data source would have a property that had been set to let it know how to filter the records of the master datasource. Just wanted to make sure I wasn't missing a built-in filtering mechanism within your existing base objects.
Thanks!
Ted,
The data will not be read unless it needs to be used. So in your sample, it only reads the 6 “Hoaglin” data and will not get any other data. It has overheads though that it needs to read the fields of “FarmName” and compare it with “Hoaglin” for all the 50 features, that’s true but my point is that creating your own feature source doesn’t help too much on that either. You can create custom data source filtering features from a master data source, yes, they only return the features you need, but think about how they filter data from the master data source, it still needs to read the “FarmName” column and do the comparision for all the 50 features, right? So I think that might not be a great solution form the performance aspects, instead, it’s better if we just split the “master” data to small data physically, we will not need any checking (as we do not need to use ValueStyle) and totally eliminate that overhead.
And just let you know that we don’t have built-in filtering methods on data source level. But with a great scalability, you can extend a data source easily.
Let me know for any comments. :)
Ben.
Thanks for the info. I strongly disagree about the efficiency of me filtering data before copying it into a feature set vs you filtering it within your routines. I may have indexed sources that let me filter the data quickly, and I don’t have to copy the geometries in my objects into those of your objects. But you’ve answered my question. There is no filtering provision, so I’ll design my feature sources to only have smaller chunks of explicitly selected data, such as representing a filtered query.
Thanks!
Ted,
I was reading over some posts and spotted this one. I think I understand what you want to and maybe an help by clearing some things up. We didn’t implement any generic filtering logic because in many cases the underlying source of the data has its own filtering logic. In Sql Server for example we would recommended a view or to override the Sql we generate and add your own where clause. In shape files we would suggest you build a custom spatial index. In many cases the query itself is what takes most of the time and resources. To filter the items out after the query doesn’t save much time. Since each data source has it’s own unique way of filtering it is hard for us to write a one size fits all API.
I do agree that filtering should be done at the lowest level possible. To suggest we filter at the data source level makes allot of sense. I think we could actually build in a generic filter and then people who write their own FeatureSource’s can choose to override it and apply filters at even lower levels.
Specifically to your custom feature source you can create just one and do your filtering. Imagine I wanted to write a new feature source against a access table. That table had lots of rows and I wanted to filter them by a column called ‘FarmName’. What I would do is create a feature source and add a property called something FarmName and in my GetAllFeaturesCore, GetFeaturesInBoundingBoxCore and all the other places I get features I would filter by the FarmName properties value. I would then create a custom layer to wrap this and on the layer create a property for FarmName as well and have the get and the set just call the get and set of the feature source. In this way for your three Layers you just create them and for each you set a different property for the FarmName property. Each one will fetch different data though they point to the same underlying data.
I hope this makes sense. From all of your posts I can see you are working through your architecture. If you ever wanted to chat about it on the phone please let me know and we can arrange some time. I can explain allot of this better that way and we can short circuit misunderstandings.
David
The reason I wanted to do filtering was so that I could use the same datasource in multiple layers. Setting a property on the data source defeats that goal.
It is no problem, though. I created a datasource that would be more like the Access databsae, and then created little tiny feature sources that are layer specific, and can have the filter property. They pass this call into GetFeatureCollection method that accepts an additional parameter of a filter.
Yep… just trying to explore all of the options for customization in MapSuite before I make any concrete design decisions about how I want to integrate MapSuite into our existing application. We are replacing an ActiveX solution that we’ve used for many years, so I’m looking for the most efficient way to slip MapSuite in under the covers. All of our data lives in “custom” data sources, rather than shapefiles or databases. And we do have the issue of dealing with thousands of surface cells or individual densely logged GPS values. This is what has always brought other “out-of-the-box” solutions to their knees, and we end up adding layer placeholders and then drawing this dense data our selves direction in GDI, etc.
Ted,
Speaking of GDI we have an extension that draws with GDI instead of GDI+. We found the speed wasn’t that compelling to release it but you are welcome to it if you want to review it. It might make a big difference in you application with its unique drawing characteristics.
On the custom feature source I think it is a smart move. There are only so many things things we can do to make an general purpose feature sources. In the end if you don’t mind a little inheritance you can get more speed knowing your own data and access patterns.
David
GDI+ is fine.
Ted,
Great, then my work here is done.
David