[Solr Wiki] Update of "IndexPartitioning" by mbryzek

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The following page has been changed by mbryzek:
http://wiki.apache.org/solr/IndexPartitioning

------------------------------------------------------------------------------
     <partitionField default="foo">objectType</partitionField>
  }}}
  
- ...which would indicate that anytime a document is added, the `objectType` field should be inspected, and used as the name of the parition (and directory of the underlying index).  if a document does not have a value for the obejctType field, the default partition of "foo" should be assumed.
+ ...which would indicate that anytime a document is added, the `objectType` field should be inspected, and used as the name of the parition (and directory of the underlying index).  if a document does not have a value for the objectType field, the default partition of "foo" should be assumed.
  
  = Things that need to be considered =
  
@@ -49, +49 @@

        * should it be possible to replicate individual partitions seperately?
     * The main motivation is performance, based on past experience showing that using partitions with lucene indexes is faster then uber-lucene indexes, but Solr already has a lot of caching improvements that may already make the issue moot -- and adding support for this could acctaully harm the performance compared to using a single index (depending on how the change would impact the caching framework).  Some serious Performance testing should be done before expending a lot of effort on this idea.
  
+ 
+ == Alternate Dynamic Partitioning Idea (number 2) ==
+ 
+ Similar to the previous Dynamic Partitioning Idea, this variant provides the constraint that the factors that go into partitioning data are known in advance. Considering confguration like this...
+ 
+ {{{
+    <partition>
+      <partitionField name="creationYear">
+       <value>2004</value>     
+       <value>2005</value>     
+       <value>2006</value>     
+      </partitionField>
+    </partition>
+ }}}
+ 
+ This would indicate that any time a document is added, we inspect the value of the objectType field to determine to which index to write data. The default partition is used whenever a value for objectType is either not provided or does not match one of the provided values. The data directory would follow the convention of partitionFieldName/partitionFieldValue (e.g. data/creationYear/2004/index).
+ 
+ = Additional considerations =
+ 
+    * Options should be considered for how defaults are handled:
+       * Does data exist in exactly one place in the index (i.e. either in the default or a partition)?
+       * Can we have an option to always keep a copy of the data in the default partition, possibly with a copy in a partition? 
+ 
+ 
+ == Query Integration ==
+ 
+ One suggestion is to integrate partition selection into the query parser. For example, assume that we have an index storing data about users, partitioned by the year in which the user was created in the system. If we issue a query like "+email:mbryzek +creationYear:2005", we may want to consider extending the query parser to identify that a query has been issued that can be rewritten to use a partiiton. Doing this provides a simple external API that should be easily understood.
+