Schema vs Dynamic Fields

[Phil Hagelberg <ph...@hagelb.org>]

On the wiki, it says:

> One of the powerful features of Lucene is that you don't have to
> pre-define every field when you first create your index. Even though
> Solr provides strong datatyping for fields, it still preserves that
> flexibility using "Dynamic Fields".

Is the use of a predefined schema primarily a "type safety" feature?
We're considering using Solr for a data set that is very free-form; will
we get much slower results if the majority of our data is in a dynamic
field such as:

  <dynamicField name="*" type="text" indexed="true" stored="true"/>

I'm a little unclear on the trade-offs involved and would appreciate
a hint.

Phil Hagelberg
http://technomancy.us

Re: Schema vs Dynamic Fields

[Chris Hostetter <ho...@fucit.org>]

: Date: Mon, 08 Jun 2009 16:44:45 -0700
: From: Phil Hagelberg
: Subject: Schema vs Dynamic Fields

: Is the use of a predefined schema primarily a "type safety" feature?
: We're considering using Solr for a data set that is very free-form; will
: we get much slower results if the majority of our data is in a dynamic
: field such as:
: 
:   <dynamicField name="*" type="text" indexed="true" stored="true"/>
: 
: I'm a little unclear on the trade-offs involved and would appreciate
: a hint.

There is some cost involved in every new "field" that exists in your index 
(regardless of wether it was explicitly declared, or sprang into existence 
because of a dynamicField declaration) but there are ways to mitigate some 
of those costs (omitNorms=true being a big one)

in general the big advantage to explicitly delcaring fields is that you 
can customize their analysis/datatypes ... you can do similar things by 
having "type specific" dynamic fields but then youre fiend names must 
follow set convnetions based on data type.



-Hoss