You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by tech20nn <vi...@gmail.com> on 2012/01/31 13:18:06 UTC
Document with longer field names and many fields
We are planning to import data from various tables of ERP DB into a single
Solr/Lucene index.
Since these tables have overlapping columns we are planing to name the
corresponding document field as <table_name>_<column_name>. I have
following questions on this.
1) Does having long field name (<table_name>_<column_name>) affect
performance ?
2) We will end up with close to 200 fields per document in schema
definition. At time of document storage only 20 field per will be indexed
and stored for each stored document. Is there a limitation here, are we
creating performance bottleneck by designing schema this way ?
Thanks
Vijay
--
View this message in context: http://lucene.472066.n3.nabble.com/Document-with-longer-field-names-and-many-fields-tp3703077p3703077.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Document with longer field names and many fields
Posted by Erick Erickson <er...@gmail.com>.
This should be OK. There's no real issue with Solr docs having
up to 200 fields, and theres no real limitation on what
portion of those fields each doc has. In other words, only
having 20 out of 200 possible fields in a doc isn't a problem.
There's no overhead for "unused" fields.
Depending upon the number of *unique* values in these fields,
you may bet some extra memory consumption, but whether
that matters depends a lot on the particulars.
By a huge margin, most of the time is spent in searching
itself, not parsing the field names so long ones really don't
matter at all I think.
Best
Erick
On Tue, Jan 31, 2012 at 7:18 AM, tech20nn <vi...@gmail.com> wrote:
> We are planning to import data from various tables of ERP DB into a single
> Solr/Lucene index.
> Since these tables have overlapping columns we are planing to name the
> corresponding document field as <table_name>_<column_name>. I have
> following questions on this.
>
> 1) Does having long field name (<table_name>_<column_name>) affect
> performance ?
> 2) We will end up with close to 200 fields per document in schema
> definition. At time of document storage only 20 field per will be indexed
> and stored for each stored document. Is there a limitation here, are we
> creating performance bottleneck by designing schema this way ?
>
> Thanks
> Vijay
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Document-with-longer-field-names-and-many-fields-tp3703077p3703077.html
> Sent from the Solr - User mailing list archive at Nabble.com.