You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by John Z <zj...@yahoo.com> on 2004/08/04 18:22:42 UTC
Question on number of fields in a document.
Hi
I had a question related to number of fields in a document. Is there any limit to the number of fields you can have in an index.
We have around 25-30 fields per document at present, about 6 are keywords, Around 6 stored, but not indexed and rest of them are text, which is analyzed and indexed fields. We are planning on adding around 24 more fields , mostly keywords.
Does anyone see any issues with this? Impact to search or index ?
Thanks
ZJ
---------------------------------
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
Re: Question on number of fields in a document.
Posted by John Z <zj...@yahoo.com>.
Thanks
I was looking at some older email on the list and found an email where Doug Cutting says that fields not analyzed, we need not store the norms , nor load them into memory.
That change in the indexer will help a lot in this situation, where we might have 24 fields indexed but not analyzed.
ZJ
Paul Elschot <pa...@xs4all.nl> wrote:
On Wednesday 04 August 2004 18:22, John Z wrote:
> Hi
>
> I had a question related to number of fields in a document. Is there any
> limit to the number of fields you can have in an index.
>
> We have around 25-30 fields per document at present, about 6 are keywords,
> Around 6 stored, but not indexed and rest of them are text, which is
> analyzed and indexed fields. We are planning on adding around 24 more
> fields , mostly keywords.
>
> Does anyone see any issues with this? Impact to search or index ?
During search one byte of RAM is needed per searched field per document
for the normalisation factors, even if a document field is empty.
This RAM is occupied the first time a field is searched after opening
an index reader.
Supposing your queries would actually search 50 fields before
closing the index reader, the norms would occupy 50 bytes/doc, or
1 GB / 20MDocs.
Regards,
Paul
Regards,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: Question on number of fields in a document.
Posted by Paul Elschot <pa...@xs4all.nl>.
On Wednesday 04 August 2004 18:22, John Z wrote:
> Hi
>
> I had a question related to number of fields in a document. Is there any
> limit to the number of fields you can have in an index.
>
> We have around 25-30 fields per document at present, about 6 are keywords,
> Around 6 stored, but not indexed and rest of them are text, which is
> analyzed and indexed fields. We are planning on adding around 24 more
> fields , mostly keywords.
>
> Does anyone see any issues with this? Impact to search or index ?
During search one byte of RAM is needed per searched field per document
for the normalisation factors, even if a document field is empty.
This RAM is occupied the first time a field is searched after opening
an index reader.
Supposing your queries would actually search 50 fields before
closing the index reader, the norms would occupy 50 bytes/doc, or
1 GB / 20MDocs.
Regards,
Paul
Regards,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
RE: Question on number of fields in a document.
Posted by Aviran <am...@infosciences.com>.
You should be fine, no problem with the number of fields
-----Original Message-----
From: John Z [mailto:zjavier_1@yahoo.com]
Sent: Wednesday, August 04, 2004 12:23 PM
To: lucene-user@jakarta.apache.org
Subject: Question on number of fields in a document.
Hi
I had a question related to number of fields in a document. Is there any
limit to the number of fields you can have in an index.
We have around 25-30 fields per document at present, about 6 are keywords,
Around 6 stored, but not indexed and rest of them are text, which is
analyzed and indexed fields. We are planning on adding around 24 more fields
, mostly keywords.
Does anyone see any issues with this? Impact to search or index ?
Thanks
ZJ
---------------------------------
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org