You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Joe Paulsen <jo...@verizon.net> on 2003/05/21 17:36:59 UTC
Indexer not fully indexing documents
I have just downloaded the Lucene 1.2 distribution.
After running through the demo, I decided to index (using IndexFiles that is used to load the demo documents) about 100 XML files that I have available - most of which range in size from 0.6 to 3 megabytes. I thought the indexer would treat these files as very large text files (no fields) and essentially index all of the "words" in the document.
After the indexing operation, I used the SearchFiles demo to query the database and noted what appears to be a problem with the way the indexing occurred. In particular, I tried to query on simple phrases (or unique words) that appeared near the end of some of these XML files and got zero results. I.E., it would appear that only terms which occurred at relatively speaking the "beginning" of these files got indexed. (Not really sure what relative beginning means in terms of size).
Is this a limit based on the classes I am using - or some kind of product limit based on document size?
Thanks to anyone who can help!
Joe
Re: Indexer not fully indexing documents
Posted by Joe Paulsen <jo...@verizon.net>.
Thanks - sorry I hadn't read enough documentation yet to avoid the
question.
Joe
----- Original Message -----
From: "Otis Gospodnetic" <ot...@yahoo.com>
To: "Lucene Users List" <lu...@jakarta.apache.org>
Sent: Wednesday, May 21, 2003 11:57 AM
Subject: Re: Indexer not fully indexing documents
> Yes, look at one of the (static?) IndexWriter fields in the javadoc.
>
> Otis
>
> --- Joe Paulsen <jo...@verizon.net> wrote:
> > I have just downloaded the Lucene 1.2 distribution.
> >
> > After running through the demo, I decided to index (using IndexFiles
> > that is used to load the demo documents) about 100 XML files that I
> > have available - most of which range in size from 0.6 to 3 megabytes.
> > I thought the indexer would treat these files as very large text
> > files (no fields) and essentially index all of the "words" in the
> > document.
> >
> > After the indexing operation, I used the SearchFiles demo to query
> > the database and noted what appears to be a problem with the way the
> > indexing occurred. In particular, I tried to query on simple phrases
> > (or unique words) that appeared near the end of some of these XML
> > files and got zero results. I.E., it would appear that only terms
> > which occurred at relatively speaking the "beginning" of these files
> > got indexed. (Not really sure what relative beginning means in terms
> > of size).
> >
> > Is this a limit based on the classes I am using - or some kind of
> > product limit based on document size?
> >
> > Thanks to anyone who can help!
> >
> > Joe
> >
> >
>
>
> __________________________________
> Do you Yahoo!?
> The New Yahoo! Search - Faster. Easier. Bingo.
> http://search.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Indexer not fully indexing documents
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Yes, look at one of the (static?) IndexWriter fields in the javadoc.
Otis
--- Joe Paulsen <jo...@verizon.net> wrote:
> I have just downloaded the Lucene 1.2 distribution.
>
> After running through the demo, I decided to index (using IndexFiles
> that is used to load the demo documents) about 100 XML files that I
> have available - most of which range in size from 0.6 to 3 megabytes.
> I thought the indexer would treat these files as very large text
> files (no fields) and essentially index all of the "words" in the
> document.
>
> After the indexing operation, I used the SearchFiles demo to query
> the database and noted what appears to be a problem with the way the
> indexing occurred. In particular, I tried to query on simple phrases
> (or unique words) that appeared near the end of some of these XML
> files and got zero results. I.E., it would appear that only terms
> which occurred at relatively speaking the "beginning" of these files
> got indexed. (Not really sure what relative beginning means in terms
> of size).
>
> Is this a limit based on the classes I am using - or some kind of
> product limit based on document size?
>
> Thanks to anyone who can help!
>
> Joe
>
>
__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org