You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Sirish <vs...@yahoo.co.in> on 2007/11/15 19:42:22 UTC

Indexing Problem

The following is my code snippet for indexing the text:

document.add(Field.Text(IFIELD_TEXT, billMeasureDoc.getText()));

When ever the text is less or short, it works perfectly. But in few of the
cases if the text is too lengthy; i.e. around 1000 lines or more then it
causes a problem.

The problem being when the text is lengthy, after indexing, the document is
getting searched only upto a certain extent of text. For eg: Lets say we
have a text:

Test Test Test Test Test Test Test Test Test
.......... .......... .......... .......... .......... .......... .....
Test1 Test1 Test1 Test1 Test1 Test1 Test1 
.......... .......... .......... .......... .......... .......... .....
Test2 Test2 Test2 Test2 Test2 Test2 Test2

Now while searching, it returns the document only if I search for either
Test or Test1 and it ignores any text that is trailing after Test1.

Can someone let me know if there is any text or character size limitation
for Field.Text in my above code. Also I would like to store this text as I
need to implement highlighting for the search text.


-- 
View this message in context: http://www.nabble.com/Indexing-Problem-tf4816336.html#a13778913
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Indexing Problem

Posted by Sirish <vs...@yahoo.co.in>.
Wow!! Thanks dude... that works... I have spent almost a day figuring out the
issue... I appreciate it!!

Erick Erickson wrote:
> 
> Your problem is probably, that by default, Lucene stops after
> 10,000 terms. See IndexWriter.SetMaxFieldLength
> 
> Best
> Erick
> 
> On Nov 15, 2007 1:42 PM, Sirish <vs...@yahoo.co.in> wrote:
> 
>>
>> The following is my code snippet for indexing the text:
>>
>> document.add(Field.Text(IFIELD_TEXT, billMeasureDoc.getText()));
>>
>> When ever the text is less or short, it works perfectly. But in few of
>> the
>> cases if the text is too lengthy; i.e. around 1000 lines or more then it
>> causes a problem.
>>
>> The problem being when the text is lengthy, after indexing, the document
>> is
>> getting searched only upto a certain extent of text. For eg: Lets say we
>> have a text:
>>
>> Test Test Test Test Test Test Test Test Test
>> .......... .......... .......... .......... .......... .......... .....
>> Test1 Test1 Test1 Test1 Test1 Test1 Test1
>> .......... .......... .......... .......... .......... .......... .....
>> Test2 Test2 Test2 Test2 Test2 Test2 Test2
>>
>> Now while searching, it returns the document only if I search for either
>> Test or Test1 and it ignores any text that is trailing after Test1.
>>
>> Can someone let me know if there is any text or character size limitation
>> for Field.Text in my above code. Also I would like to store this text as
>> I
>> need to implement highlighting for the search text.
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Indexing-Problem-tf4816336.html#a13778913
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Indexing-Problem-tf4816336.html#a13781957
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Indexing Problem

Posted by Erick Erickson <er...@gmail.com>.
Your problem is probably, that by default, Lucene stops after
10,000 terms. See IndexWriter.SetMaxFieldLength

Best
Erick

On Nov 15, 2007 1:42 PM, Sirish <vs...@yahoo.co.in> wrote:

>
> The following is my code snippet for indexing the text:
>
> document.add(Field.Text(IFIELD_TEXT, billMeasureDoc.getText()));
>
> When ever the text is less or short, it works perfectly. But in few of the
> cases if the text is too lengthy; i.e. around 1000 lines or more then it
> causes a problem.
>
> The problem being when the text is lengthy, after indexing, the document
> is
> getting searched only upto a certain extent of text. For eg: Lets say we
> have a text:
>
> Test Test Test Test Test Test Test Test Test
> .......... .......... .......... .......... .......... .......... .....
> Test1 Test1 Test1 Test1 Test1 Test1 Test1
> .......... .......... .......... .......... .......... .......... .....
> Test2 Test2 Test2 Test2 Test2 Test2 Test2
>
> Now while searching, it returns the document only if I search for either
> Test or Test1 and it ignores any text that is trailing after Test1.
>
> Can someone let me know if there is any text or character size limitation
> for Field.Text in my above code. Also I would like to store this text as I
> need to implement highlighting for the search text.
>
>
> --
> View this message in context:
> http://www.nabble.com/Indexing-Problem-tf4816336.html#a13778913
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>