You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Antoine Baudoux <ab...@taktik.be> on 2007/08/27 08:26:19 UTC
Maximum index size
Hello,
I'm indexing 2,5 millions docs.
I already have added 1,2 millions docs to the index and the indexing
speed becomes quite slow.
my index directory is 1GB .
Is there a limit to the indexing size?
Is the indexing slow when the number of terms are big?
--
Antoine Baudoux
Development Manager
ab@taktik.be
Tél.: +32 2 333 58 44
GSM: +32 499 534 538
Fax.: +32 2 648 16 53
Re: Maximum index size
Posted by Karl Wettin <ka...@gmail.com>.
27 aug 2007 kl. 14.34 skrev Antoine Baudoux:
> Ok, I will follow the documentation.
>
> But appart from optimization tips, are there intrinsic limits in
> term of :
>
> - Index size.
Integer.MAX_VALUE documents.
> - number of distinct indexed terms.
Integer.MAX_VALUE unique tokens per field.
http://www.nabble.com/How-many-documents-in-the-biggest-Lucene-index-
to-date--tf3120790.html
--
karl
>
>
> --
> Antoine Baudoux
> Development Manager
> ab@taktik.be
> Tél.: +32 2 333 58 44
> GSM: +32 499 534 538
> Fax.: +32 2 648 16 53
>
>
> On 27 Aug 2007, at 13:01, Grant Ingersoll wrote:
>
>> Have a look at: http://wiki.apache.org/lucene-java/
>> BasicsOfPerformance and the section on indexing.
>>
>> Please provide more details about what you are doing and your
>> hardware, memory, etc. if that doesn't help.
>>
>>
>> On Aug 27, 2007, at 2:26 AM, Antoine Baudoux wrote:
>>
>>> Hello,
>>>
>>> I'm indexing 2,5 millions docs.
>>>
>>> I already have added 1,2 millions docs to the index and the
>>> indexing speed becomes quite slow.
>>>
>>> my index directory is 1GB .
>>>
>>> Is there a limit to the indexing size?
>>>
>>> Is the indexing slow when the number of terms are big?
>>>
>>>
>>> --
>>> Antoine Baudoux
>>> Development Manager
>>> ab@taktik.be
>>> Tél.: +32 2 333 58 44
>>> GSM: +32 499 534 538
>>> Fax.: +32 2 648 16 53
>>>
>>>
>>
>> --------------------------
>> Grant Ingersoll
>> http://lucene.grantingersoll.com
>>
>> Lucene Helpful Hints:
>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Maximum index size
Posted by Antoine Baudoux <ab...@taktik.be>.
Ok, I will follow the documentation.
But appart from optimization tips, are there intrinsic limits in term
of :
- Index size.
- number of distinct indexed terms.
--
Antoine Baudoux
Development Manager
ab@taktik.be
Tél.: +32 2 333 58 44
GSM: +32 499 534 538
Fax.: +32 2 648 16 53
On 27 Aug 2007, at 13:01, Grant Ingersoll wrote:
> Have a look at: http://wiki.apache.org/lucene-java/
> BasicsOfPerformance and the section on indexing.
>
> Please provide more details about what you are doing and your
> hardware, memory, etc. if that doesn't help.
>
>
> On Aug 27, 2007, at 2:26 AM, Antoine Baudoux wrote:
>
>> Hello,
>>
>> I'm indexing 2,5 millions docs.
>>
>> I already have added 1,2 millions docs to the index and the
>> indexing speed becomes quite slow.
>>
>> my index directory is 1GB .
>>
>> Is there a limit to the indexing size?
>>
>> Is the indexing slow when the number of terms are big?
>>
>>
>> --
>> Antoine Baudoux
>> Development Manager
>> ab@taktik.be
>> Tél.: +32 2 333 58 44
>> GSM: +32 499 534 538
>> Fax.: +32 2 648 16 53
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://lucene.grantingersoll.com
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Maximum index size
Posted by Grant Ingersoll <gs...@apache.org>.
Have a look at: http://wiki.apache.org/lucene-java/
BasicsOfPerformance and the section on indexing.
Please provide more details about what you are doing and your
hardware, memory, etc. if that doesn't help.
On Aug 27, 2007, at 2:26 AM, Antoine Baudoux wrote:
> Hello,
>
> I'm indexing 2,5 millions docs.
>
> I already have added 1,2 millions docs to the index and the
> indexing speed becomes quite slow.
>
> my index directory is 1GB .
>
> Is there a limit to the indexing size?
>
> Is the indexing slow when the number of terms are big?
>
>
> --
> Antoine Baudoux
> Development Manager
> ab@taktik.be
> Tél.: +32 2 333 58 44
> GSM: +32 499 534 538
> Fax.: +32 2 648 16 53
>
>
--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org