You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Doron Cohen <DO...@il.ibm.com> on 2007/09/24 13:53:43 UTC
Re: index conversion
For an already optimized index calling optimize() is a no-op.
You may try this: after opening the writer and setting compound=false, add
a dummy (even empty) document to the index, then optimize(), and finally
optionally remove the dummy document.
Note that calling optimize() might be lengthy as well for a large index. In
any case, try this first on a trial index, and also make a backup of the
existing index, just in case.
Why do you want to transform the index from compound to non compound form?
Doron
"Bogdan Ghidireac" <bo...@ecstend.com> wrote on 24/09/2007 15:38:51:
> Hi,
>
> I have an optimized compound index and I want to transform it to standard
> format. I tried to optimize using the standard format but I had
> no luck.. Is
> there another way ? I don't want to re-index because it will take several
> days.
>
> IndexWriter writer = new IndexWriter(args[0], new StandardAnalyzer(),
> false);
> writer.setUseCompoundFile(false);
> writer.setMaxBufferedDocs(10000);
> writer.optimize();
> writer.close();
>
> Regards,
> Bogdan
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: index conversion
Posted by Grant Ingersoll <gs...@apache.org>.
On Sep 24, 2007, at 9:54 AM, Bogdan Ghidireac wrote:
> On 9/24/07, Doron Cohen <DO...@il.ibm.com> wrote:
>>
>> For an already optimized index calling optimize() is a no-op.
>>
>> You may try this: after opening the writer and setting
>> compound=false, add
>> a dummy (even empty) document to the index, then optimize(), and
>> finally
>> optionally remove the dummy document.
>>
>> Note that calling optimize() might be lengthy as well for a large
>> index.
>> In
>> any case, try this first on a trial index, and also make a backup
>> of the
>> existing index, just in case.
>
>
> thanks, that worked smoothly ..
>
>
> Why do you want to transform the index from compound to non
> compound form?
>
>
> My index is quite large (14GB) and this is because I use stored
> fields. The
> requirements of the project force me to have a response time below
> 500ms and
> I cannot do this at this moment (the tp99.9 is 700ms).
>
> What I am planning to do is to use the standard format so I can
> move all
> relevant files except .fdt and .fdx to a memory disk. After
> conversion the
> .fdt + .fdx files have around 11.5GB.
>
> I don't think there is an index reader that supports two locations
> (one for
> index data and another one for stored data) but I will modify an
> existing
> one and check the results.
I believe this is the use case for ParallelReader (although I have
never used it) but it comes with caveats. Search the archives and
checkout the javadocs for info.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: index conversion
Posted by Bogdan Ghidireac <bo...@ecstend.com>.
On 9/24/07, Doron Cohen <DO...@il.ibm.com> wrote:
>
> For an already optimized index calling optimize() is a no-op.
>
> You may try this: after opening the writer and setting compound=false, add
> a dummy (even empty) document to the index, then optimize(), and finally
> optionally remove the dummy document.
>
> Note that calling optimize() might be lengthy as well for a large index.
> In
> any case, try this first on a trial index, and also make a backup of the
> existing index, just in case.
thanks, that worked smoothly ..
Why do you want to transform the index from compound to non compound form?
My index is quite large (14GB) and this is because I use stored fields. The
requirements of the project force me to have a response time below 500ms and
I cannot do this at this moment (the tp99.9 is 700ms).
What I am planning to do is to use the standard format so I can move all
relevant files except .fdt and .fdx to a memory disk. After conversion the
.fdt + .fdx files have around 11.5GB.
I don't think there is an index reader that supports two locations (one for
index data and another one for stored data) but I will modify an existing
one and check the results.
Bogdan
Doron
>
> "Bogdan Ghidireac" <bo...@ecstend.com> wrote on 24/09/2007 15:38:51:
>
> > Hi,
> >
> > I have an optimized compound index and I want to transform it to
> standard
> > format. I tried to optimize using the standard format but I had
> > no luck.. Is
> > there another way ? I don't want to re-index because it will take
> several
> > days.
> >
> > IndexWriter writer = new IndexWriter(args[0], new StandardAnalyzer(),
> > false);
> > writer.setUseCompoundFile(false);
> > writer.setMaxBufferedDocs(10000);
> > writer.optimize();
> > writer.close();
> >
> > Regards,
> > Bogdan
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>