You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by kalidoss <ka...@sifycorp.com> on 2010/01/08 05:16:20 UTC
How to Split Index file.
Hi,
I would like to split the existing index by 2 index, ( inverse of
merge index function).
My index directory size around 20G and 10 Million documents.
-Kalidoss.m,
Get your world in your inbox!
Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id!
Log on to http://www.sify.com
********** DISCLAIMER **********
Information contained and transmitted by this E-MAIL is proprietary to
Sify Limited and is intended for use only by the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential or exempt from disclosure under applicable law. If this is a
forwarded message, the content of this E-MAIL may not have been sent with
the authority of the Company. If you are not the intended recipient, an
agent of the intended recipient or a person responsible for delivering the
information to the named recipient, you are notified that any use,
distribution, transmission, printing, copying or dissemination of this
information in any way or in any manner is strictly prohibited. If you have
received this communication in error, please delete this mail & notify us
immediately at admin@sifycorp.com
Re: How to Split Index file.
Posted by Andrzej Bialecki <ab...@getopt.org>.
On 2010-01-10 01:55, Lance Norskog wrote:
> Make two copies of the index. In each copy, delete the records you do
> not want. Optimize.
... which is essentially what the MultiPassIndexSplitter does, only it
avoids the initial copy (by deleting in the source index).
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
Re: How to Split Index file.
Posted by Lance Norskog <go...@gmail.com>.
Make two copies of the index. In each copy, delete the records you do
not want. Optimize.
On Fri, Jan 8, 2010 at 4:08 AM, kalidoss
<ka...@sifycorp.com> wrote:
> Thanks Koji,
>
> http://wiki.apache.org/solr/MergingSolrIndexes -this will be useful for
> merge 2 different indexes, am looking for a tool like to Split a Index
> directory by 2.
>
> Kalidoss.m,
>
>
> Koji Sekiguchi wrote:
>>
>> kalidoss wrote:
>>>
>>> Hi,
>>>
>>> I would like to split the existing index by 2 index, ( inverse of
>>> merge index function).
>>>
>>> My index directory size around 20G and 10 Million documents.
>>>
>>> -Kalidoss.m,
>>>
>> I think IndexSplitter and/or MultiPassIndexSplitter are what you are
>> looking for:
>>
>>
>> http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/IndexSplitter.html
>>
>> http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/MultiPassIndexSplitter.html
>>
>> Koji
>>
>
>
>
> Get your world in your inbox!
>
> Mail, widgets, documents, spreadsheets, organizer and much more with your
> Sifymail WIYI id!
> Log on to http://www.sify.com
>
> ********** DISCLAIMER **********
> Information contained and transmitted by this E-MAIL is proprietary to Sify
> Limited and is intended for use only by the individual or entity to which it
> is addressed, and may contain information that is privileged, confidential
> or exempt from disclosure under applicable law. If this is a forwarded
> message, the content of this E-MAIL may not have been sent with the
> authority of the Company. If you are not the intended recipient, an agent of
> the intended recipient or a person responsible for delivering the
> information to the named recipient, you are notified that any use,
> distribution, transmission, printing, copying or dissemination of this
> information in any way or in any manner is strictly prohibited. If you have
> received this communication in error, please delete this mail & notify us
> immediately at admin@sifycorp.com
>
--
Lance Norskog
goksron@gmail.com
Re: How to Split Index file.
Posted by kalidoss <ka...@sifycorp.com>.
Thanks Koji,
http://wiki.apache.org/solr/MergingSolrIndexes -this will be useful for
merge 2 different indexes, am looking for a tool like to Split a Index
directory by 2.
Kalidoss.m,
Koji Sekiguchi wrote:
> kalidoss wrote:
>> Hi,
>>
>> I would like to split the existing index by 2 index, ( inverse of
>> merge index function).
>>
>> My index directory size around 20G and 10 Million documents.
>>
>> -Kalidoss.m,
>>
> I think IndexSplitter and/or MultiPassIndexSplitter are what you are
> looking for:
>
> http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/IndexSplitter.html
>
> http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/MultiPassIndexSplitter.html
>
>
> Koji
>
Get your world in your inbox!
Mail, widgets, documents, spreadsheets, organizer and much more with your Sifymail WIYI id!
Log on to http://www.sify.com
********** DISCLAIMER **********
Information contained and transmitted by this E-MAIL is proprietary to
Sify Limited and is intended for use only by the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential or exempt from disclosure under applicable law. If this is a
forwarded message, the content of this E-MAIL may not have been sent with
the authority of the Company. If you are not the intended recipient, an
agent of the intended recipient or a person responsible for delivering the
information to the named recipient, you are notified that any use,
distribution, transmission, printing, copying or dissemination of this
information in any way or in any manner is strictly prohibited. If you have
received this communication in error, please delete this mail & notify us
immediately at admin@sifycorp.com
Re: How to Split Index file.
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
kalidoss wrote:
> Hi,
>
> I would like to split the existing index by 2 index, ( inverse of
> merge index function).
>
> My index directory size around 20G and 10 Million documents.
>
> -Kalidoss.m,
>
I think IndexSplitter and/or MultiPassIndexSplitter are what you are
looking for:
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/IndexSplitter.html
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc/contrib-misc/org/apache/lucene/index/MultiPassIndexSplitter.html
Koji
--
http://www.rondhuit.com/en/