You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jae Joo <ja...@gmail.com> on 2007/11/09 00:05:43 UTC
Multiple indexes
Hi,
I am looking for the way to utilize the multiple indexes for signle sole
instance.
I saw that there is the patch 215 available and would like to ask someone
who knows how to use multiple indexes.
Thanks,
Jae Joo
Re: Multiple indexes
Posted by John Reuning <jr...@lulu.com>.
I've had good luck with MultiCore, but you have to sync trunk from svn
and apply the most recent patch in SOLR-350.
https://issues.apache.org/jira/browse/SOLR-350
-jrr
Jae Joo wrote:
> Hi,
>
> I am looking for the way to utilize the multiple indexes for signle sole
> instance.
> I saw that there is the patch 215 available and would like to ask someone
> who knows how to use multiple indexes.
>
> Thanks,
>
> Jae Joo
>
Re: Multiple indexes
Posted by Jae Joo <ja...@gmail.com>.
I have built the master solr instance and indexed some files. Once I run
snapshotter, i complains the error.. - snapshooter -d data/index (in
solr/bin directory)
Did I missed something?
++ date '+%Y/%m/%d %H:%M:%S'
+ echo 2007/11/12 12:38:40 taking snapshot
/solr/master/solr/data/index/snapshot.20071112123840
+ [[ -n '' ]]
+ mv /solr/master/solr/data/index/temp-snapshot.20071112123840/solr/master/solr/data/index/snapshot.20071112123840
mv: cannot access /solr/master/solr/data/index/temp-snapshot.20071112123840
Jae
On Nov 12, 2007 9:09 AM, Ryan McKinley <ry...@gmail.com> wrote:
>
> just use the standard collection distribution stuff. That is what it is
> made for! http://wiki.apache.org/solr/CollectionDistribution
>
> Alternatively, open up two indexes using the same config/dir -- do your
> indexing on one and the searching on the other. when indexing is done
> (or finishes a big chunk) send <commit/> to the 'searching' one and it
> will see the new stuff.
>
> ryan
>
>
>
> Jae Joo wrote:
> > Here is my situation.
> >
> > I have 6 millions articles indexed and adding about 10k articles
> everyday.
> > If I maintain only one index, whenever the daily feeding is running, it
> > consumes the heap area and causes FGC.
> > I am thinking the way to have multiple indexes - one is for ongoing
> querying
> > service and one is for update. Once update is done, switch the index by
> > automatically and/or my application.
> >
> > Thanks,
> >
> > Jae joo
> >
> >
> > On Nov 12, 2007 8:48 AM, Ryan McKinley <ry...@gmail.com> wrote:
> >
> >> The advantages of a multi-core setup are configuration flexibility and
> >> dynamically changing available options (without a full restart).
> >>
> >> For high-performance production solr servers, I don't think there is
> >> much reason for it. You may want to split the two indexes on to two
> >> machines. You may want to run each index in a separate JVM (so if one
> >> crashes, the other does not)
> >>
> >> Maintaining 2 indexes is pretty easy, if that was a larger number or
> you
> >> need to create indexes for each user in a system then it would be worth
> >> investigating the multi-core setup (it is still in development)
> >>
> >> ryan
> >>
> >>
> >> Pierre-Yves LANDRON wrote:
> >>> Hello,
> >>>
> >>> Until now, i've used two instance of solr, one for each of my
> >> collections ; it works fine, but i wonder
> >>> if there is an advantage to use multiple indexes in one instance over
> >> several instances with one index each ?
> >>> Note that the two indexes have different schema.xml.
> >>>
> >>> Thanks.
> >>> PL
> >>>
> >>>> Date: Thu, 8 Nov 2007 18:05:43 -0500
> >>>> From: jaejoo@gmail.com
> >>>> To: solr-user@lucene.apache.org
> >>>> Subject: Multiple indexes
> >>>>
> >>>> Hi,
> >>>>
> >>>> I am looking for the way to utilize the multiple indexes for signle
> >> sole
> >>>> instance.
> >>>> I saw that there is the patch 215 available and would like to ask
> >> someone
> >>>> who knows how to use multiple indexes.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Jae Joo
> >>> _________________________________________________________________
> >>> Discover the new Windows Vista
> >>> http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
> >>
> >
>
>
Re: Multiple indexes
Posted by Ryan McKinley <ry...@gmail.com>.
just use the standard collection distribution stuff. That is what it is
made for! http://wiki.apache.org/solr/CollectionDistribution
Alternatively, open up two indexes using the same config/dir -- do your
indexing on one and the searching on the other. when indexing is done
(or finishes a big chunk) send <commit/> to the 'searching' one and it
will see the new stuff.
ryan
Jae Joo wrote:
> Here is my situation.
>
> I have 6 millions articles indexed and adding about 10k articles everyday.
> If I maintain only one index, whenever the daily feeding is running, it
> consumes the heap area and causes FGC.
> I am thinking the way to have multiple indexes - one is for ongoing querying
> service and one is for update. Once update is done, switch the index by
> automatically and/or my application.
>
> Thanks,
>
> Jae joo
>
>
> On Nov 12, 2007 8:48 AM, Ryan McKinley <ry...@gmail.com> wrote:
>
>> The advantages of a multi-core setup are configuration flexibility and
>> dynamically changing available options (without a full restart).
>>
>> For high-performance production solr servers, I don't think there is
>> much reason for it. You may want to split the two indexes on to two
>> machines. You may want to run each index in a separate JVM (so if one
>> crashes, the other does not)
>>
>> Maintaining 2 indexes is pretty easy, if that was a larger number or you
>> need to create indexes for each user in a system then it would be worth
>> investigating the multi-core setup (it is still in development)
>>
>> ryan
>>
>>
>> Pierre-Yves LANDRON wrote:
>>> Hello,
>>>
>>> Until now, i've used two instance of solr, one for each of my
>> collections ; it works fine, but i wonder
>>> if there is an advantage to use multiple indexes in one instance over
>> several instances with one index each ?
>>> Note that the two indexes have different schema.xml.
>>>
>>> Thanks.
>>> PL
>>>
>>>> Date: Thu, 8 Nov 2007 18:05:43 -0500
>>>> From: jaejoo@gmail.com
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Multiple indexes
>>>>
>>>> Hi,
>>>>
>>>> I am looking for the way to utilize the multiple indexes for signle
>> sole
>>>> instance.
>>>> I saw that there is the patch 215 available and would like to ask
>> someone
>>>> who knows how to use multiple indexes.
>>>>
>>>> Thanks,
>>>>
>>>> Jae Joo
>>> _________________________________________________________________
>>> Discover the new Windows Vista
>>> http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
>>
>
Re: Multiple indexes
Posted by Jae Joo <ja...@gmail.com>.
Here is my situation.
I have 6 millions articles indexed and adding about 10k articles everyday.
If I maintain only one index, whenever the daily feeding is running, it
consumes the heap area and causes FGC.
I am thinking the way to have multiple indexes - one is for ongoing querying
service and one is for update. Once update is done, switch the index by
automatically and/or my application.
Thanks,
Jae joo
On Nov 12, 2007 8:48 AM, Ryan McKinley <ry...@gmail.com> wrote:
> The advantages of a multi-core setup are configuration flexibility and
> dynamically changing available options (without a full restart).
>
> For high-performance production solr servers, I don't think there is
> much reason for it. You may want to split the two indexes on to two
> machines. You may want to run each index in a separate JVM (so if one
> crashes, the other does not)
>
> Maintaining 2 indexes is pretty easy, if that was a larger number or you
> need to create indexes for each user in a system then it would be worth
> investigating the multi-core setup (it is still in development)
>
> ryan
>
>
> Pierre-Yves LANDRON wrote:
> > Hello,
> >
> > Until now, i've used two instance of solr, one for each of my
> collections ; it works fine, but i wonder
> > if there is an advantage to use multiple indexes in one instance over
> several instances with one index each ?
> > Note that the two indexes have different schema.xml.
> >
> > Thanks.
> > PL
> >
> >> Date: Thu, 8 Nov 2007 18:05:43 -0500
> >> From: jaejoo@gmail.com
> >> To: solr-user@lucene.apache.org
> >> Subject: Multiple indexes
> >>
> >> Hi,
> >>
> >> I am looking for the way to utilize the multiple indexes for signle
> sole
> >> instance.
> >> I saw that there is the patch 215 available and would like to ask
> someone
> >> who knows how to use multiple indexes.
> >>
> >> Thanks,
> >>
> >> Jae Joo
> >
> > _________________________________________________________________
> > Discover the new Windows Vista
> > http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
>
>
Re: Multiple indexes
Posted by Ryan McKinley <ry...@gmail.com>.
The advantages of a multi-core setup are configuration flexibility and
dynamically changing available options (without a full restart).
For high-performance production solr servers, I don't think there is
much reason for it. You may want to split the two indexes on to two
machines. You may want to run each index in a separate JVM (so if one
crashes, the other does not)
Maintaining 2 indexes is pretty easy, if that was a larger number or you
need to create indexes for each user in a system then it would be worth
investigating the multi-core setup (it is still in development)
ryan
Pierre-Yves LANDRON wrote:
> Hello,
>
> Until now, i've used two instance of solr, one for each of my collections ; it works fine, but i wonder
> if there is an advantage to use multiple indexes in one instance over several instances with one index each ?
> Note that the two indexes have different schema.xml.
>
> Thanks.
> PL
>
>> Date: Thu, 8 Nov 2007 18:05:43 -0500
>> From: jaejoo@gmail.com
>> To: solr-user@lucene.apache.org
>> Subject: Multiple indexes
>>
>> Hi,
>>
>> I am looking for the way to utilize the multiple indexes for signle sole
>> instance.
>> I saw that there is the patch 215 available and would like to ask someone
>> who knows how to use multiple indexes.
>>
>> Thanks,
>>
>> Jae Joo
>
> _________________________________________________________________
> Discover the new Windows Vista
> http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
RE: Multiple indexes
Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
Hello,
Until now, i've used two instance of solr, one for each of my collections ; it works fine, but i wonder
if there is an advantage to use multiple indexes in one instance over several instances with one index each ?
Note that the two indexes have different schema.xml.
Thanks.
PL
> Date: Thu, 8 Nov 2007 18:05:43 -0500
> From: jaejoo@gmail.com
> To: solr-user@lucene.apache.org
> Subject: Multiple indexes
>
> Hi,
>
> I am looking for the way to utilize the multiple indexes for signle sole
> instance.
> I saw that there is the patch 215 available and would like to ask someone
> who knows how to use multiple indexes.
>
> Thanks,
>
> Jae Joo
_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE