You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by arian487 <ak...@tagged.com> on 2011/08/10 01:07:13 UTC
Cache replication
I'm wondering if the caches on all the slaves are replicated across (such as
queryResultCache). That is to say, if I hit one of my slaves and cache a
result, and I make a search later and that search happens to hit a different
slave, will that first cached result be available for use?
This is pretty important because I'm going to have a lot of slaves and if
this isn't done, then I'd have a high chance of running a lot uncached
queries.
Thanks :)
--
View this message in context: http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3240708.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Cache replication
Posted by Bill Bell <bi...@gmail.com>.
OK. But SOLR has built-in caching. Do you not like the caching? What so
you think we should change to the SOLR cache?
Bill
On 8/10/11 9:16 AM, "didier deshommes" <df...@gmail.com> wrote:
>Consider putting a cache (memcached, redis, etc) *in front* of your
>solr slaves. Just make sure to update it when replication occurs.
>
>didier
>
>On Tue, Aug 9, 2011 at 6:07 PM, arian487 <ak...@tagged.com> wrote:
>> I'm wondering if the caches on all the slaves are replicated across
>>(such as
>> queryResultCache). That is to say, if I hit one of my slaves and cache
>>a
>> result, and I make a search later and that search happens to hit a
>>different
>> slave, will that first cached result be available for use?
>>
>> This is pretty important because I'm going to have a lot of slaves and
>>if
>> this isn't done, then I'd have a high chance of running a lot uncached
>> queries.
>>
>> Thanks :)
>>
>> --
>> View this message in context:
>>http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3240708.ht
>>ml
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
Re: Cache replication
Posted by didier deshommes <df...@gmail.com>.
Consider putting a cache (memcached, redis, etc) *in front* of your
solr slaves. Just make sure to update it when replication occurs.
didier
On Tue, Aug 9, 2011 at 6:07 PM, arian487 <ak...@tagged.com> wrote:
> I'm wondering if the caches on all the slaves are replicated across (such as
> queryResultCache). That is to say, if I hit one of my slaves and cache a
> result, and I make a search later and that search happens to hit a different
> slave, will that first cached result be available for use?
>
> This is pretty important because I'm going to have a lot of slaves and if
> this isn't done, then I'd have a high chance of running a lot uncached
> queries.
>
> Thanks :)
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3240708.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Cache replication
Posted by arian487 <ak...@tagged.com>.
Thanks for the advice paul, but post processing is a must for me given the
nature of my application. I haven't had problems yet though.
--
View this message in context: http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3244202.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Cache replication
Posted by Paul Libbrecht <pa...@hoplahup.net>.
Arian,
I've been doing results post-processing in some versions of the ActiveMath server and it has been the wrong choice as much as possible.
Maybe this is not what you do, but the biggest flaw was that the post-processing was eliminating or adding results (for insiders of ActiveMath: converting symbols' matches to the list of their definitions). Pagination was severely broken and I could never fix this correctly.
Maybe this advice helps.
Indexing (with the help of multiple fields and the flexibility of analyzers) as well as query processing are the right tools to my taste.
paul
Le 10 août 2011 à 02:34, arian487 a écrit :
> Thanks for the informative response. I'll consider using the 'sticky'
> addressing as you suggested. The reason cache is so important for me is
> because I'm actually doing more processing after the query component to come
> up with my query result and I want to avoid that processing as much as
> possible. But thanks alot!
Re: Cache replication
Posted by arian487 <ak...@tagged.com>.
Thanks for the informative response. I'll consider using the 'sticky'
addressing as you suggested. The reason cache is so important for me is
because I'm actually doing more processing after the query component to come
up with my query result and I want to avoid that processing as much as
possible. But thanks alot!
--
View this message in context: http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3240853.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Cache replication
Posted by Erick Erickson <er...@gmail.com>.
No, caches are not replicated across slaves. You really have
two choices:
1> use some sort of "sticky" addressing whereby requests
from the same client are sent to the same slave.
2> don't worry about it <G>. Examine your cache stats
to see how often your caches, particularly your
QueryResultCache is utilized. If users aren't
paging into your system, it's probable that your
QueryResultCache is not buying you that much
in the first place. As for the other caches, I usually
work on the assumption that they'll hit a steady
state sometime anyway and you'll get your re-use,
especially if you use you warming queries appropriately.
And all this is moot anyway after any replication, many of
the caches have to be rebuilt. If you have a system that
you can predict what caches will be used, you can always
use warmup queries to make that happen. In particular
look at auto-warming...
Buy propagating caches would be tricky, and I suspect
far more problem than value
Best
Erick
On Tue, Aug 9, 2011 at 7:07 PM, arian487 <ak...@tagged.com> wrote:
> I'm wondering if the caches on all the slaves are replicated across (such as
> queryResultCache). That is to say, if I hit one of my slaves and cache a
> result, and I make a search later and that search happens to hit a different
> slave, will that first cached result be available for use?
>
> This is pretty important because I'm going to have a lot of slaves and if
> this isn't done, then I'd have a high chance of running a lot uncached
> queries.
>
> Thanks :)
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Cache-replication-tp3240708p3240708.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>