You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mathias Lux <ml...@itec.uni-klu.ac.at> on 2013/10/24 13:35:33 UTC
Query & result caching with custom functions
Hi all!
Got a question on the Solr cache :)
I've written a custom function, which is able to provide a distance
based on some DocValues to re-sort result lists. This basically works
great, but we've got the problem that if I don't change the query, but
the function parameters, Solr delivers a cached result without
re-ordering. I turned off caching and see there, problem solved. But
of course this is not a avenue I want to pursue further as it doesn't
make sense for a prodcutive system.
Do you have any ideas (beyond fake query modification and turning off
caching) to counteract?
btw. I'm using Solr 4.4 (so if you are aware of the issue and it has
been resolved in 4.5 I'll port it :) The code I'm using is at
https://bitbucket.org/dermotte/liresolr
regards,
Mathias
--
Dr. Mathias Lux
Assistant Professor, Klagenfurt University, Austria
http://tinyurl.com/mlux-itec
Re: Query & result caching with custom functions
Posted by Mathias Lux <ml...@itec.uni-klu.ac.at>.
Hi Joel,
I just tested with custom equals and hashcode ... what I basically did
is that I created a string object based on all the function values and
used this for the equals (with an instanceof) and for the hash method.
The result was quite the same as before, all treszults are cashed
unless I set the queryResultCache size to 0 in the solrconfig.xml
cheers,
Mathias
On Thu, Oct 24, 2013 at 4:51 PM, Joel Bernstein <jo...@gmail.com> wrote:
> Mathias,
>
> I'd have to do a close review of the function sort code to be sure, but I
> suspect if you implement the equals() method on the ValueSource it should
> solve your caching issue. Also implement hashCode().
>
> Joel
>
>
> On Thu, Oct 24, 2013 at 10:35 AM, Shawn Heisey <so...@elyograg.org> wrote:
>
>> On 10/24/2013 5:35 AM, Mathias Lux wrote:
>> > I've written a custom function, which is able to provide a distance
>> > based on some DocValues to re-sort result lists. This basically works
>> > great, but we've got the problem that if I don't change the query, but
>> > the function parameters, Solr delivers a cached result without
>> > re-ordering. I turned off caching and see there, problem solved. But
>> > of course this is not a avenue I want to pursue further as it doesn't
>> > make sense for a prodcutive system.
>> >
>> > Do you have any ideas (beyond fake query modification and turning off
>> > caching) to counteract?
>> >
>> > btw. I'm using Solr 4.4 (so if you are aware of the issue and it has
>> > been resolved in 4.5 I'll port it :) The code I'm using is at
>> > https://bitbucket.org/dermotte/liresolr
>>
>> I suspect that the queryResultCache is not paying attention to the fact
>> that parameters for your plugin have changed. This probably means that
>> your plugin must somehow inform the "cache check" code that something
>> HAS changed.
>>
>> How you actually do this is a mystery to me because it involves parts of
>> the code that are beyond my understanding, but it MIGHT involve making
>> sure that parameters related to your code are saved as part of the entry
>> that goes into the cache.
>>
>> Thanks,
>> Shawn
>>
>>
--
PD Dr. Mathias Lux
Associate Professor, Klagenfurt University, Austria
http://tinyurl.com/mlux-itec
Re: Query & result caching with custom functions
Posted by Mathias Lux <ml...@itec.uni-klu.ac.at>.
That's a possibility, I'll try that and report on the effects. Thanks,
Mathias
Am 24.10.2013 16:52 schrieb "Joel Bernstein" <jo...@gmail.com>:
> Mathias,
>
> I'd have to do a close review of the function sort code to be sure, but I
> suspect if you implement the equals() method on the ValueSource it should
> solve your caching issue. Also implement hashCode().
>
> Joel
>
>
> On Thu, Oct 24, 2013 at 10:35 AM, Shawn Heisey <so...@elyograg.org> wrote:
>
> > On 10/24/2013 5:35 AM, Mathias Lux wrote:
> > > I've written a custom function, which is able to provide a distance
> > > based on some DocValues to re-sort result lists. This basically works
> > > great, but we've got the problem that if I don't change the query, but
> > > the function parameters, Solr delivers a cached result without
> > > re-ordering. I turned off caching and see there, problem solved. But
> > > of course this is not a avenue I want to pursue further as it doesn't
> > > make sense for a prodcutive system.
> > >
> > > Do you have any ideas (beyond fake query modification and turning off
> > > caching) to counteract?
> > >
> > > btw. I'm using Solr 4.4 (so if you are aware of the issue and it has
> > > been resolved in 4.5 I'll port it :) The code I'm using is at
> > > https://bitbucket.org/dermotte/liresolr
> >
> > I suspect that the queryResultCache is not paying attention to the fact
> > that parameters for your plugin have changed. This probably means that
> > your plugin must somehow inform the "cache check" code that something
> > HAS changed.
> >
> > How you actually do this is a mystery to me because it involves parts of
> > the code that are beyond my understanding, but it MIGHT involve making
> > sure that parameters related to your code are saved as part of the entry
> > that goes into the cache.
> >
> > Thanks,
> > Shawn
> >
> >
>
Re: Query & result caching with custom functions
Posted by Joel Bernstein <jo...@gmail.com>.
Mathias,
I'd have to do a close review of the function sort code to be sure, but I
suspect if you implement the equals() method on the ValueSource it should
solve your caching issue. Also implement hashCode().
Joel
On Thu, Oct 24, 2013 at 10:35 AM, Shawn Heisey <so...@elyograg.org> wrote:
> On 10/24/2013 5:35 AM, Mathias Lux wrote:
> > I've written a custom function, which is able to provide a distance
> > based on some DocValues to re-sort result lists. This basically works
> > great, but we've got the problem that if I don't change the query, but
> > the function parameters, Solr delivers a cached result without
> > re-ordering. I turned off caching and see there, problem solved. But
> > of course this is not a avenue I want to pursue further as it doesn't
> > make sense for a prodcutive system.
> >
> > Do you have any ideas (beyond fake query modification and turning off
> > caching) to counteract?
> >
> > btw. I'm using Solr 4.4 (so if you are aware of the issue and it has
> > been resolved in 4.5 I'll port it :) The code I'm using is at
> > https://bitbucket.org/dermotte/liresolr
>
> I suspect that the queryResultCache is not paying attention to the fact
> that parameters for your plugin have changed. This probably means that
> your plugin must somehow inform the "cache check" code that something
> HAS changed.
>
> How you actually do this is a mystery to me because it involves parts of
> the code that are beyond my understanding, but it MIGHT involve making
> sure that parameters related to your code are saved as part of the entry
> that goes into the cache.
>
> Thanks,
> Shawn
>
>
Re: Query & result caching with custom functions
Posted by Shawn Heisey <so...@elyograg.org>.
On 10/24/2013 5:35 AM, Mathias Lux wrote:
> I've written a custom function, which is able to provide a distance
> based on some DocValues to re-sort result lists. This basically works
> great, but we've got the problem that if I don't change the query, but
> the function parameters, Solr delivers a cached result without
> re-ordering. I turned off caching and see there, problem solved. But
> of course this is not a avenue I want to pursue further as it doesn't
> make sense for a prodcutive system.
>
> Do you have any ideas (beyond fake query modification and turning off
> caching) to counteract?
>
> btw. I'm using Solr 4.4 (so if you are aware of the issue and it has
> been resolved in 4.5 I'll port it :) The code I'm using is at
> https://bitbucket.org/dermotte/liresolr
I suspect that the queryResultCache is not paying attention to the fact
that parameters for your plugin have changed. This probably means that
your plugin must somehow inform the "cache check" code that something
HAS changed.
How you actually do this is a mystery to me because it involves parts of
the code that are beyond my understanding, but it MIGHT involve making
sure that parameters related to your code are saved as part of the entry
that goes into the cache.
Thanks,
Shawn