You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Lord Khan Han <kh...@gmail.com> on 2011/10/01 23:55:40 UTC
SOLR HttpCache Qtime
Hi,
Is there anyway to get correct Qtime when we use http caching ? I think Solr
caching also the Qtime so giving the the same Qtime in response what ever
takes it to finish .. How I can set Qtime correcly from solr when I use
http caching On.
thanks
Re: SOLR HttpCache Qtime
Posted by Lord Khan Han <kh...@gmail.com>.
Hi ,
sorry for delay.. I got cold and still tears in my eyes .. These viruses
getting stronger each day...
what I have done is in solrconfig.xml :
<filterCache class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="50"/>
<queryResultCache class="solr.FastLRUCache"
size="512"
initialSize="512"
autowarmCount="200"/>
<documentCache class="solr.FastLRUCache"
size="10000"
initialSize="10000"
autowarmCount="0"/>
<enableLazyFieldLoading>true</enableLazyFieldLoading>
<queryResultWindowSize>20</queryResultWindowSize>
<queryResultMaxDocsCached>1000</queryResultMaxDocsCached>
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<!--
<lst><str name="q">solr</str><str name="sort">price
asc</str></lst>
<lst><str name="q">rocks</str><str name="sort">weight
asc</str></lst>
-->
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<str name="q">static firstSearcher warming in solrconfig.xml</str>
</lst>
</arr>
</listener>
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers>
</query>
<requestDispatcher handleSelect="true" >
<requestParsers enableRemoteStreaming="true"
multipartUploadLimitInKB="2048000" />
<!--<httpCaching never304="true" /> -->
<httpCaching lastModifiedFrom="openTime" etagSeed="solr" />
</requestDispatcher>
I am not using any thirdparty http caching and all queries tested through
solradmin 7124 port directly.
when I quoto <!--<httpCaching never304="true" /> --> solr caching the
query results and using it as an httpcache. And queries getting very fast
below 10 ms. Qtime in the solr response xml still the same Qtime before
cached as below.. When I look the solr log there is no hits but solr
responding normally.
ie:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">158</int>
<lst name="params">
<str name="explainOther"/>
<str name="fl">id,title,url,mrank</str>
<str name="indent">on</str>
<str name="start">0</str>
<str name="q">"fffff" +ffff -fff</str>
<str name="hl.fl"/>
<str name="qt"/>
<str name="wt"/>
<str name="fq"/>
<str name="version">2.2</str>
<str name="rows">10</str>
</lst>
</lst>
<result name="response" numFound="4575" start="0">
so this is why I confused and asked a way . May be from the cold..
thanks
On Mon, Oct 10, 2011 at 1:07 PM, Tor Henning Ueland
<to...@gmail.com>wrote:
> On Sat, Oct 1, 2011 at 11:55 PM, Lord Khan Han <kh...@gmail.com>
> wrote:
> > Is there anyway to get correct Qtime when we use http caching ? I think
> Solr
> > caching also the Qtime so giving the the same Qtime in response what ever
> > takes it to finish .. How I can set Qtime correcly from solr when I use
> > http caching On.
>
> Don`t bother with it at Varnish/Squid level, just time how long it
> takes to perform a query() in your application.
>
> --
> Mvh
> Tor Henning Ueland
>
Re: SOLR HttpCache Qtime
Posted by Tor Henning Ueland <to...@gmail.com>.
On Sat, Oct 1, 2011 at 11:55 PM, Lord Khan Han <kh...@gmail.com> wrote:
> Is there anyway to get correct Qtime when we use http caching ? I think Solr
> caching also the Qtime so giving the the same Qtime in response what ever
> takes it to finish .. How I can set Qtime correcly from solr when I use
> http caching On.
Don`t bother with it at Varnish/Squid level, just time how long it
takes to perform a query() in your application.
--
Mvh
Tor Henning Ueland
Re: SOLR HttpCache Qtime
Posted by Lance Norskog <go...@gmail.com>.
Solr supports having the browser cache the results. If your client code
supports this caching, or your code goes through an HTTP cacher like Squid,
it could return a cached page for a query. Is this what you mean?
On Tue, Oct 4, 2011 at 4:55 PM, Nicholas Chase <nc...@earthlink.net> wrote:
> Seems to me what you're asking is how to have an accurate query time when
> you're getting a response that's been cached by an HTTP cache. This might
> be from the browser, or from a proxy, or from something else, but it's not
> from Solr. The reason that the QTime doesn't change is because it's the
> entire response -- results, parameters, Qtime, and all -- that's cached.
> Solr isn't making a new request; it doesn't even know that a request has
> been made. So if you do 6 requests, and the last 5 come from the cache,
> Solr has done only one request, with one Qtime.
>
> So it sounds to me that you are looking for the RESPONSE time, which would
> be different from the QTime, and would, I suppose, come from your
> application, and not from Solr.
>
> ---- Nick
>
>
> On 10/4/2011 7:44 PM, Erick Erickson wrote:
>
>> Still doesn't make sense to me. There is no
>> Solr HTTP cache that I know of. There is a
>> queryResultCache. There is a filterCache.
>> There is a documentCache. There's may
>> even be custom cache implementations.
>> There's a fieldValueCache. There's
>> no http cache internal to Solr as far as I
>> can tell.
>>
>> If you're asking if documents returned from
>> the queryResultCache have QTimes that
>> reflect the actual time spent (near 0), I'm
>> pretty sure the answer is "yes".
>>
>> If this doesn't answer your question, please
>> take the time to formulate a complete question.
>> It'll get you your answers quicker than multiple
>> twitter-style exchanges.
>>
>> Best
>> Erick
>>
>> On Tue, Oct 4, 2011 at 2:22 PM, Lord Khan Han<kh...@gmail.com>
>> wrote:
>>
>>> I just want to be sure.. because its solr internal HTTP cache.. not an
>>> outside httpcacher
>>>
>>> On Tue, Oct 4, 2011 at 5:39 PM, Erick Erickson<er...@gmail.com>
>>> >wrote:
>>>
>>> But if the HTTP cache is what's returning the value,
>>>> Solr never sees anything at all, right? So Solr
>>>> doesn't have a chance to do anything here.
>>>>
>>>> Best
>>>> Erick
>>>>
>>>> On Tue, Oct 4, 2011 at 9:24 AM, Lord Khan Han<kh...@gmail.com>
>>>> wrote:
>>>>
>>>>> We are using this Qtime field and publishing in our front web. Even the
>>>>> httpCache decreasing the Qtime in reality, its still using the cached
>>>>> old
>>>>> Qtime value . We can use our internal qtime instead of Solr's but I
>>>>> just
>>>>> wonder is there any way to say Solr if its coming httpCache
>>>>> re-calculate
>>>>> the Qtime.
>>>>>
>>>>> On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson<erickerickson@gmail.**
>>>>> com <er...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Why do you want to? QTime is the time Solr
>>>>>> spends searching. The cached value will,
>>>>>> indeed, be from the query that filled
>>>>>> in the HTTP cache. But what are you doing
>>>>>> with that information that you want to "correct"
>>>>>> it?
>>>>>>
>>>>>> That said, I have no clue how you'd attempt to
>>>>>> do this.
>>>>>>
>>>>>> Best
>>>>>> Erick
>>>>>>
>>>>>> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han<khanuniverse1@gmail.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there anyway to get correct Qtime when we use http caching ? I
>>>>>>>
>>>>>> think
>>>>
>>>>> Solr
>>>>>>
>>>>>>> caching also the Qtime so giving the the same Qtime in response what
>>>>>>>
>>>>>> ever
>>>>
>>>>> takes it to finish .. How I can set Qtime correcly from solr when I
>>>>>>>
>>>>>> use
>>>>
>>>>> http caching On.
>>>>>>>
>>>>>>> thanks
>>>>>>>
>>>>>>>
>>
--
Lance Norskog
goksron@gmail.com
Re: SOLR HttpCache Qtime
Posted by Nicholas Chase <nc...@earthlink.net>.
Seems to me what you're asking is how to have an accurate query time
when you're getting a response that's been cached by an HTTP cache.
This might be from the browser, or from a proxy, or from something else,
but it's not from Solr. The reason that the QTime doesn't change is
because it's the entire response -- results, parameters, Qtime, and all
-- that's cached. Solr isn't making a new request; it doesn't even know
that a request has been made. So if you do 6 requests, and the last 5
come from the cache, Solr has done only one request, with one Qtime.
So it sounds to me that you are looking for the RESPONSE time, which
would be different from the QTime, and would, I suppose, come from your
application, and not from Solr.
---- Nick
On 10/4/2011 7:44 PM, Erick Erickson wrote:
> Still doesn't make sense to me. There is no
> Solr HTTP cache that I know of. There is a
> queryResultCache. There is a filterCache.
> There is a documentCache. There's may
> even be custom cache implementations.
> There's a fieldValueCache. There's
> no http cache internal to Solr as far as I
> can tell.
>
> If you're asking if documents returned from
> the queryResultCache have QTimes that
> reflect the actual time spent (near 0), I'm
> pretty sure the answer is "yes".
>
> If this doesn't answer your question, please
> take the time to formulate a complete question.
> It'll get you your answers quicker than multiple
> twitter-style exchanges.
>
> Best
> Erick
>
> On Tue, Oct 4, 2011 at 2:22 PM, Lord Khan Han<kh...@gmail.com> wrote:
>> I just want to be sure.. because its solr internal HTTP cache.. not an
>> outside httpcacher
>>
>> On Tue, Oct 4, 2011 at 5:39 PM, Erick Erickson<er...@gmail.com>wrote:
>>
>>> But if the HTTP cache is what's returning the value,
>>> Solr never sees anything at all, right? So Solr
>>> doesn't have a chance to do anything here.
>>>
>>> Best
>>> Erick
>>>
>>> On Tue, Oct 4, 2011 at 9:24 AM, Lord Khan Han<kh...@gmail.com>
>>> wrote:
>>>> We are using this Qtime field and publishing in our front web. Even the
>>>> httpCache decreasing the Qtime in reality, its still using the cached old
>>>> Qtime value . We can use our internal qtime instead of Solr's but I just
>>>> wonder is there any way to say Solr if its coming httpCache re-calculate
>>>> the Qtime.
>>>>
>>>> On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson<erickerickson@gmail.com
>>>> wrote:
>>>>
>>>>> Why do you want to? QTime is the time Solr
>>>>> spends searching. The cached value will,
>>>>> indeed, be from the query that filled
>>>>> in the HTTP cache. But what are you doing
>>>>> with that information that you want to "correct"
>>>>> it?
>>>>>
>>>>> That said, I have no clue how you'd attempt to
>>>>> do this.
>>>>>
>>>>> Best
>>>>> Erick
>>>>>
>>>>> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han<kh...@gmail.com>
>>>>> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Is there anyway to get correct Qtime when we use http caching ? I
>>> think
>>>>> Solr
>>>>>> caching also the Qtime so giving the the same Qtime in response what
>>> ever
>>>>>> takes it to finish .. How I can set Qtime correcly from solr when I
>>> use
>>>>>> http caching On.
>>>>>>
>>>>>> thanks
>>>>>>
>
Re: SOLR HttpCache Qtime
Posted by Erick Erickson <er...@gmail.com>.
Still doesn't make sense to me. There is no
Solr HTTP cache that I know of. There is a
queryResultCache. There is a filterCache.
There is a documentCache. There's may
even be custom cache implementations.
There's a fieldValueCache. There's
no http cache internal to Solr as far as I
can tell.
If you're asking if documents returned from
the queryResultCache have QTimes that
reflect the actual time spent (near 0), I'm
pretty sure the answer is "yes".
If this doesn't answer your question, please
take the time to formulate a complete question.
It'll get you your answers quicker than multiple
twitter-style exchanges.
Best
Erick
On Tue, Oct 4, 2011 at 2:22 PM, Lord Khan Han <kh...@gmail.com> wrote:
> I just want to be sure.. because its solr internal HTTP cache.. not an
> outside httpcacher
>
> On Tue, Oct 4, 2011 at 5:39 PM, Erick Erickson <er...@gmail.com>wrote:
>
>> But if the HTTP cache is what's returning the value,
>> Solr never sees anything at all, right? So Solr
>> doesn't have a chance to do anything here.
>>
>> Best
>> Erick
>>
>> On Tue, Oct 4, 2011 at 9:24 AM, Lord Khan Han <kh...@gmail.com>
>> wrote:
>> > We are using this Qtime field and publishing in our front web. Even the
>> > httpCache decreasing the Qtime in reality, its still using the cached old
>> > Qtime value . We can use our internal qtime instead of Solr's but I just
>> > wonder is there any way to say Solr if its coming httpCache re-calculate
>> > the Qtime.
>> >
>> > On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson <erickerickson@gmail.com
>> >wrote:
>> >
>> >> Why do you want to? QTime is the time Solr
>> >> spends searching. The cached value will,
>> >> indeed, be from the query that filled
>> >> in the HTTP cache. But what are you doing
>> >> with that information that you want to "correct"
>> >> it?
>> >>
>> >> That said, I have no clue how you'd attempt to
>> >> do this.
>> >>
>> >> Best
>> >> Erick
>> >>
>> >> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han <kh...@gmail.com>
>> >> wrote:
>> >> > Hi,
>> >> >
>> >> > Is there anyway to get correct Qtime when we use http caching ? I
>> think
>> >> Solr
>> >> > caching also the Qtime so giving the the same Qtime in response what
>> ever
>> >> > takes it to finish .. How I can set Qtime correcly from solr when I
>> use
>> >> > http caching On.
>> >> >
>> >> > thanks
>> >> >
>> >>
>> >
>>
>
Re: SOLR HttpCache Qtime
Posted by Lord Khan Han <kh...@gmail.com>.
I just want to be sure.. because its solr internal HTTP cache.. not an
outside httpcacher
On Tue, Oct 4, 2011 at 5:39 PM, Erick Erickson <er...@gmail.com>wrote:
> But if the HTTP cache is what's returning the value,
> Solr never sees anything at all, right? So Solr
> doesn't have a chance to do anything here.
>
> Best
> Erick
>
> On Tue, Oct 4, 2011 at 9:24 AM, Lord Khan Han <kh...@gmail.com>
> wrote:
> > We are using this Qtime field and publishing in our front web. Even the
> > httpCache decreasing the Qtime in reality, its still using the cached old
> > Qtime value . We can use our internal qtime instead of Solr's but I just
> > wonder is there any way to say Solr if its coming httpCache re-calculate
> > the Qtime.
> >
> > On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson <erickerickson@gmail.com
> >wrote:
> >
> >> Why do you want to? QTime is the time Solr
> >> spends searching. The cached value will,
> >> indeed, be from the query that filled
> >> in the HTTP cache. But what are you doing
> >> with that information that you want to "correct"
> >> it?
> >>
> >> That said, I have no clue how you'd attempt to
> >> do this.
> >>
> >> Best
> >> Erick
> >>
> >> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han <kh...@gmail.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > Is there anyway to get correct Qtime when we use http caching ? I
> think
> >> Solr
> >> > caching also the Qtime so giving the the same Qtime in response what
> ever
> >> > takes it to finish .. How I can set Qtime correcly from solr when I
> use
> >> > http caching On.
> >> >
> >> > thanks
> >> >
> >>
> >
>
Re: SOLR HttpCache Qtime
Posted by Erick Erickson <er...@gmail.com>.
But if the HTTP cache is what's returning the value,
Solr never sees anything at all, right? So Solr
doesn't have a chance to do anything here.
Best
Erick
On Tue, Oct 4, 2011 at 9:24 AM, Lord Khan Han <kh...@gmail.com> wrote:
> We are using this Qtime field and publishing in our front web. Even the
> httpCache decreasing the Qtime in reality, its still using the cached old
> Qtime value . We can use our internal qtime instead of Solr's but I just
> wonder is there any way to say Solr if its coming httpCache re-calculate
> the Qtime.
>
> On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson <er...@gmail.com>wrote:
>
>> Why do you want to? QTime is the time Solr
>> spends searching. The cached value will,
>> indeed, be from the query that filled
>> in the HTTP cache. But what are you doing
>> with that information that you want to "correct"
>> it?
>>
>> That said, I have no clue how you'd attempt to
>> do this.
>>
>> Best
>> Erick
>>
>> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han <kh...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > Is there anyway to get correct Qtime when we use http caching ? I think
>> Solr
>> > caching also the Qtime so giving the the same Qtime in response what ever
>> > takes it to finish .. How I can set Qtime correcly from solr when I use
>> > http caching On.
>> >
>> > thanks
>> >
>>
>
Re: SOLR HttpCache Qtime
Posted by Lord Khan Han <kh...@gmail.com>.
We are using this Qtime field and publishing in our front web. Even the
httpCache decreasing the Qtime in reality, its still using the cached old
Qtime value . We can use our internal qtime instead of Solr's but I just
wonder is there any way to say Solr if its coming httpCache re-calculate
the Qtime.
On Tue, Oct 4, 2011 at 4:16 AM, Erick Erickson <er...@gmail.com>wrote:
> Why do you want to? QTime is the time Solr
> spends searching. The cached value will,
> indeed, be from the query that filled
> in the HTTP cache. But what are you doing
> with that information that you want to "correct"
> it?
>
> That said, I have no clue how you'd attempt to
> do this.
>
> Best
> Erick
>
> On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han <kh...@gmail.com>
> wrote:
> > Hi,
> >
> > Is there anyway to get correct Qtime when we use http caching ? I think
> Solr
> > caching also the Qtime so giving the the same Qtime in response what ever
> > takes it to finish .. How I can set Qtime correcly from solr when I use
> > http caching On.
> >
> > thanks
> >
>
Re: SOLR HttpCache Qtime
Posted by Erick Erickson <er...@gmail.com>.
Why do you want to? QTime is the time Solr
spends searching. The cached value will,
indeed, be from the query that filled
in the HTTP cache. But what are you doing
with that information that you want to "correct"
it?
That said, I have no clue how you'd attempt to
do this.
Best
Erick
On Sat, Oct 1, 2011 at 5:55 PM, Lord Khan Han <kh...@gmail.com> wrote:
> Hi,
>
> Is there anyway to get correct Qtime when we use http caching ? I think Solr
> caching also the Qtime so giving the the same Qtime in response what ever
> takes it to finish .. How I can set Qtime correcly from solr when I use
> http caching On.
>
> thanks
>
Re: SOLR HttpCache Qtime
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hello,
I'm not sure I follow. If the response is cached, Qtime will be cached, too, no?
If you are looking to monitor Solr, have a look at http://sematext.com/spm/solr-performance-monitoring/index.html and get in touch if you'd like to try it.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message -----
> From: Lord Khan Han <kh...@gmail.com>
> To: solr-user@lucene.apache.org
> Cc:
> Sent: Saturday, October 1, 2011 5:55 PM
> Subject: SOLR HttpCache Qtime
>
> Hi,
>
> Is there anyway to get correct Qtime when we use http caching ? I think Solr
> caching also the Qtime so giving the the same Qtime in response what ever
> takes it to finish .. How I can set Qtime correcly from solr when I use
> http caching On.
>
> thanks
>