You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Amjad Khan <am...@gmail.com> on 2018/10/24 16:19:17 UTC
Slow Response for less volume
Hi,
We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
Any clue will be appreciated.
Thanks
Re: Slow Response for less volume
Posted by Walter Underwood <wu...@wunderwood.org>.
If your cache is 2048 entries, then every one of those 1600 queries is in cache.
Our logs typically have about a million lines, with distinct queries distributed according to the Zipf law. Some common queries, a long tail, that sort of thing.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Oct 24, 2018, at 10:02 AM, Amjad Khan <am...@gmail.com> wrote:
>
> Thanks Wunder for this prompt response.
>
> We are testing with 1600 different text to search with Jmeter and that keeps running continuously, and keep running continuously means cache has been built and there should be better response now. Doesn’t it?
>
> Thanks
>
>
>
>> On Oct 24, 2018, at 12:20 PM, Walter Underwood <wu...@wunderwood.org> wrote:
>>
>> Are you testing with a small number of queries? If your cache is larger than the number of queries in your benchmark, the first round will load the cache, then everything will be super fast.
>>
>> Load testing a system with caches is hard.
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>>> On Oct 24, 2018, at 9:19 AM, Amjad Khan <am...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
>>>
>>> We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
>>>
>>> Any clue will be appreciated.
>>>
>>> Thanks
>>
>
Re: Slow Response for less volume
Posted by Amjad Khan <am...@gmail.com>.
Thanks Wunder for this prompt response.
We are testing with 1600 different text to search with Jmeter and that keeps running continuously, and keep running continuously means cache has been built and there should be better response now. Doesn’t it?
Thanks
> On Oct 24, 2018, at 12:20 PM, Walter Underwood <wu...@wunderwood.org> wrote:
>
> Are you testing with a small number of queries? If your cache is larger than the number of queries in your benchmark, the first round will load the cache, then everything will be super fast.
>
> Load testing a system with caches is hard.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
>> On Oct 24, 2018, at 9:19 AM, Amjad Khan <am...@gmail.com> wrote:
>>
>> Hi,
>>
>> We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
>>
>> We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
>>
>> Any clue will be appreciated.
>>
>> Thanks
>
Re: Slow Response for less volume
Posted by Walter Underwood <wu...@wunderwood.org>.
But a zero size cache doesn’t give realistic benchmarks. It makes things slower than they will be in production.
We do this:
1. Collect production logs.
2. Split the logs into a warming log and and a benchmark log. The warming log should be at least as large as the query result cache.
3. Run the warming log with four threads (unlikely to overload the system).
4. Run the benchmark with a controlled requests/minute and enough threads to keep up with that. Might be a few hundred with a large, slow cluster. Run for at least an hour.
5. Analyze the results into percentile response times for each request handler. Warn about any errors or a benchmark that takes too long.
Then reepeat. Oh, yeah, load the prod content first.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Oct 24, 2018, at 9:52 AM, Erick Erickson <er...@gmail.com> wrote:
>
> You can set your queryResultCache and filterCache "size" parameter to
> zero in solrconfig.xml to disable those caches.
> On Wed, Oct 24, 2018 at 9:21 AM Walter Underwood <wu...@wunderwood.org> wrote:
>>
>> Are you testing with a small number of queries? If your cache is larger than the number of queries in your benchmark, the first round will load the cache, then everything will be super fast.
>>
>> Load testing a system with caches is hard.
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>>> On Oct 24, 2018, at 9:19 AM, Amjad Khan <am...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
>>>
>>> We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
>>>
>>> Any clue will be appreciated.
>>>
>>> Thanks
>>
Re: Slow Response for less volume
Posted by Amjad Khan <am...@gmail.com>.
Thanks Erick,
But do you think that disabling the cache will increase the response time instead of solving the problem here.
> On Oct 24, 2018, at 12:52 PM, Erick Erickson <er...@gmail.com> wrote:
>
> queryResultCache
Re: Slow Response for less volume
Posted by Erick Erickson <er...@gmail.com>.
You can set your queryResultCache and filterCache "size" parameter to
zero in solrconfig.xml to disable those caches.
On Wed, Oct 24, 2018 at 9:21 AM Walter Underwood <wu...@wunderwood.org> wrote:
>
> Are you testing with a small number of queries? If your cache is larger than the number of queries in your benchmark, the first round will load the cache, then everything will be super fast.
>
> Load testing a system with caches is hard.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Oct 24, 2018, at 9:19 AM, Amjad Khan <am...@gmail.com> wrote:
> >
> > Hi,
> >
> > We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
> >
> > We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
> >
> > Any clue will be appreciated.
> >
> > Thanks
>
Re: Slow Response for less volume
Posted by Walter Underwood <wu...@wunderwood.org>.
Are you testing with a small number of queries? If your cache is larger than the number of queries in your benchmark, the first round will load the cache, then everything will be super fast.
Load testing a system with caches is hard.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Oct 24, 2018, at 9:19 AM, Amjad Khan <am...@gmail.com> wrote:
>
> Hi,
>
> We recently moved to Solr Cloud (Google) with 4 nodes and have very limited number of data.
>
> We are facing very weird issue here, solr cluster response time for query is high when we have less number of hit and the moment we run our test to hit the solr cluster hard we see better response in 10ms.
>
> Any clue will be appreciated.
>
> Thanks
Re: Slow Response for less volume
Posted by Deepak Goel <de...@gmail.com>.
Are you getting errors in Jmeter?
On Wed, 24 Oct 2018, 21:49 Amjad Khan, <am...@gmail.com> wrote:
> Hi,
>
> We recently moved to Solr Cloud (Google) with 4 nodes and have very
> limited number of data.
>
> We are facing very weird issue here, solr cluster response time for query
> is high when we have less number of hit and the moment we run our test to
> hit the solr cluster hard we see better response in 10ms.
>
> Any clue will be appreciated.
>
> Thanks