You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Siddhesh Shirode <SS...@searchtechnologies.com> on 2011/07/18 16:35:15 UTC
Best practices for Calculationg QPS
Hi Everyone,
I would like to know the best practices or best tools for Calculating QPS in Solr. Thanks.
Thanks,
SIDDHESH SHIRODE
Technical Consultant
M +1 240 274 5183
SEARCH TECHNOLOGIES
THE EXPERT IN THE SEARCH SPACE
www.searchtechnologies.com<http://www.searchtechnologies.com>
Re: Best practices for Calculationg QPS
Posted by Koji Sekiguchi <ko...@r.email.ne.jp>.
(11/07/18 23:35), Siddhesh Shirode wrote:
> Hi Everyone,
>
> I would like to know the best practices or best tools for Calculating QPS in Solr. Thanks.
Just an FYI:
Admin GUI > STATISTICS > QUERY gives you avgRequestsPerSecond for each request handler.
koji
--
http://www.rondhuit.com/en/
Re: Best practices for Calculationg QPS
Posted by Lance Norskog <go...@gmail.com>.
Easiest way to count QPS:
Take one solr log file. Make it have date stamps and log entries on
the same line.
Grab all lines with 'qTime='.
Strip these lines of all text after the timestamp.
Run this Unix program to get counts in line of how many times each
timestamp appears in a row:
uniq -c
Works a treat. After this I make charts in Excel. Use the "X-Y" or
"Scatter plot" chart. Make the timestamp the X dimension, and the
count the Y dimension. This gets you a plot of QPS.
On Mon, Jul 18, 2011 at 5:08 PM, Erick Erickson <er...@gmail.com> wrote:
> Measure. On an index with your real data with real queries <G>...
>
> Being a smart-aleck aside, using something like jMeter is useful. The
> basic idea is that you can use such a tool to fire queries at a Solr
> index, configuring it with some number of threads that all run
> in parallel, and keep upping the number of threads until the server
> falls over.
>
> But it's critical that you use your real data. All of it (i.e. don't run with
> a partial set of data and expect the results to hold when you add the
> rest of the data). It's equally critical that you use real queries that
> reflect what the users actually send at your index.
>
> Of course, with a new app, getting "real" user queries isn't possible,
> and you're forced to guess. Which is much better than nothing, but
> you need to monitor what happens when real users do start using your
> system...
>
> Do be aware that what I have seen when doing this is that your
> QPS will plateau, but the response time for each query will
> increase at some threshold...
>
> FWIW
> Erick
>
> On Mon, Jul 18, 2011 at 10:35 AM, Siddhesh Shirode
> <SS...@searchtechnologies.com> wrote:
>> Hi Everyone,
>>
>> I would like to know the best practices or best tools for Calculating QPS in Solr. Thanks.
>>
>> Thanks,
>> SIDDHESH SHIRODE
>> Technical Consultant
>>
>> M +1 240 274 5183
>>
>> SEARCH TECHNOLOGIES
>> THE EXPERT IN THE SEARCH SPACE
>> www.searchtechnologies.com<http://www.searchtechnologies.com>
>>
>>
>
--
Lance Norskog
goksron@gmail.com
Re: Best practices for Calculationg QPS
Posted by Erick Erickson <er...@gmail.com>.
Measure. On an index with your real data with real queries <G>...
Being a smart-aleck aside, using something like jMeter is useful. The
basic idea is that you can use such a tool to fire queries at a Solr
index, configuring it with some number of threads that all run
in parallel, and keep upping the number of threads until the server
falls over.
But it's critical that you use your real data. All of it (i.e. don't run with
a partial set of data and expect the results to hold when you add the
rest of the data). It's equally critical that you use real queries that
reflect what the users actually send at your index.
Of course, with a new app, getting "real" user queries isn't possible,
and you're forced to guess. Which is much better than nothing, but
you need to monitor what happens when real users do start using your
system...
Do be aware that what I have seen when doing this is that your
QPS will plateau, but the response time for each query will
increase at some threshold...
FWIW
Erick
On Mon, Jul 18, 2011 at 10:35 AM, Siddhesh Shirode
<SS...@searchtechnologies.com> wrote:
> Hi Everyone,
>
> I would like to know the best practices or best tools for Calculating QPS in Solr. Thanks.
>
> Thanks,
> SIDDHESH SHIRODE
> Technical Consultant
>
> M +1 240 274 5183
>
> SEARCH TECHNOLOGIES
> THE EXPERT IN THE SEARCH SPACE
> www.searchtechnologies.com<http://www.searchtechnologies.com>
>
>