You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jay Potharaju <js...@gmail.com> on 2016/04/26 00:48:57 UTC
measuring query performance & qps per node
Hi,
I am trying to measure how will are queries performing ie how long are they
taking. In order to measure query speed I am using solrmeter with 50k
unique filter queries. And then checking if any of the queries are slower
than 50ms. Is this a good approach to measure query performance?
Are there any guidelines on how to measure if a given instance can handle a
given number of qps(query per sec)? For example if my doc size is 30
million docs and index size is 40 GB of data and the RAM on the instance is
60 GB, then how many qps can it handle? Or is this a hard question to
answer and it depends on the load and type of query running at a given time.
--
Thanks
Jay
Re: measuring query performance & qps per node
Posted by Erick Erickson <er...@gmail.com>.
In SolrCloud you can collect stats on pivot facets, see:
https://issues.apache.org/jira/browse/SOLR-6351
There are more buckets to count into and in SolrCloud you
have extra work to reconcile the partial results from
different shards.....
Best,
Erick
On Mon, Apr 25, 2016 at 8:50 PM, Jay Potharaju <js...@gmail.com> wrote:
> Thanks for the response Erick. I knew that it would depend on the number of
> factors like you mentioned.I just wanted to know whether a good
> combination of queries, facets & filters should be a good estimate of how
> solr might behave.
>
> what did you mean by "Add stats to pivots in Cloud mode."
>
> Thanks
>
> On Mon, Apr 25, 2016 at 5:05 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
>> Impossible to answer. For instance, a facet query can be very
>> heavy-duty. Add stats
>> to pivots in Cloud mode.
>>
>> As for using a bunch of fq clauses, It Depends (tm). If your expected usage
>> pattern is all queries like 'q=*:*&fq=clause1&fq=clause2" then it's
>> fine. It totally
>> falls down if, for instance, you have a bunch of facets. Or grouping.
>> Or.....
>>
>> Best,
>> Erick
>>
>> On Mon, Apr 25, 2016 at 3:48 PM, Jay Potharaju <js...@gmail.com>
>> wrote:
>> > Hi,
>> > I am trying to measure how will are queries performing ie how long are
>> they
>> > taking. In order to measure query speed I am using solrmeter with 50k
>> > unique filter queries. And then checking if any of the queries are slower
>> > than 50ms. Is this a good approach to measure query performance?
>> >
>> > Are there any guidelines on how to measure if a given instance can
>> handle a
>> > given number of qps(query per sec)? For example if my doc size is 30
>> > million docs and index size is 40 GB of data and the RAM on the instance
>> is
>> > 60 GB, then how many qps can it handle? Or is this a hard question to
>> > answer and it depends on the load and type of query running at a given
>> time.
>> >
>> > --
>> > Thanks
>> > Jay
>>
>
>
>
> --
> Thanks
> Jay Potharaju
Re: measuring query performance & qps per node
Posted by Jay Potharaju <js...@gmail.com>.
Thanks for the response Erick. I knew that it would depend on the number of
factors like you mentioned.I just wanted to know whether a good
combination of queries, facets & filters should be a good estimate of how
solr might behave.
what did you mean by "Add stats to pivots in Cloud mode."
Thanks
On Mon, Apr 25, 2016 at 5:05 PM, Erick Erickson <er...@gmail.com>
wrote:
> Impossible to answer. For instance, a facet query can be very
> heavy-duty. Add stats
> to pivots in Cloud mode.
>
> As for using a bunch of fq clauses, It Depends (tm). If your expected usage
> pattern is all queries like 'q=*:*&fq=clause1&fq=clause2" then it's
> fine. It totally
> falls down if, for instance, you have a bunch of facets. Or grouping.
> Or.....
>
> Best,
> Erick
>
> On Mon, Apr 25, 2016 at 3:48 PM, Jay Potharaju <js...@gmail.com>
> wrote:
> > Hi,
> > I am trying to measure how will are queries performing ie how long are
> they
> > taking. In order to measure query speed I am using solrmeter with 50k
> > unique filter queries. And then checking if any of the queries are slower
> > than 50ms. Is this a good approach to measure query performance?
> >
> > Are there any guidelines on how to measure if a given instance can
> handle a
> > given number of qps(query per sec)? For example if my doc size is 30
> > million docs and index size is 40 GB of data and the RAM on the instance
> is
> > 60 GB, then how many qps can it handle? Or is this a hard question to
> > answer and it depends on the load and type of query running at a given
> time.
> >
> > --
> > Thanks
> > Jay
>
--
Thanks
Jay Potharaju
Re: measuring query performance & qps per node
Posted by Erick Erickson <er...@gmail.com>.
Impossible to answer. For instance, a facet query can be very
heavy-duty. Add stats
to pivots in Cloud mode.
As for using a bunch of fq clauses, It Depends (tm). If your expected usage
pattern is all queries like 'q=*:*&fq=clause1&fq=clause2" then it's
fine. It totally
falls down if, for instance, you have a bunch of facets. Or grouping. Or.....
Best,
Erick
On Mon, Apr 25, 2016 at 3:48 PM, Jay Potharaju <js...@gmail.com> wrote:
> Hi,
> I am trying to measure how will are queries performing ie how long are they
> taking. In order to measure query speed I am using solrmeter with 50k
> unique filter queries. And then checking if any of the queries are slower
> than 50ms. Is this a good approach to measure query performance?
>
> Are there any guidelines on how to measure if a given instance can handle a
> given number of qps(query per sec)? For example if my doc size is 30
> million docs and index size is 40 GB of data and the RAM on the instance is
> 60 GB, then how many qps can it handle? Or is this a hard question to
> answer and it depends on the load and type of query running at a given time.
>
> --
> Thanks
> Jay