You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov> on 2015/04/03 17:37:47 UTC
Measuring QPS
I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
What is the best way to gather QPS information?
What is the best way to add information like this to the Admin UI, if I decide to take that step?
Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH
Re: Measuring QPS
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Daniel,
See SPM <http://sematext.com/spm/>, which will give you QPS and a bunch of
other Solr, JVM, and OS metrics, along with alerting, anomaly detection,
and not-yet-announced transaction tracing
<https://sematext.atlassian.net/wiki/display/PUBSPM/Transactions+Tracing>.
It has percentiles Wunder mentions. I see others mentioned JMeter. We use
SPM with JMeter pretty regularly when helping clients with Solr performance
issues.
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On Fri, Apr 3, 2015 at 11:37 AM, Davis, Daniel (NIH/NLM) [C] <
daniel.davis@nih.gov> wrote:
> I wanted to gather QPS for our production Solr instances, but I was
> surprised that the Admin UI did not contain this information. We are
> running a mix of versions, but mostly 4.10 at this point. We are not
> using SolrCloud at present; that's part of why I'm checking - I want to
> validate the size of our existing setup and what sort of SolrCloud setup
> would be needed to centralize several of them.
>
> What is the best way to gather QPS information?
>
> What is the best way to add information like this to the Admin UI, if I
> decide to take that step?
>
> Dan Davis, Systems/Applications Architect (Contractor),
> Office of Computer and Communications Systems,
> National Library of Medicine, NIH
>
>
Re: Measuring QPS
Posted by Walter Underwood <wu...@wunderwood.org>.
That sounds neat. Our QA people are moving to Gatling, so we probably won’t change our JMeter approach now.
We use the JMeter Plugs CMDrunner, telling it to generate only CSV.
http://jmeter-plugins.org/wiki/JMeterPluginsCMD/
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
On Apr 6, 2015, at 3:02 PM, Siegfried Goeschl <sg...@gmx.at> wrote:
> Hi Walter,
>
> sort of shameless plug - I ran into similar issues and wrote a JMeter SLA Reporting Backend - https://github.com/sgoeschl/jmeter-sla-report <https://github.com/sgoeschl/jmeter-sla-report>
>
> * It reads the CSV/XML JMeter report file and sorts the response times in logarithmic buckets
> * the XML processor uses a Stax parser to handle huge JTL files (exceeding 1 GB)
> * it also caters for merging JTL files when running multiple JMeter instances
>
> Cheers,
>
> Siegfried Goeschl
>
>
>
>> On 06 Apr 2015, at 22:57, Walter Underwood <wu...@wunderwood.org> wrote:
>>
>> The load testing is the easiest part.
>>
>> We use JMeter to replay the prod logs. We start about a hundred threads and use ConstantThroughputTimer to control the traffic level. JMeter tends to fall over with two much data graphing, so we run it headless. Then we post process with JMeter Plugins to get percentiles.
>>
>> The complicated part of the servlet filter was getting it configured in Tomcat. The code itself is not too bad.
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>> On Apr 6, 2015, at 1:49 PM, Siegfried Goeschl <sg...@gmx.at> wrote:
>>
>>> The good-sounding thing - you can do that easily with JMeter running the GUI or the command-line
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>> On 06 Apr 2015, at 21:35, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>
>>>> This sounds really good:
>>>>
>>>> "For load testing, we replay production logs to test that we meet the SLA at a given traffic level."
>>>>
>>>> The rest sounds complicated. Ah well, that's the job.
>>>>
>>>> -----Original Message-----
>>>> From: Walter Underwood [mailto:wunder@wunderwood.org]
>>>> Sent: Monday, April 06, 2015 2:48 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: Measuring QPS
>>>>
>>>> We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
>>>>
>>>> That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
>>>>
>>>> We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
>>>>
>>>> We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
>>>>
>>>> For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
>>>>
>>>> Walter Underwood
>>>> wunder@wunderwood.org
>>>> http://observer.wunderwood.org/ (my blog)
>>>>
>>>> On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>
>>>>> OK,
>>>>>
>>>>> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
>>>>> I love showing off, however, so please forgive me.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Davis, Daniel (NIH/NLM) [C]
>>>>> Sent: Monday, April 06, 2015 2:25 PM
>>>>> To: solr-user@lucene.apache.org
>>>>> Subject: RE: Measuring QPS
>>>>>
>>>>> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>>>>>
>>>>> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>> Sent: Monday, April 06, 2015 2:17 PM
>>>>> To: solr-user@lucene.apache.org
>>>>> Subject: Re: Measuring QPS
>>>>>
>>>>> Hi Daniel,
>>>>>
>>>>> interesting - I never thought of autocompletion but for keeping track
>>>>> of user behaviour :-)
>>>>>
>>>>> * the numbers are helpful for the online advertisement team to sell
>>>>> campaigns
>>>>> * it is used for sanity checks - sensible queries returning no results
>>>>> or returning too many results
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Siegfried Goeschl
>>>>>
>>>>>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>>>
>>>>>> Siegfried,
>>>>>>
>>>>>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>>>>>
>>>>>> Thanks again,
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>>> Sent: Monday, April 06, 2015 1:42 PM
>>>>>> To: solr-user@lucene.apache.org
>>>>>> Subject: Re: Measuring QPS
>>>>>>
>>>>>> Hi Dan,
>>>>>>
>>>>>> at willhaben.at (customer of mine) two SOLR components were written
>>>>>> for SOLR 3 and ported to SORL 4
>>>>>>
>>>>>> 1) SlowQueryLog which dumps long-running search requests into a log
>>>>>> file
>>>>>>
>>>>>> 2) Most Frequent Search Terms allowing to query & filter the most
>>>>>> frequent user search terms over the browser
>>>>>>
>>>>>> Some notes along the line
>>>>>>
>>>>>>
>>>>>> * For both components I have the "GO" to open source them but I never
>>>>>> had enough time to do that (shame on me) - see
>>>>>> https://issues.apache.org/jira/browse/SOLR-4056
>>>>>>
>>>>>> * The Most Frequent Search Term component actually mimics a SOLR
>>>>>> server you feed the user search terms so this might be a better
>>>>>> solution in the long run. But this requires to have a separate SOLR
>>>>>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>>>>>> moving parts in production :-)
>>>>>>
>>>>>> * If there is sufficient interest I can make a code drop on GitHub
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Siegfried Goeschl
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>>>>
>>>>>>> Siegfried,
>>>>>>>
>>>>>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>>>>>
>>>>>>> Thank you so much,
>>>>>>>
>>>>>>> Dan
>>>>>>>
>>>>>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>>>> Sent: Saturday, April 04, 2015 4:54 AM
>>>>>>> To: solr-user@lucene.apache.org
>>>>>>> Subject: Re: Measuring QPS
>>>>>>>
>>>>>>> Hi Dan,
>>>>>>>
>>>>>>> I'm using JavaMelody for my SOLR production servers - gives you the
>>>>>>> relevant HTTP stats (what's happening now & historical data) plus
>>>>>>> JVM monitoring as additional benefit. The servers are deployed on
>>>>>>> Tomcat so I'm of little help regarding Jetty - having said that
>>>>>>>
>>>>>>> * you need two Jars (javamelody & robin)
>>>>>>> * tinker with web.xml
>>>>>>>
>>>>>>> Here are two of my presentations mentioning JavaMelody (plus some
>>>>>>> other stuff)
>>>>>>>
>>>>>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>>>>>> n
>>>>>>> t
>>>>>>> -to-production-20121210.pdf
>>>>>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developm
>>>>>>> e
>>>>>>> n
>>>>>>> t-to-production-20121210.pdf>
>>>>>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>>>>>> r
>>>>>>> m
>>>>>>> ance-monitoring.pdf
>>>>>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perf
>>>>>>> o
>>>>>>> r
>>>>>>> mance-monitoring.pdf>
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Siegfried Goeschl
>>>>>>>
>>>>>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>>>>>
>>>>>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>>>>>
>>>>>>>>> What is the best way to gather QPS information?
>>>>>>>>>
>>>>>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>>>>>
>>>>>>>> As of Solr 4.1 (three years ago), request rate information is
>>>>>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>>>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>>>>>> and open the handler you wish to examine. You have
>>>>>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>>>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>>>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Shawn
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
Hi Walter,
sort of shameless plug - I ran into similar issues and wrote a JMeter SLA Reporting Backend - https://github.com/sgoeschl/jmeter-sla-report <https://github.com/sgoeschl/jmeter-sla-report>
* It reads the CSV/XML JMeter report file and sorts the response times in logarithmic buckets
* the XML processor uses a Stax parser to handle huge JTL files (exceeding 1 GB)
* it also caters for merging JTL files when running multiple JMeter instances
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 22:57, Walter Underwood <wu...@wunderwood.org> wrote:
>
> The load testing is the easiest part.
>
> We use JMeter to replay the prod logs. We start about a hundred threads and use ConstantThroughputTimer to control the traffic level. JMeter tends to fall over with two much data graphing, so we run it headless. Then we post process with JMeter Plugins to get percentiles.
>
> The complicated part of the servlet filter was getting it configured in Tomcat. The code itself is not too bad.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> On Apr 6, 2015, at 1:49 PM, Siegfried Goeschl <sg...@gmx.at> wrote:
>
>> The good-sounding thing - you can do that easily with JMeter running the GUI or the command-line
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>> On 06 Apr 2015, at 21:35, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>> This sounds really good:
>>>
>>> "For load testing, we replay production logs to test that we meet the SLA at a given traffic level."
>>>
>>> The rest sounds complicated. Ah well, that's the job.
>>>
>>> -----Original Message-----
>>> From: Walter Underwood [mailto:wunder@wunderwood.org]
>>> Sent: Monday, April 06, 2015 2:48 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
>>>
>>> That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
>>>
>>> We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
>>>
>>> We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
>>>
>>> For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
>>>
>>> Walter Underwood
>>> wunder@wunderwood.org
>>> http://observer.wunderwood.org/ (my blog)
>>>
>>> On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>>> OK,
>>>>
>>>> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
>>>> I love showing off, however, so please forgive me.
>>>>
>>>> -----Original Message-----
>>>> From: Davis, Daniel (NIH/NLM) [C]
>>>> Sent: Monday, April 06, 2015 2:25 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: RE: Measuring QPS
>>>>
>>>> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>>>>
>>>> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>>>>
>>>> -----Original Message-----
>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>> Sent: Monday, April 06, 2015 2:17 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: Measuring QPS
>>>>
>>>> Hi Daniel,
>>>>
>>>> interesting - I never thought of autocompletion but for keeping track
>>>> of user behaviour :-)
>>>>
>>>> * the numbers are helpful for the online advertisement team to sell
>>>> campaigns
>>>> * it is used for sanity checks - sensible queries returning no results
>>>> or returning too many results
>>>>
>>>> Cheers,
>>>>
>>>> Siegfried Goeschl
>>>>
>>>>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>>
>>>>> Siegfried,
>>>>>
>>>>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>>>>
>>>>> Thanks again,
>>>>>
>>>>> Dan
>>>>>
>>>>> -----Original Message-----
>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>> Sent: Monday, April 06, 2015 1:42 PM
>>>>> To: solr-user@lucene.apache.org
>>>>> Subject: Re: Measuring QPS
>>>>>
>>>>> Hi Dan,
>>>>>
>>>>> at willhaben.at (customer of mine) two SOLR components were written
>>>>> for SOLR 3 and ported to SORL 4
>>>>>
>>>>> 1) SlowQueryLog which dumps long-running search requests into a log
>>>>> file
>>>>>
>>>>> 2) Most Frequent Search Terms allowing to query & filter the most
>>>>> frequent user search terms over the browser
>>>>>
>>>>> Some notes along the line
>>>>>
>>>>>
>>>>> * For both components I have the "GO" to open source them but I never
>>>>> had enough time to do that (shame on me) - see
>>>>> https://issues.apache.org/jira/browse/SOLR-4056
>>>>>
>>>>> * The Most Frequent Search Term component actually mimics a SOLR
>>>>> server you feed the user search terms so this might be a better
>>>>> solution in the long run. But this requires to have a separate SOLR
>>>>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>>>>> moving parts in production :-)
>>>>>
>>>>> * If there is sufficient interest I can make a code drop on GitHub
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Siegfried Goeschl
>>>>>
>>>>>
>>>>>
>>>>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>>>
>>>>>> Siegfried,
>>>>>>
>>>>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>>>>
>>>>>> Thank you so much,
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>>> Sent: Saturday, April 04, 2015 4:54 AM
>>>>>> To: solr-user@lucene.apache.org
>>>>>> Subject: Re: Measuring QPS
>>>>>>
>>>>>> Hi Dan,
>>>>>>
>>>>>> I'm using JavaMelody for my SOLR production servers - gives you the
>>>>>> relevant HTTP stats (what's happening now & historical data) plus
>>>>>> JVM monitoring as additional benefit. The servers are deployed on
>>>>>> Tomcat so I'm of little help regarding Jetty - having said that
>>>>>>
>>>>>> * you need two Jars (javamelody & robin)
>>>>>> * tinker with web.xml
>>>>>>
>>>>>> Here are two of my presentations mentioning JavaMelody (plus some
>>>>>> other stuff)
>>>>>>
>>>>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>>>>> n
>>>>>> t
>>>>>> -to-production-20121210.pdf
>>>>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developm
>>>>>> e
>>>>>> n
>>>>>> t-to-production-20121210.pdf>
>>>>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>>>>> r
>>>>>> m
>>>>>> ance-monitoring.pdf
>>>>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perf
>>>>>> o
>>>>>> r
>>>>>> mance-monitoring.pdf>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Siegfried Goeschl
>>>>>>
>>>>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>>>>
>>>>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>>>>
>>>>>>>> What is the best way to gather QPS information?
>>>>>>>>
>>>>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>>>>
>>>>>>> As of Solr 4.1 (three years ago), request rate information is
>>>>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>>>>> and open the handler you wish to examine. You have
>>>>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Shawn
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Measuring QPS
Posted by Walter Underwood <wu...@wunderwood.org>.
The load testing is the easiest part.
We use JMeter to replay the prod logs. We start about a hundred threads and use ConstantThroughputTimer to control the traffic level. JMeter tends to fall over with two much data graphing, so we run it headless. Then we post process with JMeter Plugins to get percentiles.
The complicated part of the servlet filter was getting it configured in Tomcat. The code itself is not too bad.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
On Apr 6, 2015, at 1:49 PM, Siegfried Goeschl <sg...@gmx.at> wrote:
> The good-sounding thing - you can do that easily with JMeter running the GUI or the command-line
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 06 Apr 2015, at 21:35, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> This sounds really good:
>>
>> "For load testing, we replay production logs to test that we meet the SLA at a given traffic level."
>>
>> The rest sounds complicated. Ah well, that's the job.
>>
>> -----Original Message-----
>> From: Walter Underwood [mailto:wunder@wunderwood.org]
>> Sent: Monday, April 06, 2015 2:48 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
>>
>> That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
>>
>> We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
>>
>> We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
>>
>> For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
>>
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>> On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>>> OK,
>>>
>>> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
>>> I love showing off, however, so please forgive me.
>>>
>>> -----Original Message-----
>>> From: Davis, Daniel (NIH/NLM) [C]
>>> Sent: Monday, April 06, 2015 2:25 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: RE: Measuring QPS
>>>
>>> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>>>
>>> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>>>
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Monday, April 06, 2015 2:17 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> Hi Daniel,
>>>
>>> interesting - I never thought of autocompletion but for keeping track
>>> of user behaviour :-)
>>>
>>> * the numbers are helpful for the online advertisement team to sell
>>> campaigns
>>> * it is used for sanity checks - sensible queries returning no results
>>> or returning too many results
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>
>>>> Siegfried,
>>>>
>>>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>>>
>>>> Thanks again,
>>>>
>>>> Dan
>>>>
>>>> -----Original Message-----
>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>> Sent: Monday, April 06, 2015 1:42 PM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: Measuring QPS
>>>>
>>>> Hi Dan,
>>>>
>>>> at willhaben.at (customer of mine) two SOLR components were written
>>>> for SOLR 3 and ported to SORL 4
>>>>
>>>> 1) SlowQueryLog which dumps long-running search requests into a log
>>>> file
>>>>
>>>> 2) Most Frequent Search Terms allowing to query & filter the most
>>>> frequent user search terms over the browser
>>>>
>>>> Some notes along the line
>>>>
>>>>
>>>> * For both components I have the "GO" to open source them but I never
>>>> had enough time to do that (shame on me) - see
>>>> https://issues.apache.org/jira/browse/SOLR-4056
>>>>
>>>> * The Most Frequent Search Term component actually mimics a SOLR
>>>> server you feed the user search terms so this might be a better
>>>> solution in the long run. But this requires to have a separate SOLR
>>>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>>>> moving parts in production :-)
>>>>
>>>> * If there is sufficient interest I can make a code drop on GitHub
>>>>
>>>> Cheers,
>>>>
>>>> Siegfried Goeschl
>>>>
>>>>
>>>>
>>>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>>
>>>>> Siegfried,
>>>>>
>>>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>>>
>>>>> Thank you so much,
>>>>>
>>>>> Dan
>>>>>
>>>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>>> Sent: Saturday, April 04, 2015 4:54 AM
>>>>> To: solr-user@lucene.apache.org
>>>>> Subject: Re: Measuring QPS
>>>>>
>>>>> Hi Dan,
>>>>>
>>>>> I'm using JavaMelody for my SOLR production servers - gives you the
>>>>> relevant HTTP stats (what's happening now & historical data) plus
>>>>> JVM monitoring as additional benefit. The servers are deployed on
>>>>> Tomcat so I'm of little help regarding Jetty - having said that
>>>>>
>>>>> * you need two Jars (javamelody & robin)
>>>>> * tinker with web.xml
>>>>>
>>>>> Here are two of my presentations mentioning JavaMelody (plus some
>>>>> other stuff)
>>>>>
>>>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>>>> n
>>>>> t
>>>>> -to-production-20121210.pdf
>>>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developm
>>>>> e
>>>>> n
>>>>> t-to-production-20121210.pdf>
>>>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>>>> r
>>>>> m
>>>>> ance-monitoring.pdf
>>>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perf
>>>>> o
>>>>> r
>>>>> mance-monitoring.pdf>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Siegfried Goeschl
>>>>>
>>>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>>>
>>>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>>>
>>>>>>> What is the best way to gather QPS information?
>>>>>>>
>>>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>>>
>>>>>> As of Solr 4.1 (three years ago), request rate information is
>>>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>>>> and open the handler you wish to examine. You have
>>>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
The good-sounding thing - you can do that easily with JMeter running the GUI or the command-line
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 21:35, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> This sounds really good:
>
> "For load testing, we replay production logs to test that we meet the SLA at a given traffic level."
>
> The rest sounds complicated. Ah well, that's the job.
>
> -----Original Message-----
> From: Walter Underwood [mailto:wunder@wunderwood.org]
> Sent: Monday, April 06, 2015 2:48 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
>
> That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
>
> We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
>
> We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
>
> For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
>
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
>> OK,
>>
>> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
>> I love showing off, however, so please forgive me.
>>
>> -----Original Message-----
>> From: Davis, Daniel (NIH/NLM) [C]
>> Sent: Monday, April 06, 2015 2:25 PM
>> To: solr-user@lucene.apache.org
>> Subject: RE: Measuring QPS
>>
>> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>>
>> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Monday, April 06, 2015 2:17 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Daniel,
>>
>> interesting - I never thought of autocompletion but for keeping track
>> of user behaviour :-)
>>
>> * the numbers are helpful for the online advertisement team to sell
>> campaigns
>> * it is used for sanity checks - sensible queries returning no results
>> or returning too many results
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>> Siegfried,
>>>
>>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>>
>>> Thanks again,
>>>
>>> Dan
>>>
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Monday, April 06, 2015 1:42 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> Hi Dan,
>>>
>>> at willhaben.at (customer of mine) two SOLR components were written
>>> for SOLR 3 and ported to SORL 4
>>>
>>> 1) SlowQueryLog which dumps long-running search requests into a log
>>> file
>>>
>>> 2) Most Frequent Search Terms allowing to query & filter the most
>>> frequent user search terms over the browser
>>>
>>> Some notes along the line
>>>
>>>
>>> * For both components I have the "GO" to open source them but I never
>>> had enough time to do that (shame on me) - see
>>> https://issues.apache.org/jira/browse/SOLR-4056
>>>
>>> * The Most Frequent Search Term component actually mimics a SOLR
>>> server you feed the user search terms so this might be a better
>>> solution in the long run. But this requires to have a separate SOLR
>>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>>> moving parts in production :-)
>>>
>>> * If there is sufficient interest I can make a code drop on GitHub
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>
>>>
>>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>>
>>>> Siegfried,
>>>>
>>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>>
>>>> Thank you so much,
>>>>
>>>> Dan
>>>>
>>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>>
>>>> -----Original Message-----
>>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>>> Sent: Saturday, April 04, 2015 4:54 AM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: Measuring QPS
>>>>
>>>> Hi Dan,
>>>>
>>>> I'm using JavaMelody for my SOLR production servers - gives you the
>>>> relevant HTTP stats (what's happening now & historical data) plus
>>>> JVM monitoring as additional benefit. The servers are deployed on
>>>> Tomcat so I'm of little help regarding Jetty - having said that
>>>>
>>>> * you need two Jars (javamelody & robin)
>>>> * tinker with web.xml
>>>>
>>>> Here are two of my presentations mentioning JavaMelody (plus some
>>>> other stuff)
>>>>
>>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>>> n
>>>> t
>>>> -to-production-20121210.pdf
>>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developm
>>>> e
>>>> n
>>>> t-to-production-20121210.pdf>
>>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>>> r
>>>> m
>>>> ance-monitoring.pdf
>>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perf
>>>> o
>>>> r
>>>> mance-monitoring.pdf>
>>>>
>>>> Cheers,
>>>>
>>>> Siegfried Goeschl
>>>>
>>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>>
>>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>>
>>>>>> What is the best way to gather QPS information?
>>>>>>
>>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>>
>>>>> As of Solr 4.1 (three years ago), request rate information is
>>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>>> and open the handler you wish to examine. You have
>>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>>
>>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>>
>>>>> Thanks,
>>>>> Shawn
>>>>>
>>>>
>>>
>>
>
RE: Measuring QPS
Posted by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov>.
This sounds really good:
"For load testing, we replay production logs to test that we meet the SLA at a given traffic level."
The rest sounds complicated. Ah well, that's the job.
-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: Monday, April 06, 2015 2:48 PM
To: solr-user@lucene.apache.org
Subject: Re: Measuring QPS
We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
> OK,
>
> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
> I love showing off, however, so please forgive me.
>
> -----Original Message-----
> From: Davis, Daniel (NIH/NLM) [C]
> Sent: Monday, April 06, 2015 2:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Measuring QPS
>
> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>
> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 2:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Daniel,
>
> interesting - I never thought of autocompletion but for keeping track
> of user behaviour :-)
>
> * the numbers are helpful for the online advertisement team to sell
> campaigns
> * it is used for sanity checks - sensible queries returning no results
> or returning too many results
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>
>> Thanks again,
>>
>> Dan
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Monday, April 06, 2015 1:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> at willhaben.at (customer of mine) two SOLR components were written
>> for SOLR 3 and ported to SORL 4
>>
>> 1) SlowQueryLog which dumps long-running search requests into a log
>> file
>>
>> 2) Most Frequent Search Terms allowing to query & filter the most
>> frequent user search terms over the browser
>>
>> Some notes along the line
>>
>>
>> * For both components I have the "GO" to open source them but I never
>> had enough time to do that (shame on me) - see
>> https://issues.apache.org/jira/browse/SOLR-4056
>>
>> * The Most Frequent Search Term component actually mimics a SOLR
>> server you feed the user search terms so this might be a better
>> solution in the long run. But this requires to have a separate SOLR
>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>> moving parts in production :-)
>>
>> * If there is sufficient interest I can make a code drop on GitHub
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>
>>
>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>> Siegfried,
>>>
>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>
>>> Thank you so much,
>>>
>>> Dan
>>>
>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Saturday, April 04, 2015 4:54 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> Hi Dan,
>>>
>>> I'm using JavaMelody for my SOLR production servers - gives you the
>>> relevant HTTP stats (what's happening now & historical data) plus
>>> JVM monitoring as additional benefit. The servers are deployed on
>>> Tomcat so I'm of little help regarding Jetty - having said that
>>>
>>> * you need two Jars (javamelody & robin)
>>> * tinker with web.xml
>>>
>>> Here are two of my presentations mentioning JavaMelody (plus some
>>> other stuff)
>>>
>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>> n
>>> t
>>> -to-production-20121210.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developm
>>> e
>>> n
>>> t-to-production-20121210.pdf>
>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>> r
>>> m
>>> ance-monitoring.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perf
>>> o
>>> r
>>> mance-monitoring.pdf>
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>
>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>
>>>>> What is the best way to gather QPS information?
>>>>>
>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>
>>>> As of Solr 4.1 (three years ago), request rate information is
>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>> and open the handler you wish to examine. You have
>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>
>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>
>>
>
Re: Measuring QPS
Posted by Walter Underwood <wu...@wunderwood.org>.
We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library.
That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is just the wrong metric for a one-sided distribution like response times.
We use Graphite to display the 95th percentile response time for each request handler. We use Tattle for alerting on those metrics.
We also use New Relic for a different look at the performance. It is good at tracking from the front end through to Solr.
For load testing, we replay production logs to test that we meet the SLA at a given traffic level.
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
On Apr 6, 2015, at 11:31 AM, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
> OK,
>
> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
> I love showing off, however, so please forgive me.
>
> -----Original Message-----
> From: Davis, Daniel (NIH/NLM) [C]
> Sent: Monday, April 06, 2015 2:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Measuring QPS
>
> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>
> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 2:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Daniel,
>
> interesting - I never thought of autocompletion but for keeping track of user behaviour :-)
>
> * the numbers are helpful for the online advertisement team to sell campaigns
> * it is used for sanity checks - sensible queries returning no results or returning too many results
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>
>> Thanks again,
>>
>> Dan
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Monday, April 06, 2015 1:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> at willhaben.at (customer of mine) two SOLR components were written
>> for SOLR 3 and ported to SORL 4
>>
>> 1) SlowQueryLog which dumps long-running search requests into a log
>> file
>>
>> 2) Most Frequent Search Terms allowing to query & filter the most
>> frequent user search terms over the browser
>>
>> Some notes along the line
>>
>>
>> * For both components I have the “GO" to open source them but I never
>> had enough time to do that (shame on me) - see
>> https://issues.apache.org/jira/browse/SOLR-4056
>>
>> * The Most Frequent Search Term component actually mimics a SOLR
>> server you feed the user search terms so this might be a better
>> solution in the long run. But this requires to have a separate SOLR
>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>> moving parts in production :-)
>>
>> * If there is sufficient interest I can make a code drop on GitHub
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>
>>
>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>> Siegfried,
>>>
>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>
>>> Thank you so much,
>>>
>>> Dan
>>>
>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Saturday, April 04, 2015 4:54 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> Hi Dan,
>>>
>>> I’m using JavaMelody for my SOLR production servers - gives you the
>>> relevant HTTP stats (what’s happening now & historical data) plus JVM
>>> monitoring as additional benefit. The servers are deployed on Tomcat
>>> so I’m of little help regarding Jetty - having said that
>>>
>>> * you need two Jars (javamelody & robin)
>>> * tinker with web.xml
>>>
>>> Here are two of my presentations mentioning JavaMelody (plus some
>>> other stuff)
>>>
>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>>> t
>>> -to-production-20121210.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>> n
>>> t-to-production-20121210.pdf>
>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>>> m
>>> ance-monitoring.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>> r
>>> mance-monitoring.pdf>
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>
>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>
>>>>> What is the best way to gather QPS information?
>>>>>
>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>
>>>> As of Solr 4.1 (three years ago), request rate information is
>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>> and open the handler you wish to examine. You have
>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>
>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>
>>
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
Appreciated :-)
Siegfried Goeschl
> On 06 Apr 2015, at 20:31, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> OK,
>
> I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
> I love showing off, however, so please forgive me.
>
> -----Original Message-----
> From: Davis, Daniel (NIH/NLM) [C]
> Sent: Monday, April 06, 2015 2:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Measuring QPS
>
> Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
>
> We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 2:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Daniel,
>
> interesting - I never thought of autocompletion but for keeping track of user behaviour :-)
>
> * the numbers are helpful for the online advertisement team to sell campaigns
> * it is used for sanity checks - sensible queries returning no results or returning too many results
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>>
>> Thanks again,
>>
>> Dan
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Monday, April 06, 2015 1:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> at willhaben.at (customer of mine) two SOLR components were written
>> for SOLR 3 and ported to SORL 4
>>
>> 1) SlowQueryLog which dumps long-running search requests into a log
>> file
>>
>> 2) Most Frequent Search Terms allowing to query & filter the most
>> frequent user search terms over the browser
>>
>> Some notes along the line
>>
>>
>> * For both components I have the “GO" to open source them but I never
>> had enough time to do that (shame on me) - see
>> https://issues.apache.org/jira/browse/SOLR-4056
>>
>> * The Most Frequent Search Term component actually mimics a SOLR
>> server you feed the user search terms so this might be a better
>> solution in the long run. But this requires to have a separate SOLR
>> core & ingest plus GUI (check out SILK or ELK) - in other words more
>> moving parts in production :-)
>>
>> * If there is sufficient interest I can make a code drop on GitHub
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>
>>
>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>>
>>> Siegfried,
>>>
>>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>>
>>> Thank you so much,
>>>
>>> Dan
>>>
>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>>
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Saturday, April 04, 2015 4:54 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>>
>>> Hi Dan,
>>>
>>> I’m using JavaMelody for my SOLR production servers - gives you the
>>> relevant HTTP stats (what’s happening now & historical data) plus JVM
>>> monitoring as additional benefit. The servers are deployed on Tomcat
>>> so I’m of little help regarding Jetty - having said that
>>>
>>> * you need two Jars (javamelody & robin)
>>> * tinker with web.xml
>>>
>>> Here are two of my presentations mentioning JavaMelody (plus some
>>> other stuff)
>>>
>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>>> t
>>> -to-production-20121210.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>> n
>>> t-to-production-20121210.pdf>
>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>>> m
>>> ance-monitoring.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>> r
>>> mance-monitoring.pdf>
>>>
>>> Cheers,
>>>
>>> Siegfried Goeschl
>>>
>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>>
>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>>
>>>>> What is the best way to gather QPS information?
>>>>>
>>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>>
>>>> As of Solr 4.1 (three years ago), request rate information is
>>>> available in the admin UI and via JMX. In the admin UI, choose a
>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>>> and open the handler you wish to examine. You have
>>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>>
>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>
>>
>
RE: Measuring QPS
Posted by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov>.
OK,
I have a lot of chutzpah posting that here ;) The other guys answering the questions can probably explain it better.
I love showing off, however, so please forgive me.
-----Original Message-----
From: Davis, Daniel (NIH/NLM) [C]
Sent: Monday, April 06, 2015 2:25 PM
To: solr-user@lucene.apache.org
Subject: RE: Measuring QPS
Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
-----Original Message-----
From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
Sent: Monday, April 06, 2015 2:17 PM
To: solr-user@lucene.apache.org
Subject: Re: Measuring QPS
Hi Daniel,
interesting - I never thought of autocompletion but for keeping track of user behaviour :-)
* the numbers are helpful for the online advertisement team to sell campaigns
* it is used for sanity checks - sensible queries returning no results or returning too many results
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> Siegfried,
>
> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>
> Thanks again,
>
> Dan
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Dan,
>
> at willhaben.at (customer of mine) two SOLR components were written
> for SOLR 3 and ported to SORL 4
>
> 1) SlowQueryLog which dumps long-running search requests into a log
> file
>
> 2) Most Frequent Search Terms allowing to query & filter the most
> frequent user search terms over the browser
>
> Some notes along the line
>
>
> * For both components I have the “GO" to open source them but I never
> had enough time to do that (shame on me) - see
> https://issues.apache.org/jira/browse/SOLR-4056
>
> * The Most Frequent Search Term component actually mimics a SOLR
> server you feed the user search terms so this might be a better
> solution in the long run. But this requires to have a separate SOLR
> core & ingest plus GUI (check out SILK or ELK) - in other words more
> moving parts in production :-)
>
> * If there is sufficient interest I can make a code drop on GitHub
>
> Cheers,
>
> Siegfried Goeschl
>
>
>
>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>
>> Thank you so much,
>>
>> Dan
>>
>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Saturday, April 04, 2015 4:54 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> I’m using JavaMelody for my SOLR production servers - gives you the
>> relevant HTTP stats (what’s happening now & historical data) plus JVM
>> monitoring as additional benefit. The servers are deployed on Tomcat
>> so I’m of little help regarding Jetty - having said that
>>
>> * you need two Jars (javamelody & robin)
>> * tinker with web.xml
>>
>> Here are two of my presentations mentioning JavaMelody (plus some
>> other stuff)
>>
>> http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>> t
>> -to-production-20121210.pdf
>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>> n
>> t-to-production-20121210.pdf>
>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>> m
>> ance-monitoring.pdf
>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>> r
>> mance-monitoring.pdf>
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>
>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>
>>>> What is the best way to gather QPS information?
>>>>
>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>
>>> As of Solr 4.1 (three years ago), request rate information is
>>> available in the admin UI and via JMX. In the admin UI, choose a
>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>> and open the handler you wish to examine. You have
>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>
>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>
>>> Thanks,
>>> Shawn
>>>
>>
>
RE: Measuring QPS
Posted by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov>.
Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these days.
We use relevance/field boosts/phrase matching etc. to get the best guess about what results they want to see. This is similar - we use relevance, field boosting to guess what users want to search for. Zipf's law applies to searches as well as results.
-----Original Message-----
From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
Sent: Monday, April 06, 2015 2:17 PM
To: solr-user@lucene.apache.org
Subject: Re: Measuring QPS
Hi Daniel,
interesting - I never thought of autocompletion but for keeping track of user behaviour :-)
* the numbers are helpful for the online advertisement team to sell campaigns
* it is used for sanity checks - sensible queries returning no results or returning too many results
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> Siegfried,
>
> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>
> Thanks again,
>
> Dan
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Dan,
>
> at willhaben.at (customer of mine) two SOLR components were written
> for SOLR 3 and ported to SORL 4
>
> 1) SlowQueryLog which dumps long-running search requests into a log
> file
>
> 2) Most Frequent Search Terms allowing to query & filter the most
> frequent user search terms over the browser
>
> Some notes along the line
>
>
> * For both components I have the “GO" to open source them but I never
> had enough time to do that (shame on me) - see
> https://issues.apache.org/jira/browse/SOLR-4056
>
> * The Most Frequent Search Term component actually mimics a SOLR
> server you feed the user search terms so this might be a better
> solution in the long run. But this requires to have a separate SOLR
> core & ingest plus GUI (check out SILK or ELK) - in other words more
> moving parts in production :-)
>
> * If there is sufficient interest I can make a code drop on GitHub
>
> Cheers,
>
> Siegfried Goeschl
>
>
>
>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>
>> Thank you so much,
>>
>> Dan
>>
>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Saturday, April 04, 2015 4:54 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> I’m using JavaMelody for my SOLR production servers - gives you the
>> relevant HTTP stats (what’s happening now & historical data) plus JVM
>> monitoring as additional benefit. The servers are deployed on Tomcat
>> so I’m of little help regarding Jetty - having said that
>>
>> * you need two Jars (javamelody & robin)
>> * tinker with web.xml
>>
>> Here are two of my presentations mentioning JavaMelody (plus some
>> other stuff)
>>
>> http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>> t
>> -to-production-20121210.pdf
>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>> n
>> t-to-production-20121210.pdf>
>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>> m
>> ance-monitoring.pdf
>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>> r
>> mance-monitoring.pdf>
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>
>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>
>>>> What is the best way to gather QPS information?
>>>>
>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>
>>> As of Solr 4.1 (three years ago), request rate information is
>>> available in the admin UI and via JMX. In the admin UI, choose a
>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>> and open the handler you wish to examine. You have
>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>
>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>
>>> Thanks,
>>> Shawn
>>>
>>
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
Hi Daniel,
interesting - I never thought of autocompletion but for keeping track of user behaviour :-)
* the numbers are helpful for the online advertisement team to sell campaigns
* it is used for sanity checks - sensible queries returning no results or returning too many results
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> Siegfried,
>
> It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
>
> Thanks again,
>
> Dan
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 1:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Dan,
>
> at willhaben.at (customer of mine) two SOLR components were written for SOLR 3 and ported to SORL 4
>
> 1) SlowQueryLog which dumps long-running search requests into a log file
>
> 2) Most Frequent Search Terms allowing to query & filter the most frequent user search terms over the browser
>
> Some notes along the line
>
>
> * For both components I have the “GO" to open source them but I never had enough time to do that (shame on me) - see https://issues.apache.org/jira/browse/SOLR-4056
>
> * The Most Frequent Search Term component actually mimics a SOLR server you feed the user search terms so this might be a better solution in the long run. But this requires to have a separate SOLR core & ingest plus GUI (check out SILK or ELK) - in other words more moving parts in production :-)
>
> * If there is sufficient interest I can make a code drop on GitHub
>
> Cheers,
>
> Siegfried Goeschl
>
>
>
>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>>
>> Siegfried,
>>
>> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>>
>> Thank you so much,
>>
>> Dan
>>
>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>>
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Saturday, April 04, 2015 4:54 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>>
>> Hi Dan,
>>
>> I’m using JavaMelody for my SOLR production servers - gives you the
>> relevant HTTP stats (what’s happening now & historical data) plus JVM
>> monitoring as additional benefit. The servers are deployed on Tomcat
>> so I’m of little help regarding Jetty - having said that
>>
>> * you need two Jars (javamelody & robin)
>> * tinker with web.xml
>>
>> Here are two of my presentations mentioning JavaMelody (plus some
>> other stuff)
>>
>> http://people.apache.org/~sgoeschl/presentations/solr-from-development
>> -to-production-20121210.pdf
>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>> t-to-production-20121210.pdf>
>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perform
>> ance-monitoring.pdf
>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>> mance-monitoring.pdf>
>>
>> Cheers,
>>
>> Siegfried Goeschl
>>
>>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>>
>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>>
>>>> What is the best way to gather QPS information?
>>>>
>>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>>
>>> As of Solr 4.1 (three years ago), request rate information is
>>> available in the admin UI and via JMX. In the admin UI, choose a
>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>>> and open the handler you wish to examine. You have
>>> avgRequestsPerSecond, which is calculated for the entire runtime of
>>> the SolrCore, as well as 5minRateReqsPerSecond and
>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>
>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>
>>> Thanks,
>>> Shawn
>>>
>>
>
RE: Measuring QPS
Posted by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov>.
Siegfried,
It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr.
Thanks again,
Dan
-----Original Message-----
From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
Sent: Monday, April 06, 2015 1:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Measuring QPS
Hi Dan,
at willhaben.at (customer of mine) two SOLR components were written for SOLR 3 and ported to SORL 4
1) SlowQueryLog which dumps long-running search requests into a log file
2) Most Frequent Search Terms allowing to query & filter the most frequent user search terms over the browser
Some notes along the line
* For both components I have the “GO" to open source them but I never had enough time to do that (shame on me) - see https://issues.apache.org/jira/browse/SOLR-4056
* The Most Frequent Search Term component actually mimics a SOLR server you feed the user search terms so this might be a better solution in the long run. But this requires to have a separate SOLR core & ingest plus GUI (check out SILK or ELK) - in other words more moving parts in production :-)
* If there is sufficient interest I can make a code drop on GitHub
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> Siegfried,
>
> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>
> Thank you so much,
>
> Dan
>
> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Saturday, April 04, 2015 4:54 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Dan,
>
> I’m using JavaMelody for my SOLR production servers - gives you the
> relevant HTTP stats (what’s happening now & historical data) plus JVM
> monitoring as additional benefit. The servers are deployed on Tomcat
> so I’m of little help regarding Jetty - having said that
>
> * you need two Jars (javamelody & robin)
> * tinker with web.xml
>
> Here are two of my presentations mentioning JavaMelody (plus some
> other stuff)
>
> http://people.apache.org/~sgoeschl/presentations/solr-from-development
> -to-production-20121210.pdf
> <http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
> t-to-production-20121210.pdf>
> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perform
> ance-monitoring.pdf
> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
> mance-monitoring.pdf>
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>
>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>
>>> What is the best way to gather QPS information?
>>>
>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>
>> As of Solr 4.1 (three years ago), request rate information is
>> available in the admin UI and via JMX. In the admin UI, choose a
>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER,
>> and open the handler you wish to examine. You have
>> avgRequestsPerSecond, which is calculated for the entire runtime of
>> the SolrCore, as well as 5minRateReqsPerSecond and
>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>
>> https://issues.apache.org/jira/browse/SOLR-1972
>>
>> Thanks,
>> Shawn
>>
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
Hi Dan,
at willhaben.at (customer of mine) two SOLR components were written for SOLR 3 and ported to SORL 4
1) SlowQueryLog which dumps long-running search requests into a log file
2) Most Frequent Search Terms allowing to query & filter the most frequent user search terms over the browser
Some notes along the line
* For both components I have the “GO" to open source them but I never had enough time to do that (shame on me) - see https://issues.apache.org/jira/browse/SOLR-4056
* The Most Frequent Search Term component actually mimics a SOLR server you feed the user search terms so this might be a better solution in the long run. But this requires to have a separate SOLR core & ingest plus GUI (check out SILK or ELK) - in other words more moving parts in production :-)
* If there is sufficient interest I can make a code drop on GitHub
Cheers,
Siegfried Goeschl
> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <da...@nih.gov> wrote:
>
> Siegfried,
>
> This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
>
> Thank you so much,
>
> Dan
>
> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
>
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Saturday, April 04, 2015 4:54 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
>
> Hi Dan,
>
> I’m using JavaMelody for my SOLR production servers - gives you the relevant HTTP stats (what’s happening now & historical data) plus JVM monitoring as additional benefit. The servers are deployed on Tomcat so I’m of little help regarding Jetty - having said that
>
> * you need two Jars (javamelody & robin)
> * tinker with web.xml
>
> Here are two of my presentations mentioning JavaMelody (plus some other stuff)
>
> http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf <http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf>
> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf>
>
> Cheers,
>
> Siegfried Goeschl
>
>> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>>
>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>>
>>> What is the best way to gather QPS information?
>>>
>>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>>
>> As of Solr 4.1 (three years ago), request rate information is
>> available in the admin UI and via JMX. In the admin UI, choose a core
>> from the dropdown, click on Plugins/Stats, then QUERYHANDLER, and open
>> the handler you wish to examine. You have avgRequestsPerSecond, which
>> is calculated for the entire runtime of the SolrCore, as well as
>> 5minRateReqsPerSecond and 15minRateReqsPerSecond, which are far more
>> useful pieces of information.
>>
>> https://issues.apache.org/jira/browse/SOLR-1972
>>
>> Thanks,
>> Shawn
>>
>
RE: Measuring QPS
Posted by "Davis, Daniel (NIH/NLM) [C]" <da...@nih.gov>.
Siegfried,
This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in Solr?
Thank you so much,
Dan
P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger" in Berlin after a conference - Hayden Christensen was less annoying in German, because my wife and I don't speak German ;) I haven't thought of that in a while.
-----Original Message-----
From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
Sent: Saturday, April 04, 2015 4:54 AM
To: solr-user@lucene.apache.org
Subject: Re: Measuring QPS
Hi Dan,
I’m using JavaMelody for my SOLR production servers - gives you the relevant HTTP stats (what’s happening now & historical data) plus JVM monitoring as additional benefit. The servers are deployed on Tomcat so I’m of little help regarding Jetty - having said that
* you need two Jars (javamelody & robin)
* tinker with web.xml
Here are two of my presentations mentioning JavaMelody (plus some other stuff)
http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf <http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf>
http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf>
Cheers,
Siegfried Goeschl
> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>
> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>
>> What is the best way to gather QPS information?
>>
>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>
> As of Solr 4.1 (three years ago), request rate information is
> available in the admin UI and via JMX. In the admin UI, choose a core
> from the dropdown, click on Plugins/Stats, then QUERYHANDLER, and open
> the handler you wish to examine. You have avgRequestsPerSecond, which
> is calculated for the entire runtime of the SolrCore, as well as
> 5minRateReqsPerSecond and 15minRateReqsPerSecond, which are far more
> useful pieces of information.
>
> https://issues.apache.org/jira/browse/SOLR-1972
>
> Thanks,
> Shawn
>
Re: Measuring QPS
Posted by Siegfried Goeschl <sg...@gmx.at>.
Hi Dan,
I’m using JavaMelody for my SOLR production servers - gives you the relevant HTTP stats (what’s happening now & historical data) plus JVM monitoring as additional benefit. The servers are deployed on Tomcat so I’m of little help regarding Jetty - having said that
* you need two Jars (javamelody & robin)
* tinker with web.xml
Here are two of my presentations mentioning JavaMelody (plus some other stuff)
http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf <http://people.apache.org/~sgoeschl/presentations/solr-from-development-to-production-20121210.pdf>
http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-performance-monitoring.pdf>
Cheers,
Siegfried Goeschl
> On 03 Apr 2015, at 17:53, Shawn Heisey <ap...@elyograg.org> wrote:
>
> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>>
>> What is the best way to gather QPS information?
>>
>> What is the best way to add information like this to the Admin UI, if I decide to take that step?
>
> As of Solr 4.1 (three years ago), request rate information is available
> in the admin UI and via JMX. In the admin UI, choose a core from the
> dropdown, click on Plugins/Stats, then QUERYHANDLER, and open the
> handler you wish to examine. You have avgRequestsPerSecond, which is
> calculated for the entire runtime of the SolrCore, as well as
> 5minRateReqsPerSecond and 15minRateReqsPerSecond, which are far more
> useful pieces of information.
>
> https://issues.apache.org/jira/browse/SOLR-1972
>
> Thanks,
> Shawn
>
Re: Measuring QPS
Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
> I wanted to gather QPS for our production Solr instances, but I was surprised that the Admin UI did not contain this information. We are running a mix of versions, but mostly 4.10 at this point. We are not using SolrCloud at present; that's part of why I'm checking - I want to validate the size of our existing setup and what sort of SolrCloud setup would be needed to centralize several of them.
>
> What is the best way to gather QPS information?
>
> What is the best way to add information like this to the Admin UI, if I decide to take that step?
As of Solr 4.1 (three years ago), request rate information is available
in the admin UI and via JMX. In the admin UI, choose a core from the
dropdown, click on Plugins/Stats, then QUERYHANDLER, and open the
handler you wish to examine. You have avgRequestsPerSecond, which is
calculated for the entire runtime of the SolrCore, as well as
5minRateReqsPerSecond and 15minRateReqsPerSecond, which are far more
useful pieces of information.
https://issues.apache.org/jira/browse/SOLR-1972
Thanks,
Shawn