You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by ka...@gmx.de on 2011/11/04 16:51:44 UTC
Re: [Profiling] How to profile/tune Solr server
Hi Spark,
2009 there was a monitor from lucidimagination:
http://www.lucidimagination.com/about/news/releases/lucid-imagination-releases-performance-monitoring-utility-open-source-apache-lucene
A colleague of mine calls the sematext-monitor "trojan" because "SPM phone home":
"Easy in, easy out - if you try SPM and don't like it, simply stop and remove the small client-side piece that sends us your data"
http://sematext.com/spm/solr-performance-monitoring/index.html
Looks like other people using a "real profiler" like YourKit Java Profiler
http://forums.yourkit.com/viewtopic.php?f=3&t=3850
There is also an article about Zabbix
http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/
In your case any profiler would do, but if you find out a Profiler with solr-specific default-filter let me know.
Best regrads
Karsten
P.S. eMail in context
http://lucene.472066.n3.nabble.com/Profiling-How-to-profile-tune-Solr-server-td3467027.html
-------- Original-Nachricht --------
> Datum: Mon, 31 Oct 2011 18:35:32 +0800
> Von: yu shen <sh...@gmail.com>
> An: solr-user@lucene.apache.org
> Betreff: Re: [Profiling] How to profile/tune Solr server
> No idea so far, try to figure out.
>
> Spark
>
> 2011/10/31 Jan Høydahl <ja...@cominvent.com>
>
> > Hi,
> >
> > There are no official tools other than looking at the built-in stats
> pages
> > and perhaps using JConsole or similar JVM monitoring tools. Note that
> > Solr's JMX capabilities may let you hook your enterprise's existing
> > monitoring dashboard up with Solr.
> >
> > Also check out the new monitoring service from Sematext which will give
> > you graphs and all. So far it's free evaluation:
> > http://sematext.com/spm/index.html
> >
> > Do you have a clue for why the indexing is slow?
> >
> > --
> > Jan Høydahl, search solution architect
> > Cominvent AS - www.cominvent.com
> > Solr Training - www.solrtraining.com
> >
> > On 31. okt. 2011, at 04:59, yu shen wrote:
> >
> > > Hi All,
> > >
> > > I am a solr newbie. I find solr documents easy to access and use,
> which
> > is
> > > really good thing. While my problem is I did not find a solr home
> grown
> > > profiling/monitoring tool.
> > >
> > > I set up the server as a multi-core server, each core has
> approximately
> > 2GB
> > > index. And I need to update solr and re-generate index in a real time
> > > manner (In java code, using SolrJ). Sometimes the update operation is
> > slow.
> > > And it is expected that in a year, the index size may increase to 4GB.
> > And
> > > I need to do something to prevent performance downgrade.
> > >
> > > Is there any solr official monitoring & profiling tool for this?
> > >
> > > Spark
> >
> >
Re: [Profiling] How to profile/tune Solr server
Posted by yu shen <sh...@gmail.com>.
Thank you for the information.
2011/11/5 yu shen <sh...@gmail.com>
> Really helpful, thanks so much.
>
> Spark
>
> 2011/11/4 <ka...@gmx.de>
>
> Hi Spark,
>>
>> 2009 there was a monitor from lucidimagination:
>>
>> http://www.lucidimagination.com/about/news/releases/lucid-imagination-releases-performance-monitoring-utility-open-source-apache-lucene
>>
>> A colleague of mine calls the sematext-monitor "trojan" because "SPM
>> phone home":
>> "Easy in, easy out - if you try SPM and don't like it, simply stop and
>> remove the small client-side piece that sends us your data"
>> http://sematext.com/spm/solr-performance-monitoring/index.html
>>
>> Looks like other people using a "real profiler" like YourKit Java Profiler
>> http://forums.yourkit.com/viewtopic.php?f=3&t=3850
>>
>> There is also an article about Zabbix
>>
>> http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/
>>
>> In your case any profiler would do, but if you find out a Profiler with
>> solr-specific default-filter let me know.
>>
>>
>>
>> Best regrads
>> Karsten
>>
>> P.S. eMail in context
>>
>> http://lucene.472066.n3.nabble.com/Profiling-How-to-profile-tune-Solr-server-td3467027.html
>>
>> -------- Original-Nachricht --------
>> > Datum: Mon, 31 Oct 2011 18:35:32 +0800
>> > Von: yu shen <sh...@gmail.com>
>> > An: solr-user@lucene.apache.org
>> > Betreff: Re: [Profiling] How to profile/tune Solr server
>>
>> > No idea so far, try to figure out.
>> >
>> > Spark
>> >
>> > 2011/10/31 Jan Høydahl <ja...@cominvent.com>
>> >
>> > > Hi,
>> > >
>> > > There are no official tools other than looking at the built-in stats
>> > pages
>> > > and perhaps using JConsole or similar JVM monitoring tools. Note that
>> > > Solr's JMX capabilities may let you hook your enterprise's existing
>> > > monitoring dashboard up with Solr.
>> > >
>> > > Also check out the new monitoring service from Sematext which will
>> give
>> > > you graphs and all. So far it's free evaluation:
>> > > http://sematext.com/spm/index.html
>> > >
>> > > Do you have a clue for why the indexing is slow?
>> > >
>> > > --
>> > > Jan Høydahl, search solution architect
>> > > Cominvent AS - www.cominvent.com
>> > > Solr Training - www.solrtraining.com
>> > >
>> > > On 31. okt. 2011, at 04:59, yu shen wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > I am a solr newbie. I find solr documents easy to access and use,
>> > which
>> > > is
>> > > > really good thing. While my problem is I did not find a solr home
>> > grown
>> > > > profiling/monitoring tool.
>> > > >
>> > > > I set up the server as a multi-core server, each core has
>> > approximately
>> > > 2GB
>> > > > index. And I need to update solr and re-generate index in a real
>> time
>> > > > manner (In java code, using SolrJ). Sometimes the update operation
>> is
>> > > slow.
>> > > > And it is expected that in a year, the index size may increase to
>> 4GB.
>> > > And
>> > > > I need to do something to prevent performance downgrade.
>> > > >
>> > > > Is there any solr official monitoring & profiling tool for this?
>> > > >
>> > > > Spark
>> > >
>> > >
>>
>
>
Re: [Profiling] How to profile/tune Solr server
Posted by yu shen <sh...@gmail.com>.
Really helpful, thanks so much.
Spark
2011/11/4 <ka...@gmx.de>
> Hi Spark,
>
> 2009 there was a monitor from lucidimagination:
>
> http://www.lucidimagination.com/about/news/releases/lucid-imagination-releases-performance-monitoring-utility-open-source-apache-lucene
>
> A colleague of mine calls the sematext-monitor "trojan" because "SPM phone
> home":
> "Easy in, easy out - if you try SPM and don't like it, simply stop and
> remove the small client-side piece that sends us your data"
> http://sematext.com/spm/solr-performance-monitoring/index.html
>
> Looks like other people using a "real profiler" like YourKit Java Profiler
> http://forums.yourkit.com/viewtopic.php?f=3&t=3850
>
> There is also an article about Zabbix
>
> http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/
>
> In your case any profiler would do, but if you find out a Profiler with
> solr-specific default-filter let me know.
>
>
>
> Best regrads
> Karsten
>
> P.S. eMail in context
>
> http://lucene.472066.n3.nabble.com/Profiling-How-to-profile-tune-Solr-server-td3467027.html
>
> -------- Original-Nachricht --------
> > Datum: Mon, 31 Oct 2011 18:35:32 +0800
> > Von: yu shen <sh...@gmail.com>
> > An: solr-user@lucene.apache.org
> > Betreff: Re: [Profiling] How to profile/tune Solr server
>
> > No idea so far, try to figure out.
> >
> > Spark
> >
> > 2011/10/31 Jan Høydahl <ja...@cominvent.com>
> >
> > > Hi,
> > >
> > > There are no official tools other than looking at the built-in stats
> > pages
> > > and perhaps using JConsole or similar JVM monitoring tools. Note that
> > > Solr's JMX capabilities may let you hook your enterprise's existing
> > > monitoring dashboard up with Solr.
> > >
> > > Also check out the new monitoring service from Sematext which will give
> > > you graphs and all. So far it's free evaluation:
> > > http://sematext.com/spm/index.html
> > >
> > > Do you have a clue for why the indexing is slow?
> > >
> > > --
> > > Jan Høydahl, search solution architect
> > > Cominvent AS - www.cominvent.com
> > > Solr Training - www.solrtraining.com
> > >
> > > On 31. okt. 2011, at 04:59, yu shen wrote:
> > >
> > > > Hi All,
> > > >
> > > > I am a solr newbie. I find solr documents easy to access and use,
> > which
> > > is
> > > > really good thing. While my problem is I did not find a solr home
> > grown
> > > > profiling/monitoring tool.
> > > >
> > > > I set up the server as a multi-core server, each core has
> > approximately
> > > 2GB
> > > > index. And I need to update solr and re-generate index in a real time
> > > > manner (In java code, using SolrJ). Sometimes the update operation is
> > > slow.
> > > > And it is expected that in a year, the index size may increase to
> 4GB.
> > > And
> > > > I need to do something to prevent performance downgrade.
> > > >
> > > > Is there any solr official monitoring & profiling tool for this?
> > > >
> > > > Spark
> > >
> > >
>
Re: [Profiling] How to profile/tune Solr server
Posted by Lance Norskog <go...@gmail.com>.
NewRelic also offers an online Solr monitoring tool. You can sign up via
the Lucid Imagination site.
http://www.lucidimagination.com/search/?q=new+relic#/s:lucid
On Sun, Nov 6, 2011 at 3:31 AM, Sujatha Arun <su...@gmail.com> wrote:
> hi ,
>
> I am planning to try Sematext Monitoring. Is there anything to watch out
> for ?
>
> Regards
> Sujatha
>
>
>
> On Fri, Nov 4, 2011 at 9:21 PM, <ka...@gmx.de> wrote:
>
> > Hi Spark,
> >
> > 2009 there was a monitor from lucidimagination:
> >
> >
> http://www.lucidimagination.com/about/news/releases/lucid-imagination-releases-performance-monitoring-utility-open-source-apache-lucene
> >
> > A colleague of mine calls the sematext-monitor "trojan" because "SPM
> phone
> > home":
> > "Easy in, easy out - if you try SPM and don't like it, simply stop and
> > remove the small client-side piece that sends us your data"
> > http://sematext.com/spm/solr-performance-monitoring/index.html
> >
> > Looks like other people using a "real profiler" like YourKit Java
> Profiler
> > http://forums.yourkit.com/viewtopic.php?f=3&t=3850
> >
> > There is also an article about Zabbix
> >
> >
> http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/
> >
> > In your case any profiler would do, but if you find out a Profiler with
> > solr-specific default-filter let me know.
> >
> >
> >
> > Best regrads
> > Karsten
> >
> > P.S. eMail in context
> >
> >
> http://lucene.472066.n3.nabble.com/Profiling-How-to-profile-tune-Solr-server-td3467027.html
> >
> > -------- Original-Nachricht --------
> > > Datum: Mon, 31 Oct 2011 18:35:32 +0800
> > > Von: yu shen <sh...@gmail.com>
> > > An: solr-user@lucene.apache.org
> > > Betreff: Re: [Profiling] How to profile/tune Solr server
> >
> > > No idea so far, try to figure out.
> > >
> > > Spark
> > >
> > > 2011/10/31 Jan Høydahl <ja...@cominvent.com>
> > >
> > > > Hi,
> > > >
> > > > There are no official tools other than looking at the built-in stats
> > > pages
> > > > and perhaps using JConsole or similar JVM monitoring tools. Note that
> > > > Solr's JMX capabilities may let you hook your enterprise's existing
> > > > monitoring dashboard up with Solr.
> > > >
> > > > Also check out the new monitoring service from Sematext which will
> give
> > > > you graphs and all. So far it's free evaluation:
> > > > http://sematext.com/spm/index.html
> > > >
> > > > Do you have a clue for why the indexing is slow?
> > > >
> > > > --
> > > > Jan Høydahl, search solution architect
> > > > Cominvent AS - www.cominvent.com
> > > > Solr Training - www.solrtraining.com
> > > >
> > > > On 31. okt. 2011, at 04:59, yu shen wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I am a solr newbie. I find solr documents easy to access and use,
> > > which
> > > > is
> > > > > really good thing. While my problem is I did not find a solr home
> > > grown
> > > > > profiling/monitoring tool.
> > > > >
> > > > > I set up the server as a multi-core server, each core has
> > > approximately
> > > > 2GB
> > > > > index. And I need to update solr and re-generate index in a real
> time
> > > > > manner (In java code, using SolrJ). Sometimes the update operation
> is
> > > > slow.
> > > > > And it is expected that in a year, the index size may increase to
> > 4GB.
> > > > And
> > > > > I need to do something to prevent performance downgrade.
> > > > >
> > > > > Is there any solr official monitoring & profiling tool for this?
> > > > >
> > > > > Spark
> > > >
> > > >
> >
>
--
Lance Norskog
goksron@gmail.com
Re: [Profiling] How to profile/tune Solr server
Posted by Sujatha Arun <su...@gmail.com>.
hi ,
I am planning to try Sematext Monitoring. Is there anything to watch out
for ?
Regards
Sujatha
On Fri, Nov 4, 2011 at 9:21 PM, <ka...@gmx.de> wrote:
> Hi Spark,
>
> 2009 there was a monitor from lucidimagination:
>
> http://www.lucidimagination.com/about/news/releases/lucid-imagination-releases-performance-monitoring-utility-open-source-apache-lucene
>
> A colleague of mine calls the sematext-monitor "trojan" because "SPM phone
> home":
> "Easy in, easy out - if you try SPM and don't like it, simply stop and
> remove the small client-side piece that sends us your data"
> http://sematext.com/spm/solr-performance-monitoring/index.html
>
> Looks like other people using a "real profiler" like YourKit Java Profiler
> http://forums.yourkit.com/viewtopic.php?f=3&t=3850
>
> There is also an article about Zabbix
>
> http://www.lucidimagination.com/blog/2011/10/02/monitoring-apache-solr-and-lucidworks-with-zabbix/
>
> In your case any profiler would do, but if you find out a Profiler with
> solr-specific default-filter let me know.
>
>
>
> Best regrads
> Karsten
>
> P.S. eMail in context
>
> http://lucene.472066.n3.nabble.com/Profiling-How-to-profile-tune-Solr-server-td3467027.html
>
> -------- Original-Nachricht --------
> > Datum: Mon, 31 Oct 2011 18:35:32 +0800
> > Von: yu shen <sh...@gmail.com>
> > An: solr-user@lucene.apache.org
> > Betreff: Re: [Profiling] How to profile/tune Solr server
>
> > No idea so far, try to figure out.
> >
> > Spark
> >
> > 2011/10/31 Jan Høydahl <ja...@cominvent.com>
> >
> > > Hi,
> > >
> > > There are no official tools other than looking at the built-in stats
> > pages
> > > and perhaps using JConsole or similar JVM monitoring tools. Note that
> > > Solr's JMX capabilities may let you hook your enterprise's existing
> > > monitoring dashboard up with Solr.
> > >
> > > Also check out the new monitoring service from Sematext which will give
> > > you graphs and all. So far it's free evaluation:
> > > http://sematext.com/spm/index.html
> > >
> > > Do you have a clue for why the indexing is slow?
> > >
> > > --
> > > Jan Høydahl, search solution architect
> > > Cominvent AS - www.cominvent.com
> > > Solr Training - www.solrtraining.com
> > >
> > > On 31. okt. 2011, at 04:59, yu shen wrote:
> > >
> > > > Hi All,
> > > >
> > > > I am a solr newbie. I find solr documents easy to access and use,
> > which
> > > is
> > > > really good thing. While my problem is I did not find a solr home
> > grown
> > > > profiling/monitoring tool.
> > > >
> > > > I set up the server as a multi-core server, each core has
> > approximately
> > > 2GB
> > > > index. And I need to update solr and re-generate index in a real time
> > > > manner (In java code, using SolrJ). Sometimes the update operation is
> > > slow.
> > > > And it is expected that in a year, the index size may increase to
> 4GB.
> > > And
> > > > I need to do something to prevent performance downgrade.
> > > >
> > > > Is there any solr official monitoring & profiling tool for this?
> > > >
> > > > Spark
> > >
> > >
>
Re: [Profiling] How to profile/tune Solr server
Posted by Andre Bois-Crettez <an...@kelkoo.com>.
SolrMeter is useful too, it can be plugged to a production server just
to watch evolution of caches usage :
http://code.google.com/p/solrmeter/wiki/Screenshots#CacheHistoryStatistic
André