Profiling Solr

[Jean-Sebastien Vachon <js...@videotron.ca>]

Hi,

I'm trying to identify the bottleneck to get acceptable performance of a single shard containing 4.7 millions of documents using my own machine (Mac Pro - Quad Core with 8Gb of RAM with 4Gb allocated to the JVM). 

I tried using YourKit but I don't get anything about Solr classes. I'm new to Yourkit so I might be doing something wrong but it seems pretty straight forward.

I am running Solr within a Tomcat instance within Eclipse. Does anyone have an idea about what could be wrong in my setup?

I'm making individual requests (one at a time) and the response times are horrible (about 15 sec on average). I need to bring this way below 1 second.

Here is a sample query:

http://localhost:8983/jobs_part3/select/?q=*:*&collapse=true&collapse.field=hash_id&facet=true&facet.field=county_id&facet.field=advertiser_id&facet.field=county_id&sort=county_id+asc&rows=100&collapse.type=adjacent

I know that collapsing results has a big hit on performance but it is a must have for us.

Thanks for any hints.

===== JVM Parameters =====

-Xms4g -Xmx4g -d64 -server

Re: Profiling Solr

[Gora Mohanty <go...@mimirtech.com>]

On Fri, Dec 23, 2011 at 11:36 AM, Shyam Bhaskaran
<Sh...@synopsys.com> wrote:
> Hi,
>
> Can someone suggest me on performing Solr Profiling.

Have you looked at JMX: http://wiki.apache.org/solr/SolrJmx ?

Regards,
Gora

RE: Profiling Solr

[Shyam Bhaskaran <Sh...@synopsys.com>]

Hi,

Can someone suggest me on performing Solr Profiling.

We are seeing performance issues and using the debug flag it seems the highlighting component is causing the overhead.
I wanted to find out what is causing the overhead in highlighting for certain queries.

I assume IO or CPU is causing this highlighting overhead and wanted to find more details using Solr Profiling.

Let me know for any suggestions.

-Shyam 


-----Original Message-----
From: shyam bhaskaran [mailto:shyam.bhaskaran@gmail.com] 
Sent: Friday, December 23, 2011 6:26 AM
To: solr-user@lucene.apache.org
Subject: Re: Profiling Solr

Hi Jean,

I am also looking into Profiling Solr and wanted to check with you whether
you were able to use YourKit successfully for Solr Profiling and were you
able to find out the bottleneck with your situation.

Can you share how you were able to find out the performance bottleneck and
fix the issue, as I am also facing a similar issue with Solr performance.

-Shyam  

--
View this message in context: http://lucene.472066.n3.nabble.com/Profiling-Solr-tp473500p3607890.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Profiling Solr

[shyam bhaskaran <sh...@gmail.com>]

Hi Jean,

I am also looking into Profiling Solr and wanted to check with you whether
you were able to use YourKit successfully for Solr Profiling and were you
able to find out the bottleneck with your situation.

Can you share how you were able to find out the performance bottleneck and
fix the issue, as I am also facing a similar issue with Solr performance.

-Shyam  

--
View this message in context: http://lucene.472066.n3.nabble.com/Profiling-Solr-tp473500p3607890.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Profiling Solr

[Yonik Seeley <ys...@gmail.com>]

On Thu, Mar 11, 2010 at 1:11 PM, Jean-Sebastien Vachon
<js...@videotron.ca> wrote:
> Hi,
>
> I'm trying to identify the bottleneck to get acceptable performance of a single shard containing 4.7 millions of documents using my own machine (Mac Pro - Quad Core with 8Gb of RAM with 4Gb allocated to the JVM).
>
> I tried using YourKit but I don't get anything about Solr classes.

Sometimes org.apache.* can be in the "ignore" list by default along
with java.*, I guess because people are looking for bottlenecks in
their own code and don't want to look into other libraries.

-Yonik
http://www.lucidimagination.com