Solr is working very slow after certain time

[Ameya Aware <am...@gmail.com>]

Hi,

i could index around 100000 documents in couple of hours. But after that
the time for indexing very large (around just 15-20 documents per minute).

i have taken care of garbage collection.

i am passing below parameters to Solr:
-Xms6144m -Xmx6144m -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC
-XX:ConcGCThreads=6 -XX:ParallelGCThreads=6
-XX:CMSInitiatingOccupancyFraction=70 -XX:NewRatio=3
-XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled
-XX:+UseCompressedOops -XX:+ParallelRefProcEnabled -XX:+UseLargePages
-XX:+AggressiveOpts -XX:-UseGCOverheadLimit



Can anyone help to solve this problem?


Thanks,
Ameya

re: Solr is working very slow after certain time

[Chris Morley <ch...@depahelix.com>]

A page Solr Performance Factors mentions 2 big tips that may help you, but 
you have to read the rest of the page to make sure you understand the 
caveats there.

In general, adding many documents per update request is faster than one per 
update request.  

Reducing the frequency of automatic commits or disabling them entirely may 
speed indexing.  
 Source:
 http://wiki.apache.org/solr/SolrPerformanceFactors#Indexing_Performance
  
  

----------------------------------------
 From: "Ameya Aware" <am...@gmail.com>
Sent: Thursday, July 31, 2014 1:56 PM
To: solr-user@lucene.apache.org
Subject: Solr is working very slow after certain time   
Hi,

i could index around 100000 documents in couple of hours. But after that
the time for indexing very large (around just 15-20 documents per minute).

i have taken care of garbage collection.

i am passing below parameters to Solr:
-Xms6144m -Xmx6144m -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC
-XX:ConcGCThreads=6 -XX:ParallelGCThreads=6
-XX:CMSInitiatingOccupancyFraction=70 -XX:NewRatio=3
-XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled
-XX:+UseCompressedOops -XX:+ParallelRefProcEnabled -XX:+UseLargePages
-XX:+AggressiveOpts -XX:-UseGCOverheadLimit

Can anyone help to solve this problem?

Thanks,
Ameya
 

Re: Solr is working very slow after certain time

[Otis Gospodnetic <ot...@gmail.com>]

Can we look at your disk IO and CPU?  SPM <http://sematext.com/spm/> can
help.

Isn't "UseCompressedOops" a typo? And deprecated?  In general, may want to
simplify your JVM params unless you are really sure they are helping.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Thu, Jul 31, 2014 at 7:54 PM, Ameya Aware <am...@gmail.com> wrote:

> Hi,
>
> i could index around 100000 documents in couple of hours. But after that
> the time for indexing very large (around just 15-20 documents per minute).
>
> i have taken care of garbage collection.
>
> i am passing below parameters to Solr:
> -Xms6144m -Xmx6144m -XX:MaxPermSize=128m -XX:+UseConcMarkSweepGC
> -XX:ConcGCThreads=6 -XX:ParallelGCThreads=6
> -XX:CMSInitiatingOccupancyFraction=70 -XX:NewRatio=3
> -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled
> -XX:+UseCompressedOops -XX:+ParallelRefProcEnabled -XX:+UseLargePages
> -XX:+AggressiveOpts -XX:-UseGCOverheadLimit
>
>
>
> Can anyone help to solve this problem?
>
>
> Thanks,
> Ameya
>