You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Samarendra Pratap <sa...@gmail.com> on 2013/01/14 14:55:04 UTC
Solr 4.0 upgrade reduced performance
Hi All,
We recently upgraded Solr from 3.6 to 4.0 but opposite to our expectations
the search performance is noticeably degraded. Any help in debugging is
greatly appreciated.
We are running 56GB index (20 mn docs, 46 indexed fields [4 small, 1 large
text field], 25 unindexed) on a Intel Xenon (2.8 GHz) 6 core CPU machine
with 32GB RAM and SSD disk.
In Solr 3.6 the average request time for a particular type of query was
around 200 - 250 ms. After upgrading, the response time for same type of
queries has jumped to 300 - 350 ms.
We had kept the solrconfig.xml similar to what we used in older version.
There are no changes in hardware / OS settings and schema.xml. Debugging
individual queries through "debugQuery=true" is not helping. Am I missing
something?
Here are primary entries from solrconfig.xml being used by us in Solr 4.0.
<?xml version="1.0" encoding="UTF-8" ?>
<config>
<abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
<luceneMatchVersion>LUCENE_40</luceneMatchVersion>
<!-- default lib tags -->
<dataDir>${solr.data.dir:}</dataDir>
<directoryFactory name="DirectoryFactory"
class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
<indexConfig>
<writeLockTimeout>1000</writeLockTimeout>
<commitLockTimeout>10000</commitLockTimeout>
<useCompoundFile>false</useCompoundFile>
<ramBufferSizeMB>32</ramBufferSizeMB>
<mergeFactor>5</mergeFactor>
<lockType>native</lockType>
<unlockOnStartup>true</unlockOnStartup>
<reopenReaders>true</reopenReaders>
<deletionPolicy class="solr.SolrDeletionPolicy">
<str name="maxCommitsToKeep">1</str>
<str name="maxOptimizedCommitsToKeep">0</str>
</deletionPolicy>
<infoStream file="INFOSTREAM.txt">false</infoStream>
</indexConfig>
<jmx />
<updateHandler class="solr.DirectUpdateHandler2"/>
<query>
<maxBooleanClauses>1024</maxBooleanClauses>
<filterCache class="solr.FastLRUCache" size="4000" initialSize="2000"
autowarmCount="0"/>
<!-- filterCache cumulative_hitratio:0.90, auto warm value was zero in Solr
3.6 as well-->
<queryResultCache class="solr.LRUCache" size="4000" initialSize="1000"
autowarmCount="0"/>
<!-- queryResultCache cumulative_hitratio:0.18, auto warm value was zero in
Solr 3.6 as well -->
<documentCache class="solr.LRUCache" size="25000" initialSize="512"
autowarmCount="512"/>
<!-- documentCache cumulative_hitratio:0.40, auto warm value was 512 in
Solr 3.6 as well -->
<enableLazyFieldLoading>true</enableLazyFieldLoading>
<queryResultWindowSize>20</queryResultWindowSize>
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<str name="q">laptop</str>
<str name="start">0</str>
<str name="rows">25</str>
<str name="qt">im.search</str>
</lst>
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<str name="q">laptop</str>
<str name="start">0</str>
<str name="rows">25</str>
<str name="qt">im.search</str>
</lst>
</arr>
</listener>
<useColdSearcher>true</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers>
</query>
<requestDispatcher handleSelect="true" >
<requestParsers enableRemoteStreaming="true"
multipartUploadLimitInKB="2048000" />
<httpCaching lastModifiedFrom="openTime" etagSeed="Solr">
</httpCaching>
</requestDispatcher>
<requestHandler name="im.search" class="solr.SearchHandler">
<!-- request handler with 5 fields in qf and pf,
a filter with complicated query containing 5 - 6 boolean
and short text fields and 5 boost entries-->
<!-- we are also using pf2, pf3, facets, grouping, tie -->
<!-- selecting 24 fields in "fl" -->
</requestHandler>
<!-- sections below were kept as default -->
</config>
--
Regards,
Samar
Re: Solr 4.0 upgrade reduced performance
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
Are you able to look at any numbers or graphs that show various Solr or
server performance metrics before and after? For example, is Solr cache
utilization the same as before? Is the index the same size and equally
(un)optimized? Is there more disk IO now? I assume you are not using
SolrCloud?
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Mon, Jan 14, 2013 at 8:55 AM, Samarendra Pratap <sa...@gmail.com>wrote:
> Hi All,
> We recently upgraded Solr from 3.6 to 4.0 but opposite to our expectations
> the search performance is noticeably degraded. Any help in debugging is
> greatly appreciated.
>
> We are running 56GB index (20 mn docs, 46 indexed fields [4 small, 1 large
> text field], 25 unindexed) on a Intel Xenon (2.8 GHz) 6 core CPU machine
> with 32GB RAM and SSD disk.
>
> In Solr 3.6 the average request time for a particular type of query was
> around 200 - 250 ms. After upgrading, the response time for same type of
> queries has jumped to 300 - 350 ms.
>
> We had kept the solrconfig.xml similar to what we used in older version.
> There are no changes in hardware / OS settings and schema.xml. Debugging
> individual queries through "debugQuery=true" is not helping. Am I missing
> something?
>
> Here are primary entries from solrconfig.xml being used by us in Solr 4.0.
>
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <config>
>
>
> <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
> <luceneMatchVersion>LUCENE_40</luceneMatchVersion>
> <!-- default lib tags -->
>
> <dataDir>${solr.data.dir:}</dataDir>
>
> <directoryFactory name="DirectoryFactory"
> class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
>
> <indexConfig>
> <writeLockTimeout>1000</writeLockTimeout>
> <commitLockTimeout>10000</commitLockTimeout>
> <useCompoundFile>false</useCompoundFile>
> <ramBufferSizeMB>32</ramBufferSizeMB>
> <mergeFactor>5</mergeFactor>
> <lockType>native</lockType>
> <unlockOnStartup>true</unlockOnStartup>
> <reopenReaders>true</reopenReaders>
>
> <deletionPolicy class="solr.SolrDeletionPolicy">
> <str name="maxCommitsToKeep">1</str>
> <str name="maxOptimizedCommitsToKeep">0</str>
> </deletionPolicy>
>
> <infoStream file="INFOSTREAM.txt">false</infoStream>
> </indexConfig>
>
> <jmx />
>
> <updateHandler class="solr.DirectUpdateHandler2"/>
> <query>
> <maxBooleanClauses>1024</maxBooleanClauses>
>
> <filterCache class="solr.FastLRUCache" size="4000" initialSize="2000"
> autowarmCount="0"/>
> <!-- filterCache cumulative_hitratio:0.90, auto warm value was zero in Solr
> 3.6 as well-->
> <queryResultCache class="solr.LRUCache" size="4000" initialSize="1000"
> autowarmCount="0"/>
> <!-- queryResultCache cumulative_hitratio:0.18, auto warm value was zero in
> Solr 3.6 as well -->
> <documentCache class="solr.LRUCache" size="25000" initialSize="512"
> autowarmCount="512"/>
> <!-- documentCache cumulative_hitratio:0.40, auto warm value was 512 in
> Solr 3.6 as well -->
> <enableLazyFieldLoading>true</enableLazyFieldLoading>
> <queryResultWindowSize>20</queryResultWindowSize>
> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>
> <listener event="newSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
> <str name="q">laptop</str>
> <str name="start">0</str>
> <str name="rows">25</str>
> <str name="qt">im.search</str>
> </lst>
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
> <str name="q">laptop</str>
> <str name="start">0</str>
> <str name="rows">25</str>
> <str name="qt">im.search</str>
> </lst>
> </arr>
> </listener>
>
> <useColdSearcher>true</useColdSearcher>
> <maxWarmingSearchers>2</maxWarmingSearchers>
> </query>
> <requestDispatcher handleSelect="true" >
> <requestParsers enableRemoteStreaming="true"
> multipartUploadLimitInKB="2048000" />
> <httpCaching lastModifiedFrom="openTime" etagSeed="Solr">
> </httpCaching>
> </requestDispatcher>
> <requestHandler name="im.search" class="solr.SearchHandler">
> <!-- request handler with 5 fields in qf and pf,
> a filter with complicated query containing 5 - 6 boolean
> and short text fields and 5 boost entries-->
> <!-- we are also using pf2, pf3, facets, grouping, tie -->
> <!-- selecting 24 fields in "fl" -->
> </requestHandler>
>
> <!-- sections below were kept as default -->
> </config>
>
>
> --
> Regards,
> Samar
>