You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by PeterKerk <ve...@hotmail.com> on 2013/12/27 12:38:52 UTC

Disable caching on sorting to improve performance

I'm getting a lot of java heap memory full errors. I've now been reading into
solr performance (in the meantime also configuring the sematext tools to try
to drill down to the cause)

I already increased the memory available to Solr:
bash -c "cd /cygdrive/c/Databases/solr-4.3.1/example/;java
-Dsolr.solr.home="./example-DIH/solr/" -jar -Xmx200m -Xmx1200m start.jar" 

And now I read:
Factors that affect memory usage:
http://stackoverflow.com/questions/1546898/how-to-reduce-solr-memory-usage
I see that sorting affects memory usage. I have the feeling that is the case
for me, because since I implemented sorting the memory errors are going
through the roof. 

I read here
http://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters how to
do it on filter queries, but I was wondering how I can disable the caching
on the sort parameter in below statement where it now has
`&sort=clickcount%20desc,prijs%20desc,updatedate%20desc`


searchquery.Append("&fl=id,artikelnummer,titel,friendlyurl,pricerange,lang,currency,createdate")
searchquery.Append("&facet.field=pricerange")
searchquery.Append("&facet.mincount=1") 
searchquery.Append("&facet.sort=index") 
searchquery.Append("&omitHeader=true")
searchquery.Append("&sort=clickcount%20desc,prijs%20desc,updatedate%20desc") 
          



--
View this message in context: http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Disable caching on sorting to improve performance

Posted by PeterKerk <ve...@hotmail.com>.

Thanks and good call, that has been there for quite some time! 
I've changed it to: -Xms200m -Xmx1500m 
I'll look into the effect of this first.



--
View this message in context: http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356p4108362.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Disable caching on sorting to improve performance

Posted by Markus Jelsma <ma...@openindex.io>.

Hi,

Sorting uses Lucene FieldCache which you cannot disable. If you have lots of documents and sort on three fields, you must increase the heap space. Also, note that you have defined Xmx twice here, i don't know what effect that will have.

Cheers
 
 
-----Original message-----
> From:PeterKerk <ve...@hotmail.com>
> Sent: Friday 27th December 2013 12:39
> To: solr-user@lucene.apache.org
> Subject: Disable caching on sorting to improve performance
> 
> I'm getting a lot of java heap memory full errors. I've now been reading into
> solr performance (in the meantime also configuring the sematext tools to try
> to drill down to the cause)
> 
> I already increased the memory available to Solr:
> bash -c "cd /cygdrive/c/Databases/solr-4.3.1/example/;java
> -Dsolr.solr.home="./example-DIH/solr/" -jar -Xmx200m -Xmx1200m start.jar" 
> 
> And now I read:
> Factors that affect memory usage:
> http://stackoverflow.com/questions/1546898/how-to-reduce-solr-memory-usage
> I see that sorting affects memory usage. I have the feeling that is the case
> for me, because since I implemented sorting the memory errors are going
> through the roof. 
> 
> I read here
> http://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters how to
> do it on filter queries, but I was wondering how I can disable the caching
> on the sort parameter in below statement where it now has
> `&sort=clickcount%20desc,prijs%20desc,updatedate%20desc`
> 
> 
> searchquery.Append("&fl=id,artikelnummer,titel,friendlyurl,pricerange,lang,currency,createdate")
> searchquery.Append("&facet.field=pricerange")
> searchquery.Append("&facet.mincount=1") 
> searchquery.Append("&facet.sort=index") 
> searchquery.Append("&omitHeader=true")
> searchquery.Append("&sort=clickcount%20desc,prijs%20desc,updatedate%20desc") 
>           
> 
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>