You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by melb <me...@gmail.com> on 2014/12/02 09:27:52 UTC
Slow queries
Hi,
I have a solr collection with 16 millions documents and growing daily with
10000 documents
recently it is becoming slow to answer my request ( several seconds)
specially when I use multi-words query
I am running solr on a machine with 32G RAM but heavy used one
What are my options to optimize the collection and speed up querying it
is it normal with this volume of data? is sharding is a good solution?
regards,
--
View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by Erick Erickson <er...@gmail.com>.
bq: Is it better to put the solr on dedicated machine?
Yes, absolutely. Solr _likes_ memory, and on a
machine with lots of other processes you'll keep
running into this problem.
FWIW, I've seen between 10M and 300M docs fit into
16G for the JVM. But see Uwe's excellent blog on MMapDirectory
and not over-allocating memory to the JVM here:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
Also see:
https://wiki.apache.org/solr/SolrPerformanceProblems
and
http://wiki.apache.org/solr/SolrPerformanceFactors
Best,
Erick
On Tue, Dec 2, 2014 at 1:02 AM, melb <me...@gmail.com> wrote:
> Yes performance degraded over the time, I can raise the memory but I can't
> do it every time and the volume will keep growing
> Is it better to put the solr on dedicated machine?
> Is there any thing else that can be done to the solr instance for example
> deviding the collection?
>
> rgds,
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032p4172039.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by Siegfried Goeschl <sg...@gmx.at>.
Hi,
using Jetty is the recommended approach while using Tomcat is not recommend (unless you are a Tomcat shop).
But any discussion comes back to the original question - why is it slow now? Are you I/O-bound, are CPU-bound, how many documents are committed/deleted over the time, do you having expensive SOLR queries, what is your server code is doing - many questions and even more answers to that - in other words nobody can help you when the basic work is not done. And when you know your application performance-wise you probably also the solution :-)
Cheers,
Siegfried Goeschl
> On 08 Dec 2014, at 11:00, melb <me...@gmail.com> wrote:
>
> THnks for the answer
> A dedicated box will be a great solution but I will wait for that solution,
> I have restricted sources
> Is Optimze action can improve performance?
> Is using default servlet engine Jetty can be harmful for the performance,
> SHould I use an independant tomcat engine?
>
> rgds,
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032p4173092.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by melb <me...@gmail.com>.
THnks for the answer
A dedicated box will be a great solution but I will wait for that solution,
I have restricted sources
Is Optimze action can improve performance?
Is using default servlet engine Jetty can be harmful for the performance,
SHould I use an independant tomcat engine?
rgds,
--
View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032p4173092.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by Siegfried Goeschl <sg...@gmx.at>.
It might be a good idea to
* move SOLR to a dedicated box :-)
* load your SOLR server with 20.000.000 documents (the estimated number of documents after three years) and do performance testing & tuning
Afterwards you have some hard facts about hardware sizing and expected performance for the next three years :-)
Cheers,
Siegfried Goeschl
> On 02 Dec 2014, at 10:02, melb <me...@gmail.com> wrote:
>
> Yes performance degraded over the time, I can raise the memory but I can't
> do it every time and the volume will keep growing
> Is it better to put the solr on dedicated machine?
> Is there any thing else that can be done to the solr instance for example
> deviding the collection?
>
> rgds,
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032p4172039.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by melb <me...@gmail.com>.
Yes performance degraded over the time, I can raise the memory but I can't
do it every time and the volume will keep growing
Is it better to put the solr on dedicated machine?
Is there any thing else that can be done to the solr instance for example
deviding the collection?
rgds,
--
View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032p4172039.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow queries
Posted by Siegfried Goeschl <sg...@gmx.at>.
If you performance was fine but degraded over the time it might be
easier to check / increase the memory to have better disk caching.
Cheers,
Siegfried Goeschl
On 02.12.14 09:27, melb wrote:
> Hi,
>
> I have a solr collection with 16 millions documents and growing daily with
> 10000 documents
> recently it is becoming slow to answer my request ( several seconds)
> specially when I use multi-words query
> I am running solr on a machine with 32G RAM but heavy used one
>
> What are my options to optimize the collection and speed up querying it
> is it normal with this volume of data? is sharding is a good solution?
>
> regards,
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Slow-queries-tp4172032.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>