You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Sathya <sa...@gmail.com> on 2014/02/04 12:11:26 UTC
Solr Searching Issue
Hi Friends,
I am working in Solr 4.6.0 from last 2 months. i have indexed the data in
solr. Indexing size is 8.3GB which is increasing day by day. While im
searching in this index using java programming with multiple instance( more
than 15 instance), the solr is not responding to the search query. Its
getting too slow. Its taking more than 8 hours to search the 7lac data. I am
Using Ubuntu machine with 24GB ram and 1TB HD. Kindly tell me the solution
to solve this issue.
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching Issue
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Wed, 2014-02-05 at 08:17 +0100, Sathya wrote:
> I am running single instance solr and the JVM heap space is minimum 6.3gb
> and maximum 24.31gb. Nothing is running to complete the 24gb except tomcat
> server. I have only 2 copyField entries only.
Your Xmx is the same size as your RAM. It should never be that high.
When your Solr server is hammered, it will gradually expand the heap.
Once expanded, it never contracts. Each time the heap expands, you get
less RAM for disk caching and more GC overhead.
Solr benefits little from extra memory: You are normally better off by
leaving a large amount of memory for disk cache. Ideally, experiment
with Xmx until you find the minimum amount required to run, then
increase it by 50% or so (for unexpected scenarios).
At the very least, set Xmx to 16GB, so that the system is guaranteed to
have memory enough to cache most of the index.
Guessing wildly: Even 6.3GB sounds high for such a small corpus. Have
you tried with just 2GB?
- Toke Eskildsen, State and University Library, Denmark
Re: Solr Searching Issue
Posted by Sathya <sa...@gmail.com>.
Hi Shawn,
I am running single instance solr and the JVM heap space is minimum 6.3gb
and maximum 24.31gb. Nothing is running to complete the 24gb except tomcat
server. I have only 2 copyField entries only.
On Wed, Feb 5, 2014 at 11:49 AM, Shawn Heisey-4 [via Lucene] <
ml-node+s472066n4115455h80@n3.nabble.com> wrote:
> On 2/4/2014 9:49 PM, Sathya wrote:
> > Yes all the instances are reading the same 8GB data at a time. The java
> > search programs(> 15 instances) are running in different machines,
> different
> > JVM and they accessing the solr server machine(Ubuntu 64 bit). And the
> solr
> > Index is not shard. The query rates are too poor(more than 5 seconds
> per
> > search in single instance).
>
> How many instances of Solr itself (not your client application that does
> searches) are you running? How much java heap is allocated to Solr? If
> you don't know, open the admin UI dashboard and look at the "JVM-Memory"
> bar graph. The number at the far right is the one you need. What else
> is running on the same box and competing for that 24GB of RAM? The
> answers to those questions will determine what to look at next.
>
> http://wiki.apache.org/solr/SolrPerformanceProblems
>
> If I read your first message right, "7lac" means that you have 700000
> documents. The lac or lakh is not a common unit of measurement outside
> of South Asia, which means that a lot of people on this list have no
> idea what it is. In order to get an 8GB index out of 700000 documents,
> each one must be quite large, or you've done something that amplifies
> the data, like aggressive tokenization or a LOT of copyField entries.
>
> Thanks,
> Shawn
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207p4115455.html
> To unsubscribe from Solr Searching Issue, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4115207&code=c2F0aGlhLmJsYWNrc3RhckBnbWFpbC5jb218NDExNTIwN3wtMjEyNDcwMTI5OA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207p4115464.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching Issue
Posted by Shawn Heisey <so...@elyograg.org>.
On 2/4/2014 9:49 PM, Sathya wrote:
> Yes all the instances are reading the same 8GB data at a time. The java
> search programs(> 15 instances) are running in different machines, different
> JVM and they accessing the solr server machine(Ubuntu 64 bit). And the solr
> Index is not shard. The query rates are too poor(more than 5 seconds per
> search in single instance).
How many instances of Solr itself (not your client application that does
searches) are you running? How much java heap is allocated to Solr? If
you don't know, open the admin UI dashboard and look at the "JVM-Memory"
bar graph. The number at the far right is the one you need. What else
is running on the same box and competing for that 24GB of RAM? The
answers to those questions will determine what to look at next.
http://wiki.apache.org/solr/SolrPerformanceProblems
If I read your first message right, "7lac" means that you have 700000
documents. The lac or lakh is not a common unit of measurement outside
of South Asia, which means that a lot of people on this list have no
idea what it is. In order to get an 8GB index out of 700000 documents,
each one must be quite large, or you've done something that amplifies
the data, like aggressive tokenization or a LOT of copyField entries.
Thanks,
Shawn
Re: Solr Searching Issue
Posted by Sathya <sa...@gmail.com>.
Hi,
Yes all the instances are reading the same 8GB data at a time. The java
search programs(> 15 instances) are running in different machines, different
JVM and they accessing the solr server machine(Ubuntu 64 bit). And the solr
Index is not shard. The query rates are too poor(more than 5 seconds per
search in single instance).
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207p4115445.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching Issue
Posted by Daniel Collins <da...@gmail.com>.
You also said you have multiple instances (> 15) but are they all reading
the same 8Gb data (in which case it must be static or you'd get locking
problems) or is it partitioned/sharded somehow? I'd have the same
questions as the others, query rates, how are your queries distributed over
the instances, etc?
With the 15 instances, is that 15 cores in 1 JVM or 15 JVMs? The latter
would presumably be eating 24Gb of RAM quite easily (even 2GB heap per JVM
would be 30Gb of RAM), but the former would need a large heap (and 64-bit
JVM) to be of any use I would expect.
On 4 February 2014 13:24, Jack Krupansky <ja...@basetechnology.com> wrote:
> Maybe you need a larger Java heap.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Sathya
> Sent: Tuesday, February 4, 2014 6:11 AM
> To: solr-user@lucene.apache.org
> Subject: Solr Searching Issue
>
>
> Hi Friends,
>
> I am working in Solr 4.6.0 from last 2 months. i have indexed the data in
> solr. Indexing size is 8.3GB which is increasing day by day. While im
> searching in this index using java programming with multiple instance( more
> than 15 instance), the solr is not responding to the search query. Its
> getting too slow. Its taking more than 8 hours to search the 7lac data. I
> am
> Using Ubuntu machine with 24GB ram and 1TB HD. Kindly tell me the solution
> to solve this issue.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Solr-Searching-Issue-tp4115207.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr Searching Issue
Posted by Jack Krupansky <ja...@basetechnology.com>.
Maybe you need a larger Java heap.
-- Jack Krupansky
-----Original Message-----
From: Sathya
Sent: Tuesday, February 4, 2014 6:11 AM
To: solr-user@lucene.apache.org
Subject: Solr Searching Issue
Hi Friends,
I am working in Solr 4.6.0 from last 2 months. i have indexed the data in
solr. Indexing size is 8.3GB which is increasing day by day. While im
searching in this index using java programming with multiple instance( more
than 15 instance), the solr is not responding to the search query. Its
getting too slow. Its taking more than 8 hours to search the 7lac data. I am
Using Ubuntu machine with 24GB ram and 1TB HD. Kindly tell me the solution
to solve this issue.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching Issue
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi;
Which JVM parameters do you use?
Thanks;
Furkan KAMACI
2014-02-04 Sathya <sa...@gmail.com>:
> Hi Furkan,
>
> I have index the subjects that containing only 1 to 10 words per subject.
> And query rate is minimum 7 seconds for one searching. And i have single
> solr instance only.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207p4115234.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr Searching Issue
Posted by Sathya <sa...@gmail.com>.
Hi Furkan,
I have index the subjects that containing only 1 to 10 words per subject.
And query rate is minimum 7 seconds for one searching. And i have single
solr instance only.
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207p4115234.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Searching Issue
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi;
Your index size is not much for a 24 GB machine. There should be any other
problem for it. What is your document size and query rate per second? On
the other hand how do you start up your Solr instance (which parameters do
you use)?
Thanks;
Furkan KAMACI
2014-02-04 Sathya <sa...@gmail.com>:
> Hi Friends,
>
> I am working in Solr 4.6.0 from last 2 months. i have indexed the data in
> solr. Indexing size is 8.3GB which is increasing day by day. While im
> searching in this index using java programming with multiple instance( more
> than 15 instance), the solr is not responding to the search query. Its
> getting too slow. Its taking more than 8 hours to search the 7lac data. I
> am
> Using Ubuntu machine with 24GB ram and 1TB HD. Kindly tell me the solution
> to solve this issue.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Searching-Issue-tp4115207.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>