You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@lucene.apache.org by Sachin Kale <sa...@gmail.com> on 2014/10/01 17:49:48 UTC
Master-Slave setup using SolrCloud
Hello,
We are trying to move our traditional master-slave Solr configuration to
SolrCloud. As our index size is very small (around 1 GB), we are having
only one shard.
So basically, we are having same master-slave configuration with one leader
and 6 replicas.
We are experimenting with maxTime of both AutoCommit and AutoSoftCommit.
Currently, autoCommit maxTime is 15 minutes and autoSoftCommit is 1 minute
(Let me know if these values does not make sense).
Caches are set such that warmup time is at most 20 seconds.
We are having continuous indexing requests mostly for updating the existing
documents. Few requests are for deleting/adding the documents.
The problem we are facing is that we are getting very frequent
NullPointerExceptions.
We get continuous 200-300 such exceptions within a period of 30 seconds and
for next few minutes, it works fine.
I am not sure what would be causing it. My guess, whenever, it is trying to
replay tlog, we are getting these exceptions. Is anything wrong in my
configuration?
Re: Master-Slave setup using SolrCloud
Posted by Sachin Kale <sa...@gmail.com>.
Actually, the image was captured from SOLR dashboard having following lines:
*Error SolrCore java.lang.NullPointerException*
*Error SolrDispatchFilter java.lang.NullPointerException*
We are now on SolrCloud setup completely.
Memory allocated to each SOLR instance is 7GB. I guess it is more than
sufficient for 1 GB index, right?
Indexes are stored as normal, local filesystem.
We are using latest SOLR version, 4.10.0
I am using three caches:
Query Cache: Size 4096, autoWarmCount 2048
Filter cache: size 8192, autoWarmCount 4096
Document cache: size 4096
I think I am setting wrong values from commit maxTime. Just found out
following document:
http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
According to this document, if our requirement is Index-Heavy, Query-Heavy,
then autoSoftCommit should be as long as high (not sure what should be its
value in my case) and hardCommit to 15 seconds (I am using 15 minutes).
Should I change my hard commit interval to 15 minutes?
On Wed, Oct 1, 2014 at 9:58 PM, Mark Bennett <ma...@lucidworks.com>
wrote:
> Hello Sachin,
>
> FYI, at least in my email client, I don't see the embedded graphic (shows
> a broken image link). Not sure if that's on my end, or maybe the mailing
> list server.
>
> To your question, some clarification please:
>
> * You mentioned moving to SolrCloud, but are you still currently running
> v3 master/slave, or is this now a SolrCloud setup?
>
> * Is it possible you're running low on memory? Could you maybe share the
> startup switches (memory, GC, etc)
>
> * A long shot, but I have to ask, where are the indexes being stored? is
> this a "normal", local filesystem, or shard storage, or some type of more
> exotic storage? (HDFS, etc)
>
> * Exact version of Solr?
>
> * The NPE's (Null Pointer Exceptions), are those on the master or
> replicas? Have you tried temporarily running with fewer replaces, perhaps
> just 1, for temp debugging purposes?
>
> Mark
>
> --
> Mark Bennett / LucidWorks: Search & Big Data / mark.bennett@lucidworks.com
> <ma...@lucidworks.com>
> Office: 408-898-4201 / Telecommute: 408-733-0387 / Cell: 408-829-6513
>
> On Oct 1, 2014, at 8:49 AM, Sachin Kale <sachinpkale@gmail.com<mailto:
> sachinpkale@gmail.com>> wrote:
>
> Hello,
>
> We are trying to move our traditional master-slave Solr configuration to
> SolrCloud. As our index size is very small (around 1 GB), we are having
> only one shard.
> So basically, we are having same master-slave configuration with one
> leader and 6 replicas.
> We are experimenting with maxTime of both AutoCommit and AutoSoftCommit.
> Currently, autoCommit maxTime is 15 minutes and autoSoftCommit is 1 minute
> (Let me know if these values does not make sense).
>
> Caches are set such that warmup time is at most 20 seconds.
>
> We are having continuous indexing requests mostly for updating the
> existing documents. Few requests are for deleting/adding the documents.
>
> The problem we are facing is that we are getting very frequent
> NullPointerExceptions.
> We get continuous 200-300 such exceptions within a period of 30 seconds
> and for next few minutes, it works fine.
>
>
>
> I am not sure what would be causing it. My guess, whenever, it is trying
> to replay tlog, we are getting these exceptions. Is anything wrong in my
> configuration?
>
>
Re: Master-Slave setup using SolrCloud
Posted by Mark Bennett <ma...@lucidworks.com>.
Hello Sachin,
FYI, at least in my email client, I don't see the embedded graphic (shows a broken image link). Not sure if that's on my end, or maybe the mailing list server.
To your question, some clarification please:
* You mentioned moving to SolrCloud, but are you still currently running v3 master/slave, or is this now a SolrCloud setup?
* Is it possible you're running low on memory? Could you maybe share the startup switches (memory, GC, etc)
* A long shot, but I have to ask, where are the indexes being stored? is this a "normal", local filesystem, or shard storage, or some type of more exotic storage? (HDFS, etc)
* Exact version of Solr?
* The NPE's (Null Pointer Exceptions), are those on the master or replicas? Have you tried temporarily running with fewer replaces, perhaps just 1, for temp debugging purposes?
Mark
--
Mark Bennett / LucidWorks: Search & Big Data / mark.bennett@lucidworks.com<ma...@lucidworks.com>
Office: 408-898-4201 / Telecommute: 408-733-0387 / Cell: 408-829-6513
On Oct 1, 2014, at 8:49 AM, Sachin Kale <sa...@gmail.com>> wrote:
Hello,
We are trying to move our traditional master-slave Solr configuration to SolrCloud. As our index size is very small (around 1 GB), we are having only one shard.
So basically, we are having same master-slave configuration with one leader and 6 replicas.
We are experimenting with maxTime of both AutoCommit and AutoSoftCommit. Currently, autoCommit maxTime is 15 minutes and autoSoftCommit is 1 minute (Let me know if these values does not make sense).
Caches are set such that warmup time is at most 20 seconds.
We are having continuous indexing requests mostly for updating the existing documents. Few requests are for deleting/adding the documents.
The problem we are facing is that we are getting very frequent NullPointerExceptions.
We get continuous 200-300 such exceptions within a period of 30 seconds and for next few minutes, it works fine.
I am not sure what would be causing it. My guess, whenever, it is trying to replay tlog, we are getting these exceptions. Is anything wrong in my configuration?