SolrCloud not commit documents automatically

[Chet Rebman <ch...@coalliance.org>]

Hello all,

I am new to SolrCloud. I have read that solrCloud prefers to commit automatically, rather then recieve commit commands. However solrCloud never commits my documents.

I am running SolrCloud 6.1 with three solr servers and 3 zookeeper instances. I am using CloudSolrClient with all three zookeeper instances loaded into the constructor. The indexing procedes normally. After about 80,000 documents I get a ConcurrentModificationException. I believe this error is the side effect of the documents not being commited. I can wait for an hour and nothing is ever commited. As soon as I perform a 'update?commit=true' the documents appear and I can restart indexing as if nothing had happened. My solrconfing contains.

<updateHandler class="solr.DirectUpdateHandler2">
   <autoCommit>
      <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
      <openSearcher>false</openSearcher>
   </autoCommit>
   <autoSoftCommit>
      <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
   </autoSoftCommit>
</updateHandler>

My understanding is this should performa a hard commit every 15 seconds.

I uploaded my config files to solrCloud with...
./zkCli.sh -cmd upconfig -zkhost  localhost:2181 -confname XXX  -confdir YYY

tia!

..Chet

Re: SolrCloud not commit documents automatically

[Shawn Heisey <ap...@elyograg.org>]

On 7/1/2016 10:01 AM, Chet Rebman wrote:
> <updateHandler class="solr.DirectUpdateHandler2">
>    <autoCommit>
>       <maxTime>${solr.autoCommit.maxTime:15000}</maxTime>
>       <openSearcher>false</openSearcher>
>    </autoCommit>
>    <autoSoftCommit>
>       <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
>    </autoSoftCommit>
> </updateHandler>
>
> My understanding is this should performa a hard commit every 15 seconds.

Yes, that is exactly what it will do -- but none of those commits will
make new documents visible.  The autoCommit has openSearcher set to false.

Don't change openSearcher to true -- the reason this config is there has
nothing to do with document visibility -- it is about durability and
transaction log size.

http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

You'll need to use explicit commits, commitWithin, or autoSoftCommit for
document visibility.  Make these commits as infrequently as you can --
an interval of several minutes if that's possible.  Opening a searcher
is very resource-intensive, and not something you want to be doing
frequently.

Thanks,
Shawn