[jira] [Created] (SOLR-5872) Eliminate overseer queue

["Noble Paul (JIRA)" <ji...@apache.org>]

Noble Paul created SOLR-5872:
--------------------------------

             Summary: Eliminate overseer queue 
                 Key: SOLR-5872
                 URL: https://issues.apache.org/jira/browse/SOLR-5872
             Project: Solr
          Issue Type: Improvement
          Components: SolrCloud
            Reporter: Noble Paul
            Assignee: Noble Paul


The overseer queue is one of the busiest points in the entire system. The raison d'être of the queue is
 * Provide batching of operations for the main clusterstate,json so that state updates are minimized 
* Avoid race conditions and ensure order

Now , as we move the individual collection states out of the main clusterstate.json, the batching is not useful anymore.

Race conditions can easily be solved by using a compare and set in Zookeeper. 

The proposed solution  is , whenever an operation is required to be performed on the clusterstate, the same thread (and of course the same JVM)

 # read the fresh state and version of zk node  
 # construct the new state 
 # perform a compare and set
 # if compare and set fails go to step 1

This should be limited to all operations performed on external collections because batching would be required for others 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org