[jira] [Updated] (SOLR-9835) Create another replication mode for SolrCloud

["Cao Manh Dat (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-9835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cao Manh Dat updated SOLR-9835:
-------------------------------
    Attachment: SOLR-9835.patch

Here a the patch for this issue, I didn't do anything related to Leader Recovery, just wanna upload the patch soon so anyone can comment first.
- Add {{onlyLeaderIndexes}} property to {{DocCollection}}
- Modified {{SolrIndexWriter#setCommitData(IndexWriter iw, long commitCommandVersion)}} to store version of commit command to commit point.
- Start replication process to periodically poll latest segments from leader when replica become active
- Add {{UpdateCommand.IGNORE_INDEXWRITER}} so
  + DistributedUpdateProcessor can look into the role of current core to add the flag to command
  + DirectUpdateHandler2 based on the existence of {{IGNORE_INDEXWRITER}} to skip making changes to IW, just write down the updateCommand to its tlog
- Make sure that the current tlog of replicas have all uncommit updates
  + When replication process is complete
  + When replica restart



> Create another replication mode for SolrCloud
> ---------------------------------------------
>
>                 Key: SOLR-9835
>                 URL: https://issues.apache.org/jira/browse/SOLR-9835
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Cao Manh Dat
>         Attachments: SOLR-9835.patch
>
>
> The current replication mechanism of SolrCloud is called state machine, which replicas start in same initial state and for each input, the input is distributed across replicas so all replicas will end up with same next state. 
> But this type of replication have some drawbacks
> - The commit (which costly) have to run on all replicas
> - Slow recovery, because if replica miss more than N updates on its down time, the replica have to download entire index from its leader.
> So we create create another replication mode for SolrCloud called state transfer, which acts like master/slave replication. In basically
> - Leader distribute the update to other replicas, but the leader only apply the update to IW, other replicas just store the update to UpdateLog (act like replication).
> - Replicas frequently polling the latest segments from leader.
> Pros:
> - Lightweight for indexing, because only leader are running the commit, updates.
> - Very fast recovery, replicas just have to download the missing segments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org