[jira] [Updated] (USERGRID-909) Evaluate removing cache from edge shard management

["Jeffrey (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/USERGRID-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jeffrey  updated USERGRID-909:
------------------------------
    Sprint: Usergrid 28, Usergrid 29  (was: Usergrid 28)

> Evaluate removing cache from edge shard management
> --------------------------------------------------
>
>                 Key: USERGRID-909
>                 URL: https://issues.apache.org/jira/browse/USERGRID-909
>             Project: Usergrid
>          Issue Type: Story
>          Components: Stack
>            Reporter: Todd Nine
>            Assignee: Todd Nine
>
> Currently, using caches with the proposal phase for new shards has caused large numbers of tombstones.  Review the algorithm, and determine the cause of the additional shard proposals.
> Usergrid currently performs 5 shard lookups on write and most miss the cache.  We'll change the following
> *Shard allocation*
> Shard allocation will not perform a propose + check phase. The following algorithm will occur.  This must occur at EACH_QUORUM to ensure that we have consensus across regions.  
> Any node will propose a new shard pivot.  This pivot will have compacted set to false, and will have a column timestamp of a new timeuuid in micros
> The proposing node will then read back all compacted = false shard pivots.  The shard pivot with the minimum timestamp will be retained, all others will be deleted as proposals.  This will function similarly to distributed read locks in Cassandra.  Lowest proposed value always wins. 
> Allocation is complete
> *Shard compaction*
> After a successful proposal allocation, the allocating node will copy all edges from the previously compacted shard into the new shard.  It will continue to copy until no new edges are returned on read.
> Once read has been completed, all edges that were copied can be deleted from the previous shard.  
> This target shard will be marked as compacted
> *Consistency*
> When performing consistency, we will need a higher consistency level.  To ensure that shards exist worldwide, proposal + selection should occur at EACH_QUORUM.  Standard reads can occur as LOCAL_QUORUM



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)