[jira] [Issue Comment Deleted] (HELIX-659) Support Additional Associate States

["Jiajun Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HELIX-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiajun Wang updated HELIX-659:
------------------------------
    Comment: was deleted

(was: Workflow

https://www.lucidchart.com/documents/view/5217edea-896d-4ddc-8a93-c31eeb273f38)

> Support Additional Associate States
> -----------------------------------
>
>                 Key: HELIX-659
>                 URL: https://issues.apache.org/jira/browse/HELIX-659
>             Project: Apache Helix
>          Issue Type: New Feature
>          Components: helix-core
>    Affects Versions: 0.6.x
>            Reporter: Jiajun Wang
>         Attachments: Associate States Processing Flow.pdf
>
>
> Currently, Helix only supports management a single state for all resources/partitions. However, in the real world, cluster management requirements may be more complicated than that.
> In Pinot, for example, each partition need to be assigned a version for ensuring data consistency.
> When a new version comes, the system needs to replace the old partition with the new one. And the replacement is done one partition by one partition. So any reads during this period will get inconsistent data.
> Pinot system cannot directly put the version information into the section(partition) state field because it is already occupied by the main state (offline-online for instance) used by Helix controller.
> So Pinot team relies on some workarounds to implement their application logic: creating a new resource with the latest version and replace them after the resource is fully loaded. And for Helix controller, version is unknown.
> Another option is Pinot team maintaining their own config item or property store item for recording versions.
> Both ways require Pinot team implementing version control themselves.
> Another requirement is from Ambry team. Where partition can be "ONLINE:READ" or "ONLINE:WRITE".
> In both cases, single state mechanism is not sufficient for applications' requirement.
> It would be very helpful to provide a framework level feature that supports more than one states for each partition.
> Benefits: 
> # The application doesn't need to write additional code for managing additional states.
> # Avoid potential conflict when multiple states transition happens concurrently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)