[GitHub] helix pull request #294: Implement view cluster aggregator

[zhan849 <gi...@git.apache.org>]

GitHub user zhan849 opened a pull request:

    https://github.com/apache/helix/pull/294

    Implement view cluster aggregator

    Based on #266 , design of helix view aggregator, here is the implementation. Helix view aggregator will be a different module under helix repo, and the impl will use components from helix-core.
    Here is a debrief of this PR:
    
    - View cluster related information will be added to `ClusterConfig`, and related java apis will be added to helix-core.
    - `SourceClusterDataProvider` is a component that watches changes from source cluster, updates its cached data, and notify data change event via a given channel
    - `ViewClusterRefresher` is a component that does the actual refresh operation of the view cluster. It computes diff between source clusters and view cluster, make changes to view cluster accordingly
    - `SourceClusterConfigChangeAction` is a wrapper containing information about what to update for a source cluster. It takes in old and new `ClusterConfig` and compute actions to adopt view cluster to the changes
    - `HelixViewAggregator` hooks up small components and contains the main reconciliation loop for refreshing view cluster
    - Metrics recording mechanism is added
    - starting helix view aggregator in a stand-alone mode and distributed mode (by adding state model into helix participant) is supported
    - related unit tests and integration tests are added


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhan849/helix harry/view-aggregator-impl

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/helix/pull/294.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #294
    
----
commit d6b98f236d85522866e4e7584561a02ce525d142
Author: Harry Zhang <hr...@...>
Date:   2018-11-01T22:26:45Z

    [HELIX-781]: add java apis for view cluster config support

commit 759cf8a9a48805851c50681afc98b2cafe309ff7
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T21:28:23Z

    [HELIX-781] set up module structure for helix view aggregator

commit 2ae85f6127b43aa7928776ede7e9bbbfe94e9385
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T21:38:14Z

    [HELIX-781] implement SourceClusterDataProvider and add tests

commit 1df6c7e15bc31fb424883e3febcd9f8cd019de82
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T21:42:56Z

    [HELIX-781] implement ViewClusterRefresher and add tests

commit 7802cf2443d45092c69cb529b4bba17a3c45dc31
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T21:50:35Z

    [HELIX-781] implement SourceClusterConfigChangeAction and added tests

commit 985a4ac2bcc43738d9c7ab463ad606f8691f7298
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T21:59:21Z

    [HELIX-781] implement helix view aggregator main logic and added tests

commit 3e2cfb695720fc09f181b55a3a31adc2b44f94a4
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T22:08:43Z

    [HELIX-781] added metrics to helix view aggregator and added tests

commit 82d33e0aad1ad2279f864fea6d5851129ba56610
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T22:11:00Z

    [HELIX-781] added main function to start helix view aggregator via bash

commit 11a7c126ca8bb9298754cb9127af588148af0902
Author: Harry Zhang <hr...@...>
Date:   2018-11-02T22:12:57Z

    [HELIX-781] support deploy helix view aggregator in a distributed fashion using helix participant

----


---