You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@helix.apache.org by Kanak Biscuitwala <ka...@apache.org> on 2014/01/15 01:39:35 UTC
Re: Review Request 16407: [HELIX-345] Speed up the controller pipeline for
0.6.x
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16407/
-----------------------------------------------------------
(Updated Jan. 14, 2014, 4:39 p.m.)
Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
Changes
-------
Updated diff. See comment https://issues.apache.org/jira/browse/HELIX-345?focusedCommentId=13871418&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13871418
Bugs: HELIX-345
Repository: helix-git
Description
-------
commit 90e995b02a59622a0526897faaee3d6ae057b63e
Author: Kanak Biscuitwala <ka...@apache.org>
Date: Thu Dec 19 15:44:48 2013 -0800
[HELIX-345] Speed up the controller pipeline for 0.6.x
:100644 100644 5c0a94a... 5858958... M helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
:100644 100644 df893bb... 9e2452b... M helix-core/src/test/java/org/apache/helix/Mocks.java
Diffs (updated)
-----
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java 5c0a94a
helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java 192a645
helix-core/src/test/java/org/apache/helix/Mocks.java df893bb
Diff: https://reviews.apache.org/r/16407/diff/
Testing
-------
Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme cases.
Thanks,
Kanak Biscuitwala
Re: Review Request 16407: [HELIX-345] Speed up the controller pipeline for
0.6.x
Posted by Kishore Gopalakrishna <ki...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16407/#review32429
-----------------------------------------------------------
do we have a test case where controller loses leadership and becomes leader again and it ensures that the messages and entire cluster data is refreshed ?
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
<https://reviews.apache.org/r/16407/#comment61256>
can we have if traceenabled check outside of the for loop
- Kishore Gopalakrishna
On Jan. 21, 2014, 9:13 p.m., Kanak Biscuitwala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16407/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2014, 9:13 p.m.)
>
>
> Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
>
>
> Bugs: HELIX-345
>
>
> Repository: helix-git
>
>
> Description
> -------
>
> commit 90e995b02a59622a0526897faaee3d6ae057b63e
> Author: Kanak Biscuitwala <ka...@apache.org>
> Date: Thu Dec 19 15:44:48 2013 -0800
>
> [HELIX-345] Speed up the controller pipeline for 0.6.x
>
> :100644 100644 5c0a94a... 5858958... M helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
> :100644 100644 df893bb... 9e2452b... M helix-core/src/test/java/org/apache/helix/Mocks.java
>
>
> Diffs
> -----
>
> helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 3b522e6
> helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java 5c0a94a
> helix-core/src/main/java/org/apache/helix/controller/stages/ReadClusterDataStage.java f077d29
> helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java 192a645
> helix-core/src/test/java/org/apache/helix/Mocks.java df893bb
> helix-core/src/test/java/org/apache/helix/controller/stages/TestRebalancePipeline.java 1a11be3
>
> Diff: https://reviews.apache.org/r/16407/diff/
>
>
> Testing
> -------
>
> Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme cases.
>
>
> Thanks,
>
> Kanak Biscuitwala
>
>
Re: Review Request 16407: [HELIX-345] Speed up the controller pipeline for
0.6.x
Posted by Kishore Gopalakrishna <ki...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16407/#review33182
-----------------------------------------------------------
Ship it!
Ship It!
- Kishore Gopalakrishna
On Jan. 21, 2014, 11:54 p.m., Kanak Biscuitwala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16407/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2014, 11:54 p.m.)
>
>
> Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
>
>
> Bugs: HELIX-345
>
>
> Repository: helix-git
>
>
> Description
> -------
>
> commit 90e995b02a59622a0526897faaee3d6ae057b63e
> Author: Kanak Biscuitwala <ka...@apache.org>
> Date: Thu Dec 19 15:44:48 2013 -0800
>
> [HELIX-345] Speed up the controller pipeline for 0.6.x
>
> :100644 100644 5c0a94a... 5858958... M helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
> :100644 100644 df893bb... 9e2452b... M helix-core/src/test/java/org/apache/helix/Mocks.java
>
>
> Diffs
> -----
>
> helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 3b522e6
> helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java 5c0a94a
> helix-core/src/main/java/org/apache/helix/controller/stages/ReadClusterDataStage.java f077d29
> helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java 192a645
> helix-core/src/test/java/org/apache/helix/Mocks.java df893bb
> helix-core/src/test/java/org/apache/helix/controller/stages/TestRebalancePipeline.java 1a11be3
> helix-core/src/test/java/org/apache/helix/integration/TestReelectedPipelineCorrectness.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/16407/diff/
>
>
> Testing
> -------
>
> Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme cases.
>
>
> Thanks,
>
> Kanak Biscuitwala
>
>
Re: Review Request 16407: [HELIX-345] Speed up the controller pipeline for
0.6.x
Posted by Kanak Biscuitwala <ka...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16407/
-----------------------------------------------------------
(Updated Jan. 21, 2014, 3:54 p.m.)
Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
Changes
-------
Added a test to make sure that changes in live instances and ideal state are captured in the face of leader elections. Everything else (other than messages) is read every time, so it's not a concern. For messages, we always read the names from ZK, so we don't have to worry about these getting stale (message content never changes in a way that the controller cares).
Added the if (trace).
Bugs: HELIX-345
Repository: helix-git
Description
-------
commit 90e995b02a59622a0526897faaee3d6ae057b63e
Author: Kanak Biscuitwala <ka...@apache.org>
Date: Thu Dec 19 15:44:48 2013 -0800
[HELIX-345] Speed up the controller pipeline for 0.6.x
:100644 100644 5c0a94a... 5858958... M helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
:100644 100644 df893bb... 9e2452b... M helix-core/src/test/java/org/apache/helix/Mocks.java
Diffs (updated)
-----
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 3b522e6
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java 5c0a94a
helix-core/src/main/java/org/apache/helix/controller/stages/ReadClusterDataStage.java f077d29
helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java 192a645
helix-core/src/test/java/org/apache/helix/Mocks.java df893bb
helix-core/src/test/java/org/apache/helix/controller/stages/TestRebalancePipeline.java 1a11be3
helix-core/src/test/java/org/apache/helix/integration/TestReelectedPipelineCorrectness.java PRE-CREATION
Diff: https://reviews.apache.org/r/16407/diff/
Testing
-------
Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme cases.
Thanks,
Kanak Biscuitwala
Re: Review Request 16407: [HELIX-345] Speed up the controller pipeline for
0.6.x
Posted by Kanak Biscuitwala <ka...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16407/
-----------------------------------------------------------
(Updated Jan. 21, 2014, 1:13 p.m.)
Review request for helix, Zhen Zhang and Kishore Gopalakrishna.
Changes
-------
- Cached messages
- Batched current state reads
- Update ideal states, live instances, and instance configs directly from the callbacks
Bugs: HELIX-345
Repository: helix-git
Description
-------
commit 90e995b02a59622a0526897faaee3d6ae057b63e
Author: Kanak Biscuitwala <ka...@apache.org>
Date: Thu Dec 19 15:44:48 2013 -0800
[HELIX-345] Speed up the controller pipeline for 0.6.x
:100644 100644 5c0a94a... 5858958... M helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java
:100644 100644 df893bb... 9e2452b... M helix-core/src/test/java/org/apache/helix/Mocks.java
Diffs (updated)
-----
helix-core/src/main/java/org/apache/helix/controller/GenericHelixController.java 3b522e6
helix-core/src/main/java/org/apache/helix/controller/stages/ClusterDataCache.java 5c0a94a
helix-core/src/main/java/org/apache/helix/controller/stages/ReadClusterDataStage.java f077d29
helix-core/src/main/java/org/apache/helix/controller/stages/TaskAssignmentStage.java 192a645
helix-core/src/test/java/org/apache/helix/Mocks.java df893bb
helix-core/src/test/java/org/apache/helix/controller/stages/TestRebalancePipeline.java 1a11be3
Diff: https://reviews.apache.org/r/16407/diff/
Testing
-------
Tests pass locally, ReadClusterDataStage finishes 10-20% faster in extreme cases.
Thanks,
Kanak Biscuitwala