You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@helix.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/09/13 23:06:22 UTC
[jira] [Commented] (HELIX-635) GenericTaskAssignmentCalculator does
not balance task assignment
[ https://issues.apache.org/jira/browse/HELIX-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488751#comment-15488751 ]
ASF GitHub Bot commented on HELIX-635:
--------------------------------------
GitHub user dasahcc opened a pull request:
https://github.com/apache/helix/pull/51
[HELIX-635] GenericTaskAssignmentCalculator does not balance task assignment
Currently, GenericTaskAssignmentCalculator using AutoRebalancerStrategy compute the assignment mapping. There is a bug that all the tasks from same job will be assigned to one node, which is not balanced.
This fix contains:
1. Implement consistent hashing mapping calculation for GenericTaskAssignmentCalculator
2. Remove reassign logics and applied in consistent hashing
3. Add tests for GenericTaskAssignmentCalculator
4. Refactoring TaskAssignmentCalculator API, since ClusterDataCache is too large and not all the contents inside are used.
5. Support identical task initialization with job command and number of tasks
6. Remove unused MaxForcedReassignmentPerTask field
7. Refactor logics of failure.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dasahcc/helix helix-0.6.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/51.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #51
----
commit 0a18726fcad7b8a0fe5e77d7a2c9848b86461ccc
Author: Junkai Xue <jx...@linkedin.com>
Date: 2016-09-13T22:28:57Z
[HELIX-635] GenericTaskAssignmentCalculator rebalance with consistent hashing
1. Implement consistent hashing mapping calculation
2. Remove reassign logics and applied in consistent hashing
3. Add tests for GenericTaskAssignmentCalculator
commit 7bb2a9db2396a00bb9a721634a2432240679c657
Author: Junkai Xue <jx...@linkedin.com>
Date: 2016-09-13T23:00:08Z
Refactor TaskAssignmentCalculator API
Refactoring TaskAssignmentCalculator API, since ClusterDataCache is too large and not all the contents inside are used.
commit 27cb5ecfe5f3b07c5c97534222558702a05e5aae
Author: Junkai Xue <jx...@linkedin.com>
Date: 2016-09-13T23:01:39Z
Job Config and logic refactoring
1. Support identical task initialization with job command and number of tasks
2. Remove unused MaxForcedReassignmentPerTask field
3. Refactor logics of failure.
----
> GenericTaskAssignmentCalculator does not balance task assignment
> ----------------------------------------------------------------
>
> Key: HELIX-635
> URL: https://issues.apache.org/jira/browse/HELIX-635
> Project: Apache Helix
> Issue Type: Bug
> Components: helix-core
> Reporter: Junkai Xue
> Assignee: Junkai Xue
> Fix For: 0.6.x
>
>
> Currently, GenericTaskAssignmentCalculator using AutoRebalancerStrategy compute the assignment mapping. There is a bug that all the tasks from same job will be assigned to one node, which is not balanced.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)