You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@helix.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/06/19 20:57:02 UTC
[jira] [Commented] (HELIX-601) Allow work flow to schedule
dependency jobs in parallel
[ https://issues.apache.org/jira/browse/HELIX-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14593786#comment-14593786 ]
ASF GitHub Bot commented on HELIX-601:
--------------------------------------
GitHub user jicongrui opened a pull request:
https://github.com/apache/helix/pull/30
[HELIX-601] Allow work flow to schedule dependency jobs in parallel
Currently, Helix won't schedule dependency jobs in a same work flow. For example, if Job2 depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.
However, if some participant is very slow, then all dependency jobs is waiting for that single participant.
Helix should be able to schedule multiple jobs according to a parameter.
A.C.
1. Introduce parallel count parameter in work flow and job queue.
2. Dependency jobs can be scheduled according to the parameter (Now the parameter is always 1, so no parallel)
3. If Job2 depends on Job1, Job1 is scheduled before Job2.
4. No parallel jobs on the same instance. If a instance is running Job1, it won't run Job2 until Job1 is finished.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jicongrui/helix helix-0.6.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/30.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #30
----
commit 8819220738b18c54652e4b32b9677ea78d585da2
Author: Congrui Ji <cj...@linkedin.com>
Date: 2015-06-19T18:51:19Z
[HELIX-601] Allow work flow to schedule dependency jobs in parallel
Currently, Helix won't schedule dependency jobs in a same work flow. For example, if Job2 depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.
However, if some participant is very slow, then all dependency jobs is waiting for that single participant.
Helix should be able to schedule multiple jobs according to a parameter.
A.C.
1. Introduce parallel count parameter in work flow and job queue.
2. Dependency jobs can be scheduled according to the parameter (Now the parameter is always 1, so no parallel)
3. If Job2 depends on Job1, Job1 is scheduled before Job2.
4. No parallel jobs on the same instance. If a instance is running Job1, it won't run Job2 until Job1 is finished.
----
> Allow work flow to schedule dependency jobs in parallel
> -------------------------------------------------------
>
> Key: HELIX-601
> URL: https://issues.apache.org/jira/browse/HELIX-601
> Project: Apache Helix
> Issue Type: New Feature
> Reporter: Congrui Ji
>
> Currently, Helix won't schedule dependency jobs in a same work flow. For example, if Job2 depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.
> However, if some participant is very slow, then all dependency jobs is waiting for that single participant.
> Helix should be able to schedule multiple jobs according to a parameter.
> A.C.
> 1. Introduce parallel count parameter in work flow and job queue.
> 2. Dependency jobs can be scheduled according to the parameter (Now the parameter is always 1, so no parallel)
> 3. If Job2 depends on Job1, Job1 is scheduled before Job2.
> 4. No parallel jobs on the same instance. If a instance is running Job1, it won't run Job2 until Job1 is finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)