You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@helix.apache.org by lei-xia <gi...@git.apache.org> on 2016/08/17 01:43:34 UTC

[GitHub] helix pull request #46: All task framework changes

GitHub user lei-xia opened a pull request:

    https://github.com/apache/helix/pull/46

    All task framework changes

    This change contains all pending commits we have already committed and running within LinkedIn internally. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lei-xia/helix helix-0.6.x

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/helix/pull/46.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #46
    
----
commit 760f8e3caca39aeb53a171303ec24334cd59bf47
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-01-20T02:15:07Z

    Fix task framework unit tests.

commit 03b90012939e99eb3556c715ff8b8eaab710bb79
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-01-20T23:33:20Z

    More fixes and cleanup on task unit tests.

commit 37819298939b61d8f7a589a3404fd3414fbdbd06
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-01-28T02:04:23Z

    Fix the statemodelFactories in Example Process.

commit 7a47070362c536126e945138f10fb077215f706e
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-01-26T17:57:43Z

    Support changes of workflow configuration.

commit d213c1a7b3fc310d38749f9555e5660858c37c3b
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-02-12T23:12:06Z

    Add methods in TaskDriver for getting Workflow/Job configuration and context. External users should call these methods instead of TaskUtil.

commit 6d42db462bc65ed9f94f22ca6e2de83cb703ea87
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-02-23T18:16:03Z

    TaskUtil.getWorkflowCfg throws NPE if workflow doesn't exist.

commit 579d82fd2aa8fdce8ec0e0c4d6da73cb8209729d
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-02-24T01:32:35Z

    Refactor TaskUtil class to move as many as methods out of the class, and make other methods in it as internal API as possible. Expose necessary APIs in TaskDriver instead.

commit be660245fc1a9f4b22fba58c4b25a1af19555066
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-01-27T18:10:31Z

    Add new job option to allow contining a job even its direct dependent job fails.

commit 1dad0b805338a8dae72e0661ac1a803b30808623
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-07T23:15:10Z

    Fix task framework unit test failure.

commit 66dba1f5df1d2d72a40aabfa6c841152cc068a70
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-08T02:10:34Z

    Add old task rebalancers back for back-compatible and rolling upgrade.

commit d386aff394f2e4e7202f13fe2ed5e6533a8cfb29
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-02-24T01:06:35Z

    Refactor Workflow and Jobqueue builders to make the builder API more clean.

commit aeb6f3ec7ab973316fd3468e9cbb0052a6a4306e
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-21T20:46:01Z

    Add static methods into TaskDriver for getting configuration/context for jobs and workflows.

commit 57bfc4a211b2f3454c8f00fa3f26262cd1d3922b
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-22T16:08:09Z

    Fix a task unit test.

commit b6b89de5cf00c1d1d1cba2cd09fcd25054b2247d
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-30T17:08:42Z

    Add Builder class for TaskConfig, and add unit test for testing generic jobs.

commit 1f683b863df23f16bd893fc675f88ed8b7f3d3b8
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-03-30T20:59:59Z

    More cleanup on workflow and workflowConfig builders.

commit ead83012ee07610859fabdc73712cf4532473e88
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-08T20:40:04Z

    Add TaskTestBase and refactor 2 tests
    1. Add TaskTestBase for common initialization.
    2. Refactor 2 tests to test the test base.

commit 9f802064fbc782fd6820d551bb1cf39ba4c53108
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-04-08T17:52:23Z

    Added method to TaskDriver to get all workflows from a cluster. Added methods to convert HelixProperty to WorkflowConfig and JobConfig.

commit 99a40083ab7bc1e1480d66107fff83f0479fa068
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-12T18:48:54Z

    Refactor tests with TaskTestBase and remove duplicated code.

commit be78a4f14076736cca511b1d3116546162e026fc
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-21T20:04:22Z

    Add WorkflowType and JobType in WorkflowConfig and JobConfig
    
    Add WorkflowType and JobType in WorkflowConfig and JobConfig
    
    RB=710119
    G=nuage-reviewers
    R=lxia,cji
    A=lxia

commit ddc11f9aca9e3e525d517860006e1d5cbb91ddd4
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-20T23:33:50Z

    Add job dependency workflow test
    1. Add an option to enable number of partion varies in TaskTestBase
    2. Add new test for workflow test for parallel jobs testing.

commit 7f184839aebcb817e3c9b0626bcc2a9b44c6926c
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-07-05T22:58:51Z

    Add a new task state (TASK_ABORTED) to TaskResult. This allows client to abort a task and let Helix not retry it even Task.RetryCount is bigger than 1.

commit 4f7fe1306431210ee681a9eae64687f788f46bb3
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-04-19T20:40:29Z

    Add unit test to retrieve workflow and job configurations.

commit 2efa4485145ddfc8533c66c3e67077afe3a1ae9c
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-04-21T17:58:49Z

    Add unit tests to retrieve all workflows and job info from a cluster.

commit ddefebb245f4f43c8c64a9f8cc1a2536ae25eab4
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-28T18:36:20Z

    Add Partition task start time
    
    Add setup and get start time for partition tasks.
    
    RB=715086
    G=nuage-reviewers
    R=lxia,cji
    A=lxia

commit e8ecdf389daf5976b726e132d05b343986eae5b2
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-27T23:22:43Z

    Add test to test task retry with and without delay.

commit 2f62cd1fc60eab43a2250224576b28742f023a73
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-04-28T22:24:03Z

    Refactor redundant code TestTaskRebalancerRetryLimit
    
    RB=715304
    G=nuage-reviewers
    R=lxia,cji
    A=lxia

commit 7e9041f0d83a920f585300f8082bf4233d10f983
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-05-02T18:11:11Z

    Fix missing workflowtype assign in builder

commit c3624e082b4025945f1d7755c73ce999bcac6783
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-05-05T22:51:58Z

    Check whether instance is disable when assigning tasks to instances in TaskRebalancer.

commit 9508a1acfae1d915148138daccb2abd5f9dce430
Author: Lei Xia <lx...@linkedin.com>
Date:   2016-05-05T18:25:22Z

    Populate Error message from running client's task and persist it into JobContext for better error reporting.

commit 9a30df46bc054514a708b5070f166ee5aa2e7cd1
Author: Junkai Xue <jx...@linkedin.com>
Date:   2016-05-06T19:05:30Z

    Fix task assignment in instance group tag check
    
    1. Fixed task assignment with instacnce group. When target state is not set, it ont check the state of instance.
    2. Add a new test to assign task. The instance will be assign to has been disabled. So the task should be hanging there.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] helix issue #46: All task framework changes

Posted by kishoreg <gi...@git.apache.org>.

Github user kishoreg commented on the issue:

    https://github.com/apache/helix/pull/46
  
    Lgtm.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] helix pull request #46: All task framework changes

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/helix/pull/46


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---