[jira] [Commented] (YARN-1564) add some basic workflow YARN services

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13862988#comment-13862988 ] 

Steve Loughran commented on YARN-1564:
--------------------------------------

Services are all in [[https://github.com/hortonworks/hoya/tree/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service]]

* [[Parent|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service/Parent.java]]: interface that makes {{addService}} public and adds an option to list services. This could be retrofitted to {{CompositeService}}
* [[CompoundService|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service/CompoundService.java] : subclass of {{CompositeService}} that finishes when all of its children have *successfully* completed, or when any of its children fail.
* [[SequenceService|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service/SequenceService.java]]  subclass of {{CompositeService}} that executes its children in sequence, starting one when the previous one successfully finishes. Again, failures are propagated up immediately.
* [[EventNotifyingService|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service/EventNotifyingService.java]] service which triggers a callback to a supplied interface implementation when started -or after a specified delay from the start time.
** [[ForkedProcessService.java|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/yarn/service/ForkedProcessService.java]]: more complex, this forks a potentially long lived application (via [[RunLongLivedApp|https://github.com/hortonworks/hoya/blob/develop/hoya-core/src/main/java/org/apache/hadoop/hoya/exec/RunLongLivedApp.java]]), completing the service when that process finishes.

The set allows you build up sequences of operations, as well as actions to run in parallel, and to notify a parent service or other object when they complete or fail.

There's tests for the simple ones (Sequence, compound, events) in groovy; nothing yet for forked processes.


> add some basic workflow YARN services
> -------------------------------------
>
>                 Key: YARN-1564
>                 URL: https://issues.apache.org/jira/browse/YARN-1564
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api
>    Affects Versions: 2.2.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> I've been using some alternative composite services to help build workflows of process execution in a YARN AM.
> They and their tests could be moved in YARN for the use by others -this would make it easier to build aggregate services in an AM



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)