You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Junping Du (JIRA)" <ji...@apache.org> on 2015/04/01 17:45:54 UTC

[jira] [Commented] (YARN-3391) Clearly define flow ID/ flow run / flow version in API and storage

    [ https://issues.apache.org/jira/browse/YARN-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390844#comment-14390844 ] 

Junping Du commented on YARN-3391:
----------------------------------

Thanks [~zjshen] for delivering the patch! 
To be honest, I am getting more confused on these concepts from some discussion above:
>From what I was understanding, flow is a group of applications that will get run (sequential or parallel) in a batch, and flow_run is one run branch for subset of flow applications (apps in flow_run only get run in sequence, however, different flow_runs under one flow could run in parallel). Does "flow version" sounds like a timestamp concept (from HBase prospective) which represent a specific run time for the flow?
Just quickly go through the attached patch, I didn't find answer there. I think we should document the concept/definition of "flow", "flow run" and "flow version" clearly in Javadoc (web doc could be later when we finish the feature) which could help reviewer and developers to understand better. 

> Clearly define flow ID/ flow run / flow version in API and storage
> ------------------------------------------------------------------
>
>                 Key: YARN-3391
>                 URL: https://issues.apache.org/jira/browse/YARN-3391
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-3391.1.patch
>
>
> To continue the discussion in YARN-3040, let's figure out the best way to describe the flow.
> Some key issues that we need to conclude on:
> - How do we include the flow version in the context so that it gets passed into the collector and to the storage eventually?
> - Flow run id should be a number as opposed to a generic string?
> - Default behavior for the flow run id if it is missing (i.e. client did not set it)
> - How do we handle flow attributes in case of nested levels of flows?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)