Clarification on registry data model

[Supun Nakandala <su...@gmail.com>]

Hi All,

I was going to through the registry data model and I found three levels at
which input and output are defined.

1. Experiment level
       -> Exp. Input
       -> Exp. Output
2. Workflow Node Detail level
       -> Node Input
       -> Node Output
3. Task Detail
       -> App. Input
       -> App. Output

I found that every experiment (whether it is a single job or workflow) is
executed as a single task or set of tasks (in the workflow scenario). So
why is input and output defined at several levels?

Also in the project model there is a user_name field. But there is a
separate model "users" for storing user information with project_user
relation to link the two. Is there any special reason for having user_name
in project?

Thanks
Supun

Re: Clarification on registry data model

[Suresh Marru <sm...@apache.org>]

Hi Supun,

Currently the transition of data models is Experiment -> Workflow Node-> Task. This was crafted when a single application execution was considered a simplest case of the workflow. Since then, simple applications have been treated as first class within the API and the modified data model is yet to be adopted. It is at https://creately.com/diagram/hyhi571b1/xU5rAEsUHBwXnMII4ITdcjQTP4%3D <https://creately.com/diagram/hyhi571b1/xU5rAEsUHBwXnMII4ITdcjQTP4=>

So to answer your question, the data models are designed so that a parent clones data and creates a child model and the component which is executing the lowest level child does not have to know the context. So a experiment input output might belong to a high level workflow which then get decomposed into workflow level and subsequently task level inputs. This looks redundant in a simple case, but might start making sense as you assemble a complex workflow graph.

Yes the user models evolved ad-hocly and needs revisiting. Based on proposed GSoC project by Hasini, we need to properly design the user models and unify them. 

Suresh

> On Apr 17, 2015, at 6:04 PM, Supun Nakandala <su...@gmail.com> wrote:
> 
> Hi All,
> 
> I was going to through the registry data model and I found three levels at which input and output are defined. 
> 
> 1. Experiment level
>        -> Exp. Input
>        -> Exp. Output
> 2. Workflow Node Detail level
>        -> Node Input
>        -> Node Output
> 3. Task Detail
>        -> App. Input
>        -> App. Output
> 
> I found that every experiment (whether it is a single job or workflow) is executed as a single task or set of tasks (in the workflow scenario). So why is input and output defined at several levels?
> 
> Also in the project model there is a user_name field. But there is a separate model "users" for storing user information with project_user relation to link the two. Is there any special reason for having user_name in project?
> 
> Thanks
> Supun