You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Thomas Weise <th...@datatorrent.com> on 2015/09/02 23:53:36 UTC
Re: Modules support in Apex
Vlad,
Can you send a pointer to the fork you are working on (or mention it on the
JIRA)?
Thanks
On Mon, Aug 31, 2015 at 2:17 PM, Vlad Rozov <v....@datatorrent.com> wrote:
> Atri,
>
> As a first cut module is a predefined subgraph that can be inserted into a
> DAG. Generally speaking module is not required to expand to the same
> subgraph. Depending on module properties or configuration, module may
> expand to different subgraphs. At the same time, similar to an operator
> module has predefined input/output ports and properties.
>
> Thank you,
>
> Vlad
>
>
> On 8/31/15 11:46, Atri Sharma wrote:
>
>> No, I dont think APEX-3's functionality needs to be exceeded for this.
>> What
>> I am trying to understand here is the concept of Module. Is it a family of
>> operators defined by same interface implementation *or* is it a defined
>> subgraph that can be replaced and used as a part of a subgraph instead of
>> building the whole connection again?
>>
>> On Mon, Aug 31, 2015 at 11:27 PM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>>
>> Yes, you are correct. The APEX-3 is for dynamic expansion of the DAG, If
>>> we
>>> can
>>> expand the DAG dynamically then we can use that functionality to expand
>>> the
>>> DAG
>>> with know list of operator and connections between them. In a way APEX-3
>>> provides
>>> bigger functionality than APEX-55.
>>>
>>> Let me know if you think that module functionality requires more support
>>> that what will be provided by APEX-3.
>>>
>>>
>>>
>>> On Mon, Aug 31, 2015 at 10:39 AM, Atri Sharma <at...@apache.org> wrote:
>>>
>>> Looks interesting.
>>>>
>>>> Can you explain a bit on how it helps APEX-3? The objective of APEX-3 is
>>>>
>>> to
>>>
>>>> have dynamic expansion of sub DAGs. If I understand correctly, the
>>>> static
>>>> list is to be known when declaring a module.
>>>>
>>>> Please correct me if I am wrong.
>>>>
>>>> On Mon, Aug 31, 2015 at 11:05 PM, Tushar Gosavi <tushar@datatorrent.com
>>>> >
>>>> wrote:
>>>>
>>>> Hi All,
>>>>>
>>>>> We are working on adding support for Modules in Apex. A module is a
>>>>>
>>>> group
>>>
>>>> of operators that will have their own existence and will ease the way
>>>>>
>>>> we
>>>
>>>> are currently defining an application.
>>>>>
>>>>> A module is defined by:
>>>>>
>>>>> - list of operators.
>>>>> - list of input and output ports.
>>>>> - set of properties for the module.
>>>>> - set of attributes for the module.
>>>>>
>>>>>
>>>>> Details on the proposed work is given at:
>>>>>
>>>>> https://malhar.atlassian.net/browse/APEX-55 . The work is also related
>>>>>
>>>> to
>>>>
>>>>> https://malhar.atlassian.net/browse/APEX-3
>>>>>
>>>>> Regards,
>>>>> -Tushar.
>>>>>
>>>>>
>
Re: Modules support in Apex
Posted by Vlad Rozov <v....@datatorrent.com>.
Hi Thomas,
I'll update JIRA after I rebase my work into the apache apex fork.
Thank you,
Vlad
On 9/2/15 14:53, Thomas Weise wrote:
> Vlad,
>
> Can you send a pointer to the fork you are working on (or mention it on the
> JIRA)?
>
> Thanks
>
> On Mon, Aug 31, 2015 at 2:17 PM, Vlad Rozov <v....@datatorrent.com> wrote:
>
>> Atri,
>>
>> As a first cut module is a predefined subgraph that can be inserted into a
>> DAG. Generally speaking module is not required to expand to the same
>> subgraph. Depending on module properties or configuration, module may
>> expand to different subgraphs. At the same time, similar to an operator
>> module has predefined input/output ports and properties.
>>
>> Thank you,
>>
>> Vlad
>>
>>
>> On 8/31/15 11:46, Atri Sharma wrote:
>>
>>> No, I dont think APEX-3's functionality needs to be exceeded for this.
>>> What
>>> I am trying to understand here is the concept of Module. Is it a family of
>>> operators defined by same interface implementation *or* is it a defined
>>> subgraph that can be replaced and used as a part of a subgraph instead of
>>> building the whole connection again?
>>>
>>> On Mon, Aug 31, 2015 at 11:27 PM, Tushar Gosavi <tu...@datatorrent.com>
>>> wrote:
>>>
>>> Yes, you are correct. The APEX-3 is for dynamic expansion of the DAG, If
>>>> we
>>>> can
>>>> expand the DAG dynamically then we can use that functionality to expand
>>>> the
>>>> DAG
>>>> with know list of operator and connections between them. In a way APEX-3
>>>> provides
>>>> bigger functionality than APEX-55.
>>>>
>>>> Let me know if you think that module functionality requires more support
>>>> that what will be provided by APEX-3.
>>>>
>>>>
>>>>
>>>> On Mon, Aug 31, 2015 at 10:39 AM, Atri Sharma <at...@apache.org> wrote:
>>>>
>>>> Looks interesting.
>>>>> Can you explain a bit on how it helps APEX-3? The objective of APEX-3 is
>>>>>
>>>> to
>>>>
>>>>> have dynamic expansion of sub DAGs. If I understand correctly, the
>>>>> static
>>>>> list is to be known when declaring a module.
>>>>>
>>>>> Please correct me if I am wrong.
>>>>>
>>>>> On Mon, Aug 31, 2015 at 11:05 PM, Tushar Gosavi <tushar@datatorrent.com
>>>>> wrote:
>>>>>
>>>>> Hi All,
>>>>>> We are working on adding support for Modules in Apex. A module is a
>>>>>>
>>>>> group
>>>>> of operators that will have their own existence and will ease the way
>>>>> we
>>>>> are currently defining an application.
>>>>>> A module is defined by:
>>>>>>
>>>>>> - list of operators.
>>>>>> - list of input and output ports.
>>>>>> - set of properties for the module.
>>>>>> - set of attributes for the module.
>>>>>>
>>>>>>
>>>>>> Details on the proposed work is given at:
>>>>>>
>>>>>> https://malhar.atlassian.net/browse/APEX-55 . The work is also related
>>>>>>
>>>>> to
>>>>>
>>>>>> https://malhar.atlassian.net/browse/APEX-3
>>>>>>
>>>>>> Regards,
>>>>>> -Tushar.
>>>>>>
>>>>>>