You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Tushar Gosavi <tu...@datatorrent.com> on 2015/12/01 05:08:31 UTC
[APEX-3/APEX-104] Module Separator.
Hi All,
As part of module support, we will need to generate names for operators
internal to the module before adding it to the original DAG to avoid
conflict of names. For example if a developer adds a operator with name "A"
and module is name "M" in the DAG. Developer of 'M' is adding a operator
'A' into the module DAG. To avoid name conflict
with the operator in the main DAG we will generate name for internal
operators by concatenation of module name and operator name. The 'A' within
module 'M' will be added to main DAG as 'M_A'. This works for recursive
modules too. (i.e module containing module).
Which separator should we use for module namespace? The requirement are
- Less chances of collision. (User can add operator with name 'M_A' too).
- Compatible with json format. (Should not have a problem while parsing
json response from rest api)
- Should be able to specify property in dt-site.xml.
While '_' works, In the pull request it has been pointed out that it has
higher chance of collision. Few choices are
1) _ (Underscore)
2) __ (double underscore)
3) : (colon, I will check compatibility with rest API, as it is used to
separate key and value in json)
4) :: (Double colon).
5) Let developer choose the separator by specifying dag property, but
default is still needed.
6) Any other option?
Please suggest which approach to take?
Regards,
-Tushar.
Re: [APEX-3/APEX-104] Module Separator.
Posted by Vlad Rozov <v....@datatorrent.com>.
+1 for "/". It may help with REST API (no need to worry about "/" in the
operator/module name).
Thank you,
Vlad
On 12/1/15 08:03, Amol Kekre wrote:
> This char will also be in rest API from stram. So we will need to look at URL sensitivity.
>
> Amol
>
> Sent from my iPhone
>
>> On Dec 1, 2015, at 2:11 AM, "Ganelin, Ilya" <Il...@capitalone.com> wrote:
>>
>> Regardless of which symbol we select, we should add an explicit run-time check to guard against collisions.
>>
>>
>>
>> Thank you,
>> Ilya Ganelin
>>
>>
>>
>> -----Original Message-----
>> From: Priyanka Gugale [priyanka@datatorrent.com<ma...@datatorrent.com>]
>> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
>> To: dev@apex.incubator.apache.org
>> Subject: Re: [APEX-3/APEX-104] Module Separator.
>>
>>
>> +1 for $.
>>
>> -Priyanka
>>
>> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
>> wrote:
>>
>>> I will also consider tilde as a separator. Module1~Operator1.
>>>
>>> Less conflict and more readable.
>>>
>>> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
>>> wrote:
>>>
>>>> +1 for $ or (::) double colon
>>>>
>>>> Thanks,
>>>> Shubham
>>>>
>>>> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>>> chinmay@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I want to propose 2 more options for this:
>>>>> 1) $ (Dollar delimited)
>>>>> Dollar sign is usually to indicate inner classes in Java. Considering
>>>> that
>>>>> I believe this can work in REST APIs and CLI as well.
>>>>>
>>>>> Hence the operator inside module can look like:
>>>>> Module1*$*Module2*$*Operator1
>>>>>
>>>>> 2) | (Pipe Delimited)
>>>>> Operator name would look like:
>>>>> Module1*|*Module2*|*Operator1
>>>>>
>>>>> 3) / (Unix File Seperator)
>>>>> This seperate is common and used in most of the places. file seperator
>>> is
>>>>> also used to indicate the content inside something.
>>>>> Operator name would look like:
>>>>> Module1*/*Module2*/*Operator1
>>>>>
>>>>> If there are technical challenges in above, I would vote for (::)
>>> double
>>>>> colon.
>>>>>
>>>>> -Chinmay.
>>>>>
>>>>>
>>>>> ~ Chinmay.
>>>>>
>>>>> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
>>>>> wrote:
>>>>>
>>>>>> dot can not be used in dt-site.xml, as '.' is used as separator for
>>>>> parsing
>>>>>> various component from configuration file.
>>>>>>
>>>>>> - Tushar.
>>>>>>
>>>>>> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>>> yogidevendra@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> I would prefer '.' (dot).
>>>>>>> If there are some technical challenges for dot then I vote for :
>>>>> (colon)
>>>>>> or
>>>>>>> :: (double colon) in that sequence.
>>>>>>>
>>>>>>> ~ Yogi
>>>>>>>
>>>>>>>> On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> What about using “#”? We use this for Unifiers.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> - Gaurav
>>>>>>>>
>>>>>>>>> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>>> tushar@datatorrent.com
>>>>>>>> wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> As part of module support, we will need to generate names for
>>>>>> operators
>>>>>>>>> internal to the module before adding it to the original DAG to
>>>>> avoid
>>>>>>>>> conflict of names. For example if a developer adds a operator
>>>> with
>>>>>> name
>>>>>>>> "A"
>>>>>>>>> and module is name "M" in the DAG. Developer of 'M' is adding a
>>>>>>> operator
>>>>>>>>> 'A' into the module DAG. To avoid name conflict
>>>>>>>>> with the operator in the main DAG we will generate name for
>>>>> internal
>>>>>>>>> operators by concatenation of module name and operator name.
>>> The
>>>>> 'A'
>>>>>>>> within
>>>>>>>>> module 'M' will be added to main DAG as 'M_A'. This works for
>>>>>>> recursive
>>>>>>>>> modules too. (i.e module containing module).
>>>>>>>>>
>>>>>>>>> Which separator should we use for module namespace? The
>>>> requirement
>>>>>> are
>>>>>>>>> - Less chances of collision. (User can add operator with name
>>>> 'M_A'
>>>>>>> too).
>>>>>>>>> - Compatible with json format. (Should not have a problem while
>>>>>> parsing
>>>>>>>>> json response from rest api)
>>>>>>>>> - Should be able to specify property in dt-site.xml.
>>>>>>>>>
>>>>>>>>> While '_' works, In the pull request it has been pointed out
>>> that
>>>>> it
>>>>>>> has
>>>>>>>>> higher chance of collision. Few choices are
>>>>>>>>>
>>>>>>>>> 1) _ (Underscore)
>>>>>>>>> 2) __ (double underscore)
>>>>>>>>> 3) : (colon, I will check compatibility with rest API, as it is
>>>>> used
>>>>>> to
>>>>>>>>> separate key and value in json)
>>>>>>>>> 4) :: (Double colon).
>>>>>>>>> 5) Let developer choose the separator by specifying dag
>>> property,
>>>>> but
>>>>>>>>> default is still needed.
>>>>>>>>> 6) Any other option?
>>>>>>>>>
>>>>>>>>> Please suggest which approach to take?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> -Tushar.
>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: [APEX-3/APEX-104] Module Separator.
Posted by Amol Kekre <am...@datatorrent.com>.
This char will also be in rest API from stram. So we will need to look at URL sensitivity.
Amol
Sent from my iPhone
> On Dec 1, 2015, at 2:11 AM, "Ganelin, Ilya" <Il...@capitalone.com> wrote:
>
> Regardless of which symbol we select, we should add an explicit run-time check to guard against collisions.
>
>
>
> Thank you,
> Ilya Ganelin
>
>
>
> -----Original Message-----
> From: Priyanka Gugale [priyanka@datatorrent.com<ma...@datatorrent.com>]
> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
> To: dev@apex.incubator.apache.org
> Subject: Re: [APEX-3/APEX-104] Module Separator.
>
>
> +1 for $.
>
> -Priyanka
>
> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
> wrote:
>
>> I will also consider tilde as a separator. Module1~Operator1.
>>
>> Less conflict and more readable.
>>
>> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
>> wrote:
>>
>>> +1 for $ or (::) double colon
>>>
>>> Thanks,
>>> Shubham
>>>
>>> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>> chinmay@datatorrent.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to propose 2 more options for this:
>>>> 1) $ (Dollar delimited)
>>>> Dollar sign is usually to indicate inner classes in Java. Considering
>>> that
>>>> I believe this can work in REST APIs and CLI as well.
>>>>
>>>> Hence the operator inside module can look like:
>>>> Module1*$*Module2*$*Operator1
>>>>
>>>> 2) | (Pipe Delimited)
>>>> Operator name would look like:
>>>> Module1*|*Module2*|*Operator1
>>>>
>>>> 3) / (Unix File Seperator)
>>>> This seperate is common and used in most of the places. file seperator
>> is
>>>> also used to indicate the content inside something.
>>>> Operator name would look like:
>>>> Module1*/*Module2*/*Operator1
>>>>
>>>> If there are technical challenges in above, I would vote for (::)
>> double
>>>> colon.
>>>>
>>>> -Chinmay.
>>>>
>>>>
>>>> ~ Chinmay.
>>>>
>>>> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
>>>
>>>> wrote:
>>>>
>>>>> dot can not be used in dt-site.xml, as '.' is used as separator for
>>>> parsing
>>>>> various component from configuration file.
>>>>>
>>>>> - Tushar.
>>>>>
>>>>> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>> yogidevendra@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I would prefer '.' (dot).
>>>>>> If there are some technical challenges for dot then I vote for :
>>>> (colon)
>>>>> or
>>>>>> :: (double colon) in that sequence.
>>>>>>
>>>>>> ~ Yogi
>>>>>>
>>>>>>> On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> What about using “#”? We use this for Unifiers.
>>>>>>>
>>>>>>> Thanks
>>>>>>> - Gaurav
>>>>>>>
>>>>>>>> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>> tushar@datatorrent.com
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> As part of module support, we will need to generate names for
>>>>> operators
>>>>>>>> internal to the module before adding it to the original DAG to
>>>> avoid
>>>>>>>> conflict of names. For example if a developer adds a operator
>>> with
>>>>> name
>>>>>>> "A"
>>>>>>>> and module is name "M" in the DAG. Developer of 'M' is adding a
>>>>>> operator
>>>>>>>> 'A' into the module DAG. To avoid name conflict
>>>>>>>> with the operator in the main DAG we will generate name for
>>>> internal
>>>>>>>> operators by concatenation of module name and operator name.
>> The
>>>> 'A'
>>>>>>> within
>>>>>>>> module 'M' will be added to main DAG as 'M_A'. This works for
>>>>>> recursive
>>>>>>>> modules too. (i.e module containing module).
>>>>>>>>
>>>>>>>> Which separator should we use for module namespace? The
>>> requirement
>>>>> are
>>>>>>>> - Less chances of collision. (User can add operator with name
>>> 'M_A'
>>>>>> too).
>>>>>>>> - Compatible with json format. (Should not have a problem while
>>>>> parsing
>>>>>>>> json response from rest api)
>>>>>>>> - Should be able to specify property in dt-site.xml.
>>>>>>>>
>>>>>>>> While '_' works, In the pull request it has been pointed out
>> that
>>>> it
>>>>>> has
>>>>>>>> higher chance of collision. Few choices are
>>>>>>>>
>>>>>>>> 1) _ (Underscore)
>>>>>>>> 2) __ (double underscore)
>>>>>>>> 3) : (colon, I will check compatibility with rest API, as it is
>>>> used
>>>>> to
>>>>>>>> separate key and value in json)
>>>>>>>> 4) :: (Double colon).
>>>>>>>> 5) Let developer choose the separator by specifying dag
>> property,
>>>> but
>>>>>>>> default is still needed.
>>>>>>>> 6) Any other option?
>>>>>>>>
>>>>>>>> Please suggest which approach to take?
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> -Tushar.
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: [APEX-3/APEX-104] Module Separator.
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
+1 for "/", that makes it easy for hierarchical representation like unix
directory structure.
Regards,
Sandeep
On Wed, Dec 2, 2015 at 12:24 AM, Vlad Rozov <v....@datatorrent.com> wrote:
> I'd suggest to deprecate the symbol unless there is already one that is
> not allowed (afaik there is no explicit check to disallow any specific
> character). I mean that usage of a special character in an application that
> uses Module API is an error, but if Module API is not used, it is a warning
> for couple of releases.
>
> Thank you,
>
> Vlad
>
>
> On 12/1/15 02:11, Ganelin, Ilya wrote:
>
>> Regardless of which symbol we select, we should add an explicit run-time
>> check to guard against collisions.
>>
>>
>>
>> Thank you,
>> Ilya Ganelin
>>
>>
>>
>> -----Original Message-----
>> From: Priyanka Gugale [priyanka@datatorrent.com<mailto:
>> priyanka@datatorrent.com>]
>> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
>> To: dev@apex.incubator.apache.org
>> Subject: Re: [APEX-3/APEX-104] Module Separator.
>>
>>
>> +1 for $.
>>
>> -Priyanka
>>
>> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
>> wrote:
>>
>> I will also consider tilde as a separator. Module1~Operator1.
>>>
>>> Less conflict and more readable.
>>>
>>> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <shubham@datatorrent.com
>>> >
>>> wrote:
>>>
>>> +1 for $ or (::) double colon
>>>>
>>>> Thanks,
>>>> Shubham
>>>>
>>>> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>>> chinmay@datatorrent.com>
>>>> wrote:
>>>>
>>>> Hi,
>>>>>
>>>>> I want to propose 2 more options for this:
>>>>> 1) $ (Dollar delimited)
>>>>> Dollar sign is usually to indicate inner classes in Java. Considering
>>>>>
>>>> that
>>>>
>>>>> I believe this can work in REST APIs and CLI as well.
>>>>>
>>>>> Hence the operator inside module can look like:
>>>>> Module1*$*Module2*$*Operator1
>>>>>
>>>>> 2) | (Pipe Delimited)
>>>>> Operator name would look like:
>>>>> Module1*|*Module2*|*Operator1
>>>>>
>>>>> 3) / (Unix File Seperator)
>>>>> This seperate is common and used in most of the places. file seperator
>>>>>
>>>> is
>>>
>>>> also used to indicate the content inside something.
>>>>> Operator name would look like:
>>>>> Module1*/*Module2*/*Operator1
>>>>>
>>>>> If there are technical challenges in above, I would vote for (::)
>>>>>
>>>> double
>>>
>>>> colon.
>>>>>
>>>>> -Chinmay.
>>>>>
>>>>>
>>>>> ~ Chinmay.
>>>>>
>>>>> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
>>>>> wrote:
>>>>>
>>>>> dot can not be used in dt-site.xml, as '.' is used as separator for
>>>>>>
>>>>> parsing
>>>>>
>>>>>> various component from configuration file.
>>>>>>
>>>>>> - Tushar.
>>>>>>
>>>>>> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>>>>>
>>>>> yogidevendra@apache.org>
>>>>
>>>>> wrote:
>>>>>>
>>>>>> I would prefer '.' (dot).
>>>>>>> If there are some technical challenges for dot then I vote for :
>>>>>>>
>>>>>> (colon)
>>>>>
>>>>>> or
>>>>>>
>>>>>>> :: (double colon) in that sequence.
>>>>>>>
>>>>>>> ~ Yogi
>>>>>>>
>>>>>>> On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> What about using “#”? We use this for Unifiers.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> - Gaurav
>>>>>>>>
>>>>>>>> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>>>>>>>>
>>>>>>>> tushar@datatorrent.com
>>>>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> As part of module support, we will need to generate names for
>>>>>>>>>
>>>>>>>> operators
>>>>>>
>>>>>>> internal to the module before adding it to the original DAG to
>>>>>>>>>
>>>>>>>> avoid
>>>>>
>>>>>> conflict of names. For example if a developer adds a operator
>>>>>>>>>
>>>>>>>> with
>>>>
>>>>> name
>>>>>>
>>>>>>> "A"
>>>>>>>>
>>>>>>>>> and module is name "M" in the DAG. Developer of 'M' is adding a
>>>>>>>>>
>>>>>>>> operator
>>>>>>>
>>>>>>>> 'A' into the module DAG. To avoid name conflict
>>>>>>>>> with the operator in the main DAG we will generate name for
>>>>>>>>>
>>>>>>>> internal
>>>>>
>>>>>> operators by concatenation of module name and operator name.
>>>>>>>>>
>>>>>>>> The
>>>
>>>> 'A'
>>>>>
>>>>>> within
>>>>>>>>
>>>>>>>>> module 'M' will be added to main DAG as 'M_A'. This works for
>>>>>>>>>
>>>>>>>> recursive
>>>>>>>
>>>>>>>> modules too. (i.e module containing module).
>>>>>>>>>
>>>>>>>>> Which separator should we use for module namespace? The
>>>>>>>>>
>>>>>>>> requirement
>>>>
>>>>> are
>>>>>>
>>>>>>> - Less chances of collision. (User can add operator with name
>>>>>>>>>
>>>>>>>> 'M_A'
>>>>
>>>>> too).
>>>>>>>
>>>>>>>> - Compatible with json format. (Should not have a problem while
>>>>>>>>>
>>>>>>>> parsing
>>>>>>
>>>>>>> json response from rest api)
>>>>>>>>> - Should be able to specify property in dt-site.xml.
>>>>>>>>>
>>>>>>>>> While '_' works, In the pull request it has been pointed out
>>>>>>>>>
>>>>>>>> that
>>>
>>>> it
>>>>>
>>>>>> has
>>>>>>>
>>>>>>>> higher chance of collision. Few choices are
>>>>>>>>>
>>>>>>>>> 1) _ (Underscore)
>>>>>>>>> 2) __ (double underscore)
>>>>>>>>> 3) : (colon, I will check compatibility with rest API, as it is
>>>>>>>>>
>>>>>>>> used
>>>>>
>>>>>> to
>>>>>>
>>>>>>> separate key and value in json)
>>>>>>>>> 4) :: (Double colon).
>>>>>>>>> 5) Let developer choose the separator by specifying dag
>>>>>>>>>
>>>>>>>> property,
>>>
>>>> but
>>>>>
>>>>>> default is still needed.
>>>>>>>>> 6) Any other option?
>>>>>>>>>
>>>>>>>>> Please suggest which approach to take?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> -Tushar.
>>>>>>>>>
>>>>>>>>
>>>>>>>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>>
>
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Vlad Rozov <v....@datatorrent.com>.
I'd suggest to deprecate the symbol unless there is already one that is
not allowed (afaik there is no explicit check to disallow any specific
character). I mean that usage of a special character in an application
that uses Module API is an error, but if Module API is not used, it is a
warning for couple of releases.
Thank you,
Vlad
On 12/1/15 02:11, Ganelin, Ilya wrote:
> Regardless of which symbol we select, we should add an explicit run-time check to guard against collisions.
>
>
>
> Thank you,
> Ilya Ganelin
>
>
>
> -----Original Message-----
> From: Priyanka Gugale [priyanka@datatorrent.com<ma...@datatorrent.com>]
> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
> To: dev@apex.incubator.apache.org
> Subject: Re: [APEX-3/APEX-104] Module Separator.
>
>
> +1 for $.
>
> -Priyanka
>
> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
> wrote:
>
>> I will also consider tilde as a separator. Module1~Operator1.
>>
>> Less conflict and more readable.
>>
>> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
>> wrote:
>>
>>> +1 for $ or (::) double colon
>>>
>>> Thanks,
>>> Shubham
>>>
>>> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>> chinmay@datatorrent.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to propose 2 more options for this:
>>>> 1) $ (Dollar delimited)
>>>> Dollar sign is usually to indicate inner classes in Java. Considering
>>> that
>>>> I believe this can work in REST APIs and CLI as well.
>>>>
>>>> Hence the operator inside module can look like:
>>>> Module1*$*Module2*$*Operator1
>>>>
>>>> 2) | (Pipe Delimited)
>>>> Operator name would look like:
>>>> Module1*|*Module2*|*Operator1
>>>>
>>>> 3) / (Unix File Seperator)
>>>> This seperate is common and used in most of the places. file seperator
>> is
>>>> also used to indicate the content inside something.
>>>> Operator name would look like:
>>>> Module1*/*Module2*/*Operator1
>>>>
>>>> If there are technical challenges in above, I would vote for (::)
>> double
>>>> colon.
>>>>
>>>> -Chinmay.
>>>>
>>>>
>>>> ~ Chinmay.
>>>>
>>>> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
>>>> wrote:
>>>>
>>>>> dot can not be used in dt-site.xml, as '.' is used as separator for
>>>> parsing
>>>>> various component from configuration file.
>>>>>
>>>>> - Tushar.
>>>>>
>>>>> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>> yogidevendra@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I would prefer '.' (dot).
>>>>>> If there are some technical challenges for dot then I vote for :
>>>> (colon)
>>>>> or
>>>>>> :: (double colon) in that sequence.
>>>>>>
>>>>>> ~ Yogi
>>>>>>
>>>>>> On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
>>>>> wrote:
>>>>>>> What about using “#”? We use this for Unifiers.
>>>>>>>
>>>>>>> Thanks
>>>>>>> - Gaurav
>>>>>>>
>>>>>>>> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>> tushar@datatorrent.com
>>>>>>> wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> As part of module support, we will need to generate names for
>>>>> operators
>>>>>>>> internal to the module before adding it to the original DAG to
>>>> avoid
>>>>>>>> conflict of names. For example if a developer adds a operator
>>> with
>>>>> name
>>>>>>> "A"
>>>>>>>> and module is name "M" in the DAG. Developer of 'M' is adding a
>>>>>> operator
>>>>>>>> 'A' into the module DAG. To avoid name conflict
>>>>>>>> with the operator in the main DAG we will generate name for
>>>> internal
>>>>>>>> operators by concatenation of module name and operator name.
>> The
>>>> 'A'
>>>>>>> within
>>>>>>>> module 'M' will be added to main DAG as 'M_A'. This works for
>>>>>> recursive
>>>>>>>> modules too. (i.e module containing module).
>>>>>>>>
>>>>>>>> Which separator should we use for module namespace? The
>>> requirement
>>>>> are
>>>>>>>> - Less chances of collision. (User can add operator with name
>>> 'M_A'
>>>>>> too).
>>>>>>>> - Compatible with json format. (Should not have a problem while
>>>>> parsing
>>>>>>>> json response from rest api)
>>>>>>>> - Should be able to specify property in dt-site.xml.
>>>>>>>>
>>>>>>>> While '_' works, In the pull request it has been pointed out
>> that
>>>> it
>>>>>> has
>>>>>>>> higher chance of collision. Few choices are
>>>>>>>>
>>>>>>>> 1) _ (Underscore)
>>>>>>>> 2) __ (double underscore)
>>>>>>>> 3) : (colon, I will check compatibility with rest API, as it is
>>>> used
>>>>> to
>>>>>>>> separate key and value in json)
>>>>>>>> 4) :: (Double colon).
>>>>>>>> 5) Let developer choose the separator by specifying dag
>> property,
>>>> but
>>>>>>>> default is still needed.
>>>>>>>> 6) Any other option?
>>>>>>>>
>>>>>>>> Please suggest which approach to take?
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> -Tushar.
>>>>>>>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: [APEX-3/APEX-104] Module Separator.
Posted by Vlad Rozov <v....@datatorrent.com>.
It is actually good that currently "/" is not properly handled by the
monitoring UI - it means that "/" was not used in operators names as
nobody reported this issue before. I think we should fix UI and REST API
if necessary to handle forward slash, instead of using '$'.
Thank you,
Vlad
On 12/8/15 01:40, Tushar Gosavi wrote:
> The '/' has a problem with the monitoring UI. When I click at the link with
> expanded module name it does not
> show the correct data. But if I paste the url directly I get the correct
> response.
>
> The browser replaces '/' with '%2F' and generates following url for
> operator (Me/M1/O1)
> http://localhost:9090/static/#/ops/apps/application_1449161397714_0017/logicalPlan/operators/Me%2FM1%2FO1
>
> I will use next voted char ('$') as a separator
>
> - Tushar.
>
> On Wed, Dec 2, 2015 at 11:30 PM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
>
>> Hi All,
>>
>> Following are the choices for separator
>> Char Votes
>> / 3
>> $ 3
>> :: 3
>> ~ 1
>> # 1
>>
>> I will '/' as a separator as it will look like as a hierarchical path in
>> REST api,
>> but need to confirm that as web framework might not support '/' character
>> in
>> parameter. If there is any issue while processing REST calls, I will use
>> '$'
>> as a separator.
>>
>> -Tushar.
>>
>> On Wed, Dec 2, 2015 at 6:04 PM, Tushar Gosavi <tu...@datatorrent.com>
>> wrote:
>>
>>> We already have explicit check to guard against duplicate operator names.
>>>
>>> On Tue, Dec 1, 2015 at 3:41 PM, Ganelin, Ilya <
>>> Ilya.Ganelin@capitalone.com> wrote:
>>>
>>>> Regardless of which symbol we select, we should add an explicit run-time
>>>> check to guard against collisions.
>>>>
>>>>
>>>>
>>>> Thank you,
>>>> Ilya Ganelin
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Priyanka Gugale [priyanka@datatorrent.com<mailto:
>>>> priyanka@datatorrent.com>]
>>>> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
>>>> To: dev@apex.incubator.apache.org
>>>> Subject: Re: [APEX-3/APEX-104] Module Separator.
>>>>
>>>>
>>>> +1 for $.
>>>>
>>>> -Priyanka
>>>>
>>>> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> I will also consider tilde as a separator. Module1~Operator1.
>>>>>
>>>>> Less conflict and more readable.
>>>>>
>>>>> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <
>>>> shubham@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> +1 for $ or (::) double colon
>>>>>>
>>>>>> Thanks,
>>>>>> Shubham
>>>>>>
>>>>>> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>>>>> chinmay@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I want to propose 2 more options for this:
>>>>>>> 1) $ (Dollar delimited)
>>>>>>> Dollar sign is usually to indicate inner classes in Java.
>>>> Considering
>>>>>> that
>>>>>>> I believe this can work in REST APIs and CLI as well.
>>>>>>>
>>>>>>> Hence the operator inside module can look like:
>>>>>>> Module1*$*Module2*$*Operator1
>>>>>>>
>>>>>>> 2) | (Pipe Delimited)
>>>>>>> Operator name would look like:
>>>>>>> Module1*|*Module2*|*Operator1
>>>>>>>
>>>>>>> 3) / (Unix File Seperator)
>>>>>>> This seperate is common and used in most of the places. file
>>>> seperator
>>>>> is
>>>>>>> also used to indicate the content inside something.
>>>>>>> Operator name would look like:
>>>>>>> Module1*/*Module2*/*Operator1
>>>>>>>
>>>>>>> If there are technical challenges in above, I would vote for (::)
>>>>> double
>>>>>>> colon.
>>>>>>>
>>>>>>> -Chinmay.
>>>>>>>
>>>>>>>
>>>>>>> ~ Chinmay.
>>>>>>>
>>>>>>> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <
>>>> tushar@datatorrent.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>> dot can not be used in dt-site.xml, as '.' is used as separator
>>>> for
>>>>>>> parsing
>>>>>>>> various component from configuration file.
>>>>>>>>
>>>>>>>> - Tushar.
>>>>>>>>
>>>>>>>> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>>>>> yogidevendra@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I would prefer '.' (dot).
>>>>>>>>> If there are some technical challenges for dot then I vote for
>>>> :
>>>>>>> (colon)
>>>>>>>> or
>>>>>>>>> :: (double colon) in that sequence.
>>>>>>>>>
>>>>>>>>> ~ Yogi
>>>>>>>>>
>>>>>>>>> On 1 December 2015 at 10:17, Gaurav Gupta <
>>>> gaurav@datatorrent.com>
>>>>>>>> wrote:
>>>>>>>>>> What about using “#”? We use this for Unifiers.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> - Gaurav
>>>>>>>>>>
>>>>>>>>>>> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>>>>> tushar@datatorrent.com
>>>>>>>>>> wrote:
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> As part of module support, we will need to generate names
>>>> for
>>>>>>>> operators
>>>>>>>>>>> internal to the module before adding it to the original
>>>> DAG to
>>>>>>> avoid
>>>>>>>>>>> conflict of names. For example if a developer adds a
>>>> operator
>>>>>> with
>>>>>>>> name
>>>>>>>>>> "A"
>>>>>>>>>>> and module is name "M" in the DAG. Developer of 'M' is
>>>> adding a
>>>>>>>>> operator
>>>>>>>>>>> 'A' into the module DAG. To avoid name conflict
>>>>>>>>>>> with the operator in the main DAG we will generate name for
>>>>>>> internal
>>>>>>>>>>> operators by concatenation of module name and operator
>>>> name.
>>>>> The
>>>>>>> 'A'
>>>>>>>>>> within
>>>>>>>>>>> module 'M' will be added to main DAG as 'M_A'. This works
>>>> for
>>>>>>>>> recursive
>>>>>>>>>>> modules too. (i.e module containing module).
>>>>>>>>>>>
>>>>>>>>>>> Which separator should we use for module namespace? The
>>>>>> requirement
>>>>>>>> are
>>>>>>>>>>> - Less chances of collision. (User can add operator with
>>>> name
>>>>>> 'M_A'
>>>>>>>>> too).
>>>>>>>>>>> - Compatible with json format. (Should not have a problem
>>>> while
>>>>>>>> parsing
>>>>>>>>>>> json response from rest api)
>>>>>>>>>>> - Should be able to specify property in dt-site.xml.
>>>>>>>>>>>
>>>>>>>>>>> While '_' works, In the pull request it has been pointed
>>>> out
>>>>> that
>>>>>>> it
>>>>>>>>> has
>>>>>>>>>>> higher chance of collision. Few choices are
>>>>>>>>>>>
>>>>>>>>>>> 1) _ (Underscore)
>>>>>>>>>>> 2) __ (double underscore)
>>>>>>>>>>> 3) : (colon, I will check compatibility with rest API, as
>>>> it is
>>>>>>> used
>>>>>>>> to
>>>>>>>>>>> separate key and value in json)
>>>>>>>>>>> 4) :: (Double colon).
>>>>>>>>>>> 5) Let developer choose the separator by specifying dag
>>>>> property,
>>>>>>> but
>>>>>>>>>>> default is still needed.
>>>>>>>>>>> 6) Any other option?
>>>>>>>>>>>
>>>>>>>>>>> Please suggest which approach to take?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> -Tushar.
>>>>>>>>>>
>>>> ________________________________________________________
>>>>
>>>> The information contained in this e-mail is confidential and/or
>>>> proprietary to Capital One and/or its affiliates and may only be used
>>>> solely in performance of work or services for Capital One. The information
>>>> transmitted herewith is intended only for use by the individual or entity
>>>> to which it is addressed. If the reader of this message is not the intended
>>>> recipient, you are hereby notified that any review, retransmission,
>>>> dissemination, distribution, copying or other use of, or taking of any
>>>> action in reliance upon this information is strictly prohibited. If you
>>>> have received this communication in error, please contact the sender and
>>>> delete the material from your computer.
>>>>
>>>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Tushar Gosavi <tu...@datatorrent.com>.
The '/' has a problem with the monitoring UI. When I click at the link with
expanded module name it does not
show the correct data. But if I paste the url directly I get the correct
response.
The browser replaces '/' with '%2F' and generates following url for
operator (Me/M1/O1)
http://localhost:9090/static/#/ops/apps/application_1449161397714_0017/logicalPlan/operators/Me%2FM1%2FO1
I will use next voted char ('$') as a separator
- Tushar.
On Wed, Dec 2, 2015 at 11:30 PM, Tushar Gosavi <tu...@datatorrent.com>
wrote:
> Hi All,
>
> Following are the choices for separator
> Char Votes
> / 3
> $ 3
> :: 3
> ~ 1
> # 1
>
> I will '/' as a separator as it will look like as a hierarchical path in
> REST api,
> but need to confirm that as web framework might not support '/' character
> in
> parameter. If there is any issue while processing REST calls, I will use
> '$'
> as a separator.
>
> -Tushar.
>
> On Wed, Dec 2, 2015 at 6:04 PM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
>
>> We already have explicit check to guard against duplicate operator names.
>>
>> On Tue, Dec 1, 2015 at 3:41 PM, Ganelin, Ilya <
>> Ilya.Ganelin@capitalone.com> wrote:
>>
>>> Regardless of which symbol we select, we should add an explicit run-time
>>> check to guard against collisions.
>>>
>>>
>>>
>>> Thank you,
>>> Ilya Ganelin
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Priyanka Gugale [priyanka@datatorrent.com<mailto:
>>> priyanka@datatorrent.com>]
>>> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
>>> To: dev@apex.incubator.apache.org
>>> Subject: Re: [APEX-3/APEX-104] Module Separator.
>>>
>>>
>>> +1 for $.
>>>
>>> -Priyanka
>>>
>>> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
>>> wrote:
>>>
>>> > I will also consider tilde as a separator. Module1~Operator1.
>>> >
>>> > Less conflict and more readable.
>>> >
>>> > On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <
>>> shubham@datatorrent.com>
>>> > wrote:
>>> >
>>> > > +1 for $ or (::) double colon
>>> > >
>>> > > Thanks,
>>> > > Shubham
>>> > >
>>> > > On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>>> > > chinmay@datatorrent.com>
>>> > > wrote:
>>> > >
>>> > > > Hi,
>>> > > >
>>> > > > I want to propose 2 more options for this:
>>> > > > 1) $ (Dollar delimited)
>>> > > > Dollar sign is usually to indicate inner classes in Java.
>>> Considering
>>> > > that
>>> > > > I believe this can work in REST APIs and CLI as well.
>>> > > >
>>> > > > Hence the operator inside module can look like:
>>> > > > Module1*$*Module2*$*Operator1
>>> > > >
>>> > > > 2) | (Pipe Delimited)
>>> > > > Operator name would look like:
>>> > > > Module1*|*Module2*|*Operator1
>>> > > >
>>> > > > 3) / (Unix File Seperator)
>>> > > > This seperate is common and used in most of the places. file
>>> seperator
>>> > is
>>> > > > also used to indicate the content inside something.
>>> > > > Operator name would look like:
>>> > > > Module1*/*Module2*/*Operator1
>>> > > >
>>> > > > If there are technical challenges in above, I would vote for (::)
>>> > double
>>> > > > colon.
>>> > > >
>>> > > > -Chinmay.
>>> > > >
>>> > > >
>>> > > > ~ Chinmay.
>>> > > >
>>> > > > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <
>>> tushar@datatorrent.com
>>> > >
>>> > > > wrote:
>>> > > >
>>> > > > > dot can not be used in dt-site.xml, as '.' is used as separator
>>> for
>>> > > > parsing
>>> > > > > various component from configuration file.
>>> > > > >
>>> > > > > - Tushar.
>>> > > > >
>>> > > > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>>> > > yogidevendra@apache.org>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > I would prefer '.' (dot).
>>> > > > > > If there are some technical challenges for dot then I vote for
>>> :
>>> > > > (colon)
>>> > > > > or
>>> > > > > > :: (double colon) in that sequence.
>>> > > > > >
>>> > > > > > ~ Yogi
>>> > > > > >
>>> > > > > > On 1 December 2015 at 10:17, Gaurav Gupta <
>>> gaurav@datatorrent.com>
>>> > > > > wrote:
>>> > > > > >
>>> > > > > > > What about using “#”? We use this for Unifiers.
>>> > > > > > >
>>> > > > > > > Thanks
>>> > > > > > > - Gaurav
>>> > > > > > >
>>> > > > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>>> > > tushar@datatorrent.com
>>> > > > >
>>> > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > Hi All,
>>> > > > > > > >
>>> > > > > > > > As part of module support, we will need to generate names
>>> for
>>> > > > > operators
>>> > > > > > > > internal to the module before adding it to the original
>>> DAG to
>>> > > > avoid
>>> > > > > > > > conflict of names. For example if a developer adds a
>>> operator
>>> > > with
>>> > > > > name
>>> > > > > > > "A"
>>> > > > > > > > and module is name "M" in the DAG. Developer of 'M' is
>>> adding a
>>> > > > > > operator
>>> > > > > > > > 'A' into the module DAG. To avoid name conflict
>>> > > > > > > > with the operator in the main DAG we will generate name for
>>> > > > internal
>>> > > > > > > > operators by concatenation of module name and operator
>>> name.
>>> > The
>>> > > > 'A'
>>> > > > > > > within
>>> > > > > > > > module 'M' will be added to main DAG as 'M_A'. This works
>>> for
>>> > > > > > recursive
>>> > > > > > > > modules too. (i.e module containing module).
>>> > > > > > > >
>>> > > > > > > > Which separator should we use for module namespace? The
>>> > > requirement
>>> > > > > are
>>> > > > > > > > - Less chances of collision. (User can add operator with
>>> name
>>> > > 'M_A'
>>> > > > > > too).
>>> > > > > > > > - Compatible with json format. (Should not have a problem
>>> while
>>> > > > > parsing
>>> > > > > > > > json response from rest api)
>>> > > > > > > > - Should be able to specify property in dt-site.xml.
>>> > > > > > > >
>>> > > > > > > > While '_' works, In the pull request it has been pointed
>>> out
>>> > that
>>> > > > it
>>> > > > > > has
>>> > > > > > > > higher chance of collision. Few choices are
>>> > > > > > > >
>>> > > > > > > > 1) _ (Underscore)
>>> > > > > > > > 2) __ (double underscore)
>>> > > > > > > > 3) : (colon, I will check compatibility with rest API, as
>>> it is
>>> > > > used
>>> > > > > to
>>> > > > > > > > separate key and value in json)
>>> > > > > > > > 4) :: (Double colon).
>>> > > > > > > > 5) Let developer choose the separator by specifying dag
>>> > property,
>>> > > > but
>>> > > > > > > > default is still needed.
>>> > > > > > > > 6) Any other option?
>>> > > > > > > >
>>> > > > > > > > Please suggest which approach to take?
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Regards,
>>> > > > > > > > -Tushar.
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> ________________________________________________________
>>>
>>> The information contained in this e-mail is confidential and/or
>>> proprietary to Capital One and/or its affiliates and may only be used
>>> solely in performance of work or services for Capital One. The information
>>> transmitted herewith is intended only for use by the individual or entity
>>> to which it is addressed. If the reader of this message is not the intended
>>> recipient, you are hereby notified that any review, retransmission,
>>> dissemination, distribution, copying or other use of, or taking of any
>>> action in reliance upon this information is strictly prohibited. If you
>>> have received this communication in error, please contact the sender and
>>> delete the material from your computer.
>>>
>>
>>
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Tushar Gosavi <tu...@datatorrent.com>.
Hi All,
Following are the choices for separator
Char Votes
/ 3
$ 3
:: 3
~ 1
# 1
I will '/' as a separator as it will look like as a hierarchical path in
REST api,
but need to confirm that as web framework might not support '/' character in
parameter. If there is any issue while processing REST calls, I will use '$'
as a separator.
-Tushar.
On Wed, Dec 2, 2015 at 6:04 PM, Tushar Gosavi <tu...@datatorrent.com>
wrote:
> We already have explicit check to guard against duplicate operator names.
>
> On Tue, Dec 1, 2015 at 3:41 PM, Ganelin, Ilya <Ilya.Ganelin@capitalone.com
> > wrote:
>
>> Regardless of which symbol we select, we should add an explicit run-time
>> check to guard against collisions.
>>
>>
>>
>> Thank you,
>> Ilya Ganelin
>>
>>
>>
>> -----Original Message-----
>> From: Priyanka Gugale [priyanka@datatorrent.com<mailto:
>> priyanka@datatorrent.com>]
>> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
>> To: dev@apex.incubator.apache.org
>> Subject: Re: [APEX-3/APEX-104] Module Separator.
>>
>>
>> +1 for $.
>>
>> -Priyanka
>>
>> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
>> wrote:
>>
>> > I will also consider tilde as a separator. Module1~Operator1.
>> >
>> > Less conflict and more readable.
>> >
>> > On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <
>> shubham@datatorrent.com>
>> > wrote:
>> >
>> > > +1 for $ or (::) double colon
>> > >
>> > > Thanks,
>> > > Shubham
>> > >
>> > > On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
>> > > chinmay@datatorrent.com>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I want to propose 2 more options for this:
>> > > > 1) $ (Dollar delimited)
>> > > > Dollar sign is usually to indicate inner classes in Java.
>> Considering
>> > > that
>> > > > I believe this can work in REST APIs and CLI as well.
>> > > >
>> > > > Hence the operator inside module can look like:
>> > > > Module1*$*Module2*$*Operator1
>> > > >
>> > > > 2) | (Pipe Delimited)
>> > > > Operator name would look like:
>> > > > Module1*|*Module2*|*Operator1
>> > > >
>> > > > 3) / (Unix File Seperator)
>> > > > This seperate is common and used in most of the places. file
>> seperator
>> > is
>> > > > also used to indicate the content inside something.
>> > > > Operator name would look like:
>> > > > Module1*/*Module2*/*Operator1
>> > > >
>> > > > If there are technical challenges in above, I would vote for (::)
>> > double
>> > > > colon.
>> > > >
>> > > > -Chinmay.
>> > > >
>> > > >
>> > > > ~ Chinmay.
>> > > >
>> > > > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <
>> tushar@datatorrent.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > dot can not be used in dt-site.xml, as '.' is used as separator
>> for
>> > > > parsing
>> > > > > various component from configuration file.
>> > > > >
>> > > > > - Tushar.
>> > > > >
>> > > > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
>> > > yogidevendra@apache.org>
>> > > > > wrote:
>> > > > >
>> > > > > > I would prefer '.' (dot).
>> > > > > > If there are some technical challenges for dot then I vote for :
>> > > > (colon)
>> > > > > or
>> > > > > > :: (double colon) in that sequence.
>> > > > > >
>> > > > > > ~ Yogi
>> > > > > >
>> > > > > > On 1 December 2015 at 10:17, Gaurav Gupta <
>> gaurav@datatorrent.com>
>> > > > > wrote:
>> > > > > >
>> > > > > > > What about using “#”? We use this for Unifiers.
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > - Gaurav
>> > > > > > >
>> > > > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
>> > > tushar@datatorrent.com
>> > > > >
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Hi All,
>> > > > > > > >
>> > > > > > > > As part of module support, we will need to generate names
>> for
>> > > > > operators
>> > > > > > > > internal to the module before adding it to the original DAG
>> to
>> > > > avoid
>> > > > > > > > conflict of names. For example if a developer adds a
>> operator
>> > > with
>> > > > > name
>> > > > > > > "A"
>> > > > > > > > and module is name "M" in the DAG. Developer of 'M' is
>> adding a
>> > > > > > operator
>> > > > > > > > 'A' into the module DAG. To avoid name conflict
>> > > > > > > > with the operator in the main DAG we will generate name for
>> > > > internal
>> > > > > > > > operators by concatenation of module name and operator name.
>> > The
>> > > > 'A'
>> > > > > > > within
>> > > > > > > > module 'M' will be added to main DAG as 'M_A'. This works
>> for
>> > > > > > recursive
>> > > > > > > > modules too. (i.e module containing module).
>> > > > > > > >
>> > > > > > > > Which separator should we use for module namespace? The
>> > > requirement
>> > > > > are
>> > > > > > > > - Less chances of collision. (User can add operator with
>> name
>> > > 'M_A'
>> > > > > > too).
>> > > > > > > > - Compatible with json format. (Should not have a problem
>> while
>> > > > > parsing
>> > > > > > > > json response from rest api)
>> > > > > > > > - Should be able to specify property in dt-site.xml.
>> > > > > > > >
>> > > > > > > > While '_' works, In the pull request it has been pointed out
>> > that
>> > > > it
>> > > > > > has
>> > > > > > > > higher chance of collision. Few choices are
>> > > > > > > >
>> > > > > > > > 1) _ (Underscore)
>> > > > > > > > 2) __ (double underscore)
>> > > > > > > > 3) : (colon, I will check compatibility with rest API, as
>> it is
>> > > > used
>> > > > > to
>> > > > > > > > separate key and value in json)
>> > > > > > > > 4) :: (Double colon).
>> > > > > > > > 5) Let developer choose the separator by specifying dag
>> > property,
>> > > > but
>> > > > > > > > default is still needed.
>> > > > > > > > 6) Any other option?
>> > > > > > > >
>> > > > > > > > Please suggest which approach to take?
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > -Tushar.
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> ________________________________________________________
>>
>> The information contained in this e-mail is confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>>
>
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Tushar Gosavi <tu...@datatorrent.com>.
We already have explicit check to guard against duplicate operator names.
On Tue, Dec 1, 2015 at 3:41 PM, Ganelin, Ilya <Il...@capitalone.com>
wrote:
> Regardless of which symbol we select, we should add an explicit run-time
> check to guard against collisions.
>
>
>
> Thank you,
> Ilya Ganelin
>
>
>
> -----Original Message-----
> From: Priyanka Gugale [priyanka@datatorrent.com<mailto:
> priyanka@datatorrent.com>]
> Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
> To: dev@apex.incubator.apache.org
> Subject: Re: [APEX-3/APEX-104] Module Separator.
>
>
> +1 for $.
>
> -Priyanka
>
> On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
> wrote:
>
> > I will also consider tilde as a separator. Module1~Operator1.
> >
> > Less conflict and more readable.
> >
> > On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <shubham@datatorrent.com
> >
> > wrote:
> >
> > > +1 for $ or (::) double colon
> > >
> > > Thanks,
> > > Shubham
> > >
> > > On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
> > > chinmay@datatorrent.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to propose 2 more options for this:
> > > > 1) $ (Dollar delimited)
> > > > Dollar sign is usually to indicate inner classes in Java. Considering
> > > that
> > > > I believe this can work in REST APIs and CLI as well.
> > > >
> > > > Hence the operator inside module can look like:
> > > > Module1*$*Module2*$*Operator1
> > > >
> > > > 2) | (Pipe Delimited)
> > > > Operator name would look like:
> > > > Module1*|*Module2*|*Operator1
> > > >
> > > > 3) / (Unix File Seperator)
> > > > This seperate is common and used in most of the places. file
> seperator
> > is
> > > > also used to indicate the content inside something.
> > > > Operator name would look like:
> > > > Module1*/*Module2*/*Operator1
> > > >
> > > > If there are technical challenges in above, I would vote for (::)
> > double
> > > > colon.
> > > >
> > > > -Chinmay.
> > > >
> > > >
> > > > ~ Chinmay.
> > > >
> > > > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <
> tushar@datatorrent.com
> > >
> > > > wrote:
> > > >
> > > > > dot can not be used in dt-site.xml, as '.' is used as separator for
> > > > parsing
> > > > > various component from configuration file.
> > > > >
> > > > > - Tushar.
> > > > >
> > > > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
> > > yogidevendra@apache.org>
> > > > > wrote:
> > > > >
> > > > > > I would prefer '.' (dot).
> > > > > > If there are some technical challenges for dot then I vote for :
> > > > (colon)
> > > > > or
> > > > > > :: (double colon) in that sequence.
> > > > > >
> > > > > > ~ Yogi
> > > > > >
> > > > > > On 1 December 2015 at 10:17, Gaurav Gupta <
> gaurav@datatorrent.com>
> > > > > wrote:
> > > > > >
> > > > > > > What about using “#”? We use this for Unifiers.
> > > > > > >
> > > > > > > Thanks
> > > > > > > - Gaurav
> > > > > > >
> > > > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
> > > tushar@datatorrent.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > As part of module support, we will need to generate names for
> > > > > operators
> > > > > > > > internal to the module before adding it to the original DAG
> to
> > > > avoid
> > > > > > > > conflict of names. For example if a developer adds a operator
> > > with
> > > > > name
> > > > > > > "A"
> > > > > > > > and module is name "M" in the DAG. Developer of 'M' is
> adding a
> > > > > > operator
> > > > > > > > 'A' into the module DAG. To avoid name conflict
> > > > > > > > with the operator in the main DAG we will generate name for
> > > > internal
> > > > > > > > operators by concatenation of module name and operator name.
> > The
> > > > 'A'
> > > > > > > within
> > > > > > > > module 'M' will be added to main DAG as 'M_A'. This works
> for
> > > > > > recursive
> > > > > > > > modules too. (i.e module containing module).
> > > > > > > >
> > > > > > > > Which separator should we use for module namespace? The
> > > requirement
> > > > > are
> > > > > > > > - Less chances of collision. (User can add operator with name
> > > 'M_A'
> > > > > > too).
> > > > > > > > - Compatible with json format. (Should not have a problem
> while
> > > > > parsing
> > > > > > > > json response from rest api)
> > > > > > > > - Should be able to specify property in dt-site.xml.
> > > > > > > >
> > > > > > > > While '_' works, In the pull request it has been pointed out
> > that
> > > > it
> > > > > > has
> > > > > > > > higher chance of collision. Few choices are
> > > > > > > >
> > > > > > > > 1) _ (Underscore)
> > > > > > > > 2) __ (double underscore)
> > > > > > > > 3) : (colon, I will check compatibility with rest API, as it
> is
> > > > used
> > > > > to
> > > > > > > > separate key and value in json)
> > > > > > > > 4) :: (Double colon).
> > > > > > > > 5) Let developer choose the separator by specifying dag
> > property,
> > > > but
> > > > > > > > default is still needed.
> > > > > > > > 6) Any other option?
> > > > > > > >
> > > > > > > > Please suggest which approach to take?
> > > > > > > >
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > -Tushar.
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
RE: [APEX-3/APEX-104] Module Separator.
Posted by "Ganelin, Ilya" <Il...@capitalone.com>.
Regardless of which symbol we select, we should add an explicit run-time check to guard against collisions.
Thank you,
Ilya Ganelin
-----Original Message-----
From: Priyanka Gugale [priyanka@datatorrent.com<ma...@datatorrent.com>]
Sent: Tuesday, December 01, 2015 04:01 AM Eastern Standard Time
To: dev@apex.incubator.apache.org
Subject: Re: [APEX-3/APEX-104] Module Separator.
+1 for $.
-Priyanka
On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
wrote:
> I will also consider tilde as a separator. Module1~Operator1.
>
> Less conflict and more readable.
>
> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
> wrote:
>
> > +1 for $ or (::) double colon
> >
> > Thanks,
> > Shubham
> >
> > On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
> > chinmay@datatorrent.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I want to propose 2 more options for this:
> > > 1) $ (Dollar delimited)
> > > Dollar sign is usually to indicate inner classes in Java. Considering
> > that
> > > I believe this can work in REST APIs and CLI as well.
> > >
> > > Hence the operator inside module can look like:
> > > Module1*$*Module2*$*Operator1
> > >
> > > 2) | (Pipe Delimited)
> > > Operator name would look like:
> > > Module1*|*Module2*|*Operator1
> > >
> > > 3) / (Unix File Seperator)
> > > This seperate is common and used in most of the places. file seperator
> is
> > > also used to indicate the content inside something.
> > > Operator name would look like:
> > > Module1*/*Module2*/*Operator1
> > >
> > > If there are technical challenges in above, I would vote for (::)
> double
> > > colon.
> > >
> > > -Chinmay.
> > >
> > >
> > > ~ Chinmay.
> > >
> > > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
> >
> > > wrote:
> > >
> > > > dot can not be used in dt-site.xml, as '.' is used as separator for
> > > parsing
> > > > various component from configuration file.
> > > >
> > > > - Tushar.
> > > >
> > > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
> > yogidevendra@apache.org>
> > > > wrote:
> > > >
> > > > > I would prefer '.' (dot).
> > > > > If there are some technical challenges for dot then I vote for :
> > > (colon)
> > > > or
> > > > > :: (double colon) in that sequence.
> > > > >
> > > > > ~ Yogi
> > > > >
> > > > > On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
> > > > wrote:
> > > > >
> > > > > > What about using “#”? We use this for Unifiers.
> > > > > >
> > > > > > Thanks
> > > > > > - Gaurav
> > > > > >
> > > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
> > tushar@datatorrent.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > As part of module support, we will need to generate names for
> > > > operators
> > > > > > > internal to the module before adding it to the original DAG to
> > > avoid
> > > > > > > conflict of names. For example if a developer adds a operator
> > with
> > > > name
> > > > > > "A"
> > > > > > > and module is name "M" in the DAG. Developer of 'M' is adding a
> > > > > operator
> > > > > > > 'A' into the module DAG. To avoid name conflict
> > > > > > > with the operator in the main DAG we will generate name for
> > > internal
> > > > > > > operators by concatenation of module name and operator name.
> The
> > > 'A'
> > > > > > within
> > > > > > > module 'M' will be added to main DAG as 'M_A'. This works for
> > > > > recursive
> > > > > > > modules too. (i.e module containing module).
> > > > > > >
> > > > > > > Which separator should we use for module namespace? The
> > requirement
> > > > are
> > > > > > > - Less chances of collision. (User can add operator with name
> > 'M_A'
> > > > > too).
> > > > > > > - Compatible with json format. (Should not have a problem while
> > > > parsing
> > > > > > > json response from rest api)
> > > > > > > - Should be able to specify property in dt-site.xml.
> > > > > > >
> > > > > > > While '_' works, In the pull request it has been pointed out
> that
> > > it
> > > > > has
> > > > > > > higher chance of collision. Few choices are
> > > > > > >
> > > > > > > 1) _ (Underscore)
> > > > > > > 2) __ (double underscore)
> > > > > > > 3) : (colon, I will check compatibility with rest API, as it is
> > > used
> > > > to
> > > > > > > separate key and value in json)
> > > > > > > 4) :: (Double colon).
> > > > > > > 5) Let developer choose the separator by specifying dag
> property,
> > > but
> > > > > > > default is still needed.
> > > > > > > 6) Any other option?
> > > > > > >
> > > > > > > Please suggest which approach to take?
> > > > > > >
> > > > > > >
> > > > > > > Regards,
> > > > > > > -Tushar.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
Re: [APEX-3/APEX-104] Module Separator.
Posted by Priyanka Gugale <pr...@datatorrent.com>.
+1 for $.
-Priyanka
On Tue, Dec 1, 2015 at 2:26 PM, Chandni Singh <ch...@datatorrent.com>
wrote:
> I will also consider tilde as a separator. Module1~Operator1.
>
> Less conflict and more readable.
>
> On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
> wrote:
>
> > +1 for $ or (::) double colon
> >
> > Thanks,
> > Shubham
> >
> > On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
> > chinmay@datatorrent.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I want to propose 2 more options for this:
> > > 1) $ (Dollar delimited)
> > > Dollar sign is usually to indicate inner classes in Java. Considering
> > that
> > > I believe this can work in REST APIs and CLI as well.
> > >
> > > Hence the operator inside module can look like:
> > > Module1*$*Module2*$*Operator1
> > >
> > > 2) | (Pipe Delimited)
> > > Operator name would look like:
> > > Module1*|*Module2*|*Operator1
> > >
> > > 3) / (Unix File Seperator)
> > > This seperate is common and used in most of the places. file seperator
> is
> > > also used to indicate the content inside something.
> > > Operator name would look like:
> > > Module1*/*Module2*/*Operator1
> > >
> > > If there are technical challenges in above, I would vote for (::)
> double
> > > colon.
> > >
> > > -Chinmay.
> > >
> > >
> > > ~ Chinmay.
> > >
> > > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tushar@datatorrent.com
> >
> > > wrote:
> > >
> > > > dot can not be used in dt-site.xml, as '.' is used as separator for
> > > parsing
> > > > various component from configuration file.
> > > >
> > > > - Tushar.
> > > >
> > > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
> > yogidevendra@apache.org>
> > > > wrote:
> > > >
> > > > > I would prefer '.' (dot).
> > > > > If there are some technical challenges for dot then I vote for :
> > > (colon)
> > > > or
> > > > > :: (double colon) in that sequence.
> > > > >
> > > > > ~ Yogi
> > > > >
> > > > > On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
> > > > wrote:
> > > > >
> > > > > > What about using “#”? We use this for Unifiers.
> > > > > >
> > > > > > Thanks
> > > > > > - Gaurav
> > > > > >
> > > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
> > tushar@datatorrent.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > As part of module support, we will need to generate names for
> > > > operators
> > > > > > > internal to the module before adding it to the original DAG to
> > > avoid
> > > > > > > conflict of names. For example if a developer adds a operator
> > with
> > > > name
> > > > > > "A"
> > > > > > > and module is name "M" in the DAG. Developer of 'M' is adding a
> > > > > operator
> > > > > > > 'A' into the module DAG. To avoid name conflict
> > > > > > > with the operator in the main DAG we will generate name for
> > > internal
> > > > > > > operators by concatenation of module name and operator name.
> The
> > > 'A'
> > > > > > within
> > > > > > > module 'M' will be added to main DAG as 'M_A'. This works for
> > > > > recursive
> > > > > > > modules too. (i.e module containing module).
> > > > > > >
> > > > > > > Which separator should we use for module namespace? The
> > requirement
> > > > are
> > > > > > > - Less chances of collision. (User can add operator with name
> > 'M_A'
> > > > > too).
> > > > > > > - Compatible with json format. (Should not have a problem while
> > > > parsing
> > > > > > > json response from rest api)
> > > > > > > - Should be able to specify property in dt-site.xml.
> > > > > > >
> > > > > > > While '_' works, In the pull request it has been pointed out
> that
> > > it
> > > > > has
> > > > > > > higher chance of collision. Few choices are
> > > > > > >
> > > > > > > 1) _ (Underscore)
> > > > > > > 2) __ (double underscore)
> > > > > > > 3) : (colon, I will check compatibility with rest API, as it is
> > > used
> > > > to
> > > > > > > separate key and value in json)
> > > > > > > 4) :: (Double colon).
> > > > > > > 5) Let developer choose the separator by specifying dag
> property,
> > > but
> > > > > > > default is still needed.
> > > > > > > 6) Any other option?
> > > > > > >
> > > > > > > Please suggest which approach to take?
> > > > > > >
> > > > > > >
> > > > > > > Regards,
> > > > > > > -Tushar.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Chandni Singh <ch...@datatorrent.com>.
I will also consider tilde as a separator. Module1~Operator1.
Less conflict and more readable.
On Tue, Dec 1, 2015 at 12:22 AM, Shubham Pathak <sh...@datatorrent.com>
wrote:
> +1 for $ or (::) double colon
>
> Thanks,
> Shubham
>
> On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <
> chinmay@datatorrent.com>
> wrote:
>
> > Hi,
> >
> > I want to propose 2 more options for this:
> > 1) $ (Dollar delimited)
> > Dollar sign is usually to indicate inner classes in Java. Considering
> that
> > I believe this can work in REST APIs and CLI as well.
> >
> > Hence the operator inside module can look like:
> > Module1*$*Module2*$*Operator1
> >
> > 2) | (Pipe Delimited)
> > Operator name would look like:
> > Module1*|*Module2*|*Operator1
> >
> > 3) / (Unix File Seperator)
> > This seperate is common and used in most of the places. file seperator is
> > also used to indicate the content inside something.
> > Operator name would look like:
> > Module1*/*Module2*/*Operator1
> >
> > If there are technical challenges in above, I would vote for (::) double
> > colon.
> >
> > -Chinmay.
> >
> >
> > ~ Chinmay.
> >
> > On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tu...@datatorrent.com>
> > wrote:
> >
> > > dot can not be used in dt-site.xml, as '.' is used as separator for
> > parsing
> > > various component from configuration file.
> > >
> > > - Tushar.
> > >
> > > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <
> yogidevendra@apache.org>
> > > wrote:
> > >
> > > > I would prefer '.' (dot).
> > > > If there are some technical challenges for dot then I vote for :
> > (colon)
> > > or
> > > > :: (double colon) in that sequence.
> > > >
> > > > ~ Yogi
> > > >
> > > > On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
> > > wrote:
> > > >
> > > > > What about using “#”? We use this for Unifiers.
> > > > >
> > > > > Thanks
> > > > > - Gaurav
> > > > >
> > > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <
> tushar@datatorrent.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > As part of module support, we will need to generate names for
> > > operators
> > > > > > internal to the module before adding it to the original DAG to
> > avoid
> > > > > > conflict of names. For example if a developer adds a operator
> with
> > > name
> > > > > "A"
> > > > > > and module is name "M" in the DAG. Developer of 'M' is adding a
> > > > operator
> > > > > > 'A' into the module DAG. To avoid name conflict
> > > > > > with the operator in the main DAG we will generate name for
> > internal
> > > > > > operators by concatenation of module name and operator name. The
> > 'A'
> > > > > within
> > > > > > module 'M' will be added to main DAG as 'M_A'. This works for
> > > > recursive
> > > > > > modules too. (i.e module containing module).
> > > > > >
> > > > > > Which separator should we use for module namespace? The
> requirement
> > > are
> > > > > > - Less chances of collision. (User can add operator with name
> 'M_A'
> > > > too).
> > > > > > - Compatible with json format. (Should not have a problem while
> > > parsing
> > > > > > json response from rest api)
> > > > > > - Should be able to specify property in dt-site.xml.
> > > > > >
> > > > > > While '_' works, In the pull request it has been pointed out that
> > it
> > > > has
> > > > > > higher chance of collision. Few choices are
> > > > > >
> > > > > > 1) _ (Underscore)
> > > > > > 2) __ (double underscore)
> > > > > > 3) : (colon, I will check compatibility with rest API, as it is
> > used
> > > to
> > > > > > separate key and value in json)
> > > > > > 4) :: (Double colon).
> > > > > > 5) Let developer choose the separator by specifying dag property,
> > but
> > > > > > default is still needed.
> > > > > > 6) Any other option?
> > > > > >
> > > > > > Please suggest which approach to take?
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > -Tushar.
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Shubham Pathak <sh...@datatorrent.com>.
+1 for $ or (::) double colon
Thanks,
Shubham
On Tue, Dec 1, 2015 at 11:11 AM, Chinmay Kolhatkar <ch...@datatorrent.com>
wrote:
> Hi,
>
> I want to propose 2 more options for this:
> 1) $ (Dollar delimited)
> Dollar sign is usually to indicate inner classes in Java. Considering that
> I believe this can work in REST APIs and CLI as well.
>
> Hence the operator inside module can look like:
> Module1*$*Module2*$*Operator1
>
> 2) | (Pipe Delimited)
> Operator name would look like:
> Module1*|*Module2*|*Operator1
>
> 3) / (Unix File Seperator)
> This seperate is common and used in most of the places. file seperator is
> also used to indicate the content inside something.
> Operator name would look like:
> Module1*/*Module2*/*Operator1
>
> If there are technical challenges in above, I would vote for (::) double
> colon.
>
> -Chinmay.
>
>
> ~ Chinmay.
>
> On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
>
> > dot can not be used in dt-site.xml, as '.' is used as separator for
> parsing
> > various component from configuration file.
> >
> > - Tushar.
> >
> > On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <yo...@apache.org>
> > wrote:
> >
> > > I would prefer '.' (dot).
> > > If there are some technical challenges for dot then I vote for :
> (colon)
> > or
> > > :: (double colon) in that sequence.
> > >
> > > ~ Yogi
> > >
> > > On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
> > wrote:
> > >
> > > > What about using “#”? We use this for Unifiers.
> > > >
> > > > Thanks
> > > > - Gaurav
> > > >
> > > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <tushar@datatorrent.com
> >
> > > > wrote:
> > > > >
> > > > > Hi All,
> > > > >
> > > > > As part of module support, we will need to generate names for
> > operators
> > > > > internal to the module before adding it to the original DAG to
> avoid
> > > > > conflict of names. For example if a developer adds a operator with
> > name
> > > > "A"
> > > > > and module is name "M" in the DAG. Developer of 'M' is adding a
> > > operator
> > > > > 'A' into the module DAG. To avoid name conflict
> > > > > with the operator in the main DAG we will generate name for
> internal
> > > > > operators by concatenation of module name and operator name. The
> 'A'
> > > > within
> > > > > module 'M' will be added to main DAG as 'M_A'. This works for
> > > recursive
> > > > > modules too. (i.e module containing module).
> > > > >
> > > > > Which separator should we use for module namespace? The requirement
> > are
> > > > > - Less chances of collision. (User can add operator with name 'M_A'
> > > too).
> > > > > - Compatible with json format. (Should not have a problem while
> > parsing
> > > > > json response from rest api)
> > > > > - Should be able to specify property in dt-site.xml.
> > > > >
> > > > > While '_' works, In the pull request it has been pointed out that
> it
> > > has
> > > > > higher chance of collision. Few choices are
> > > > >
> > > > > 1) _ (Underscore)
> > > > > 2) __ (double underscore)
> > > > > 3) : (colon, I will check compatibility with rest API, as it is
> used
> > to
> > > > > separate key and value in json)
> > > > > 4) :: (Double colon).
> > > > > 5) Let developer choose the separator by specifying dag property,
> but
> > > > > default is still needed.
> > > > > 6) Any other option?
> > > > >
> > > > > Please suggest which approach to take?
> > > > >
> > > > >
> > > > > Regards,
> > > > > -Tushar.
> > > >
> > > >
> > >
> >
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Chinmay Kolhatkar <ch...@datatorrent.com>.
Hi,
I want to propose 2 more options for this:
1) $ (Dollar delimited)
Dollar sign is usually to indicate inner classes in Java. Considering that
I believe this can work in REST APIs and CLI as well.
Hence the operator inside module can look like:
Module1*$*Module2*$*Operator1
2) | (Pipe Delimited)
Operator name would look like:
Module1*|*Module2*|*Operator1
3) / (Unix File Seperator)
This seperate is common and used in most of the places. file seperator is
also used to indicate the content inside something.
Operator name would look like:
Module1*/*Module2*/*Operator1
If there are technical challenges in above, I would vote for (::) double
colon.
-Chinmay.
~ Chinmay.
On Tue, Dec 1, 2015 at 11:01 AM, Tushar Gosavi <tu...@datatorrent.com>
wrote:
> dot can not be used in dt-site.xml, as '.' is used as separator for parsing
> various component from configuration file.
>
> - Tushar.
>
> On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <yo...@apache.org>
> wrote:
>
> > I would prefer '.' (dot).
> > If there are some technical challenges for dot then I vote for : (colon)
> or
> > :: (double colon) in that sequence.
> >
> > ~ Yogi
> >
> > On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com>
> wrote:
> >
> > > What about using “#”? We use this for Unifiers.
> > >
> > > Thanks
> > > - Gaurav
> > >
> > > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <tu...@datatorrent.com>
> > > wrote:
> > > >
> > > > Hi All,
> > > >
> > > > As part of module support, we will need to generate names for
> operators
> > > > internal to the module before adding it to the original DAG to avoid
> > > > conflict of names. For example if a developer adds a operator with
> name
> > > "A"
> > > > and module is name "M" in the DAG. Developer of 'M' is adding a
> > operator
> > > > 'A' into the module DAG. To avoid name conflict
> > > > with the operator in the main DAG we will generate name for internal
> > > > operators by concatenation of module name and operator name. The 'A'
> > > within
> > > > module 'M' will be added to main DAG as 'M_A'. This works for
> > recursive
> > > > modules too. (i.e module containing module).
> > > >
> > > > Which separator should we use for module namespace? The requirement
> are
> > > > - Less chances of collision. (User can add operator with name 'M_A'
> > too).
> > > > - Compatible with json format. (Should not have a problem while
> parsing
> > > > json response from rest api)
> > > > - Should be able to specify property in dt-site.xml.
> > > >
> > > > While '_' works, In the pull request it has been pointed out that it
> > has
> > > > higher chance of collision. Few choices are
> > > >
> > > > 1) _ (Underscore)
> > > > 2) __ (double underscore)
> > > > 3) : (colon, I will check compatibility with rest API, as it is used
> to
> > > > separate key and value in json)
> > > > 4) :: (Double colon).
> > > > 5) Let developer choose the separator by specifying dag property, but
> > > > default is still needed.
> > > > 6) Any other option?
> > > >
> > > > Please suggest which approach to take?
> > > >
> > > >
> > > > Regards,
> > > > -Tushar.
> > >
> > >
> >
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Tushar Gosavi <tu...@datatorrent.com>.
dot can not be used in dt-site.xml, as '.' is used as separator for parsing
various component from configuration file.
- Tushar.
On Tue, Dec 1, 2015 at 10:48 AM, Yogi Devendra <yo...@apache.org>
wrote:
> I would prefer '.' (dot).
> If there are some technical challenges for dot then I vote for : (colon) or
> :: (double colon) in that sequence.
>
> ~ Yogi
>
> On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com> wrote:
>
> > What about using “#”? We use this for Unifiers.
> >
> > Thanks
> > - Gaurav
> >
> > > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <tu...@datatorrent.com>
> > wrote:
> > >
> > > Hi All,
> > >
> > > As part of module support, we will need to generate names for operators
> > > internal to the module before adding it to the original DAG to avoid
> > > conflict of names. For example if a developer adds a operator with name
> > "A"
> > > and module is name "M" in the DAG. Developer of 'M' is adding a
> operator
> > > 'A' into the module DAG. To avoid name conflict
> > > with the operator in the main DAG we will generate name for internal
> > > operators by concatenation of module name and operator name. The 'A'
> > within
> > > module 'M' will be added to main DAG as 'M_A'. This works for
> recursive
> > > modules too. (i.e module containing module).
> > >
> > > Which separator should we use for module namespace? The requirement are
> > > - Less chances of collision. (User can add operator with name 'M_A'
> too).
> > > - Compatible with json format. (Should not have a problem while parsing
> > > json response from rest api)
> > > - Should be able to specify property in dt-site.xml.
> > >
> > > While '_' works, In the pull request it has been pointed out that it
> has
> > > higher chance of collision. Few choices are
> > >
> > > 1) _ (Underscore)
> > > 2) __ (double underscore)
> > > 3) : (colon, I will check compatibility with rest API, as it is used to
> > > separate key and value in json)
> > > 4) :: (Double colon).
> > > 5) Let developer choose the separator by specifying dag property, but
> > > default is still needed.
> > > 6) Any other option?
> > >
> > > Please suggest which approach to take?
> > >
> > >
> > > Regards,
> > > -Tushar.
> >
> >
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Yogi Devendra <yo...@apache.org>.
I would prefer '.' (dot).
If there are some technical challenges for dot then I vote for : (colon) or
:: (double colon) in that sequence.
~ Yogi
On 1 December 2015 at 10:17, Gaurav Gupta <ga...@datatorrent.com> wrote:
> What about using “#”? We use this for Unifiers.
>
> Thanks
> - Gaurav
>
> > On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <tu...@datatorrent.com>
> wrote:
> >
> > Hi All,
> >
> > As part of module support, we will need to generate names for operators
> > internal to the module before adding it to the original DAG to avoid
> > conflict of names. For example if a developer adds a operator with name
> "A"
> > and module is name "M" in the DAG. Developer of 'M' is adding a operator
> > 'A' into the module DAG. To avoid name conflict
> > with the operator in the main DAG we will generate name for internal
> > operators by concatenation of module name and operator name. The 'A'
> within
> > module 'M' will be added to main DAG as 'M_A'. This works for recursive
> > modules too. (i.e module containing module).
> >
> > Which separator should we use for module namespace? The requirement are
> > - Less chances of collision. (User can add operator with name 'M_A' too).
> > - Compatible with json format. (Should not have a problem while parsing
> > json response from rest api)
> > - Should be able to specify property in dt-site.xml.
> >
> > While '_' works, In the pull request it has been pointed out that it has
> > higher chance of collision. Few choices are
> >
> > 1) _ (Underscore)
> > 2) __ (double underscore)
> > 3) : (colon, I will check compatibility with rest API, as it is used to
> > separate key and value in json)
> > 4) :: (Double colon).
> > 5) Let developer choose the separator by specifying dag property, but
> > default is still needed.
> > 6) Any other option?
> >
> > Please suggest which approach to take?
> >
> >
> > Regards,
> > -Tushar.
>
>
Re: [APEX-3/APEX-104] Module Separator.
Posted by Gaurav Gupta <ga...@datatorrent.com>.
What about using “#”? We use this for Unifiers.
Thanks
- Gaurav
> On Nov 30, 2015, at 8:08 PM, Tushar Gosavi <tu...@datatorrent.com> wrote:
>
> Hi All,
>
> As part of module support, we will need to generate names for operators
> internal to the module before adding it to the original DAG to avoid
> conflict of names. For example if a developer adds a operator with name "A"
> and module is name "M" in the DAG. Developer of 'M' is adding a operator
> 'A' into the module DAG. To avoid name conflict
> with the operator in the main DAG we will generate name for internal
> operators by concatenation of module name and operator name. The 'A' within
> module 'M' will be added to main DAG as 'M_A'. This works for recursive
> modules too. (i.e module containing module).
>
> Which separator should we use for module namespace? The requirement are
> - Less chances of collision. (User can add operator with name 'M_A' too).
> - Compatible with json format. (Should not have a problem while parsing
> json response from rest api)
> - Should be able to specify property in dt-site.xml.
>
> While '_' works, In the pull request it has been pointed out that it has
> higher chance of collision. Few choices are
>
> 1) _ (Underscore)
> 2) __ (double underscore)
> 3) : (colon, I will check compatibility with rest API, as it is used to
> separate key and value in json)
> 4) :: (Double colon).
> 5) Let developer choose the separator by specifying dag property, but
> default is still needed.
> 6) Any other option?
>
> Please suggest which approach to take?
>
>
> Regards,
> -Tushar.