Tang.BindListEntry

[Matteo Interlandi <m....@gmail.com>]

Hello,

I have implemented a BindListEntry in Tang matching the behavior of
BindSetEntry but maintaining order. Do you want to me to fill a PR?

In addition I found that the Avro serialized/deserializer of Tang is
missing the implementation for configurations added using the BindList API.
I can also fill a PR for that.

Best,
Matteo

Re: Tang.BindListEntry

[Matteo Interlandi <m....@gmail.com>]

Gotcha. My current API is sort of nice because I recursively call the
operator.next and pass a configuration ref so that each operator append its
parameters to the configuration.

I guess I will have to change my API to use BindList..

On Thu, Jan 4, 2018, 6:19 PM Markus Weimer <ma...@weimo.de> wrote:

> On Thu, Jan 4, 2018 at 5:51 PM, Matteo Interlandi <m....@gmail.com>
> wrote:
>
> > 1) I actually need order because I am serializing pipelines of operators
> > one a time and I need to maintain the order in which operators appear. Is
> > this against Tang design?
> >
>
> Kinda. To make Tang safe, the order of .`set()` and `.bind()` calls should
> not matter. That is why you can add elements to a set one at a time, but
> have to bind whole lists as atomic values. If you can live with that, you
> should be fine :)
>
> Markus
>

Re: Tang.BindListEntry

[Gyewon Lee <st...@gmail.com>]

Hi, Matteo.

For 2), I used Avro array for List-binding ConfigurationEntry, because we
don't want to use special separators. As other ConfigurationEntry is
represented in avro string, avro union is necessary to support both array
and existing string entries.

Best,
Gyewon

2018-01-05 11:18 GMT+09:00 Markus Weimer <ma...@weimo.de>:

> On Thu, Jan 4, 2018 at 5:51 PM, Matteo Interlandi <m....@gmail.com>
> wrote:
>
> > 1) I actually need order because I am serializing pipelines of operators
> > one a time and I need to maintain the order in which operators appear. Is
> > this against Tang design?
> >
>
> Kinda. To make Tang safe, the order of .`set()` and `.bind()` calls should
> not matter. That is why you can add elements to a set one at a time, but
> have to bind whole lists as atomic values. If you can live with that, you
> should be fine :)
>
> Markus
>

Re: Tang.BindListEntry

[Markus Weimer <ma...@weimo.de>]

On Thu, Jan 4, 2018 at 5:51 PM, Matteo Interlandi <m....@gmail.com>
wrote:

> 1) I actually need order because I am serializing pipelines of operators
> one a time and I need to maintain the order in which operators appear. Is
> this against Tang design?
>

Kinda. To make Tang safe, the order of .`set()` and `.bind()` calls should
not matter. That is why you can add elements to a set one at a time, but
have to bind whole lists as atomic values. If you can live with that, you
should be fine :)

Markus

Re: Tang.BindListEntry

[Matteo Interlandi <m....@gmail.com>]

Hi Gyewon,

thanks for the comments. I have few questions:

1) I actually need order because I am serializing pipelines of operators
one a time and I need to maintain the order in which operators appear. Is
this against Tang design?

2) Mmm my implementation for Avro is basically a copy of how sets are
serialized: they are mapped to strings and enumerated. Where was union
required?

Best,
Matteo

On Thu, Jan 4, 2018 at 5:32 PM, Gyewon Lee <st...@gmail.com> wrote:

> Hi, Matteo.
>
> Thanks a lot for your work. I have implemented the APIs for binding List in
> Tang.
>
> 1) I didn't implement bindListEntry, because unlike Set, the order of
> elements is important in List. As far as I know, any Tang configuration
> should not be affected by the order of its bindings. For that reason, Tang
> binds the whole List, which has the complete order information among
> elements.
> 2) I also implemented Avro serialization part for List in Tang (for Java
> side). However, at that time we didn't merge it because avro "union" schema
> was necessary to support it, which Avro in Nuget (which had lower version)
> didn't support. I think I can check whether C# avro supports it now or not,
> and if it does I can merge my previous work as well as C#-part
> serialization.
>
> Best,
> Gyewon
>
> 2018-01-05 10:18 GMT+09:00 Matteo Interlandi <m....@gmail.com>:
>
> > Hello,
> >
> > I have implemented a BindListEntry in Tang matching the behavior of
> > BindSetEntry but maintaining order. Do you want to me to fill a PR?
> >
> > In addition I found that the Avro serialized/deserializer of Tang is
> > missing the implementation for configurations added using the BindList
> API.
> > I can also fill a PR for that.
> >
> > Best,
> > Matteo
> >
>

Re: Tang.BindListEntry

[Gyewon Lee <st...@gmail.com>]

Hi, Matteo.

Thanks a lot for your work. I have implemented the APIs for binding List in
Tang.

1) I didn't implement bindListEntry, because unlike Set, the order of
elements is important in List. As far as I know, any Tang configuration
should not be affected by the order of its bindings. For that reason, Tang
binds the whole List, which has the complete order information among
elements.
2) I also implemented Avro serialization part for List in Tang (for Java
side). However, at that time we didn't merge it because avro "union" schema
was necessary to support it, which Avro in Nuget (which had lower version)
didn't support. I think I can check whether C# avro supports it now or not,
and if it does I can merge my previous work as well as C#-part
serialization.

Best,
Gyewon

2018-01-05 10:18 GMT+09:00 Matteo Interlandi <m....@gmail.com>:

> Hello,
>
> I have implemented a BindListEntry in Tang matching the behavior of
> BindSetEntry but maintaining order. Do you want to me to fill a PR?
>
> In addition I found that the Avro serialized/deserializer of Tang is
> missing the implementation for configurations added using the BindList API.
> I can also fill a PR for that.
>
> Best,
> Matteo
>