Re: Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

[Jorge Esteban Quilcate Otoya <qu...@gmail.com>]

Hi Chris,

Thanks for your feedback!

1. Yes, it will be context-dependent. I have added rules and scenarios to
the nested notation to cover the happy path and edge cases. In short,
backticks will be not be considered as part of the field name when they are
wrapping a field name: first backtick at the beginning of the path or after
a dot, and closing backtick before a dot or at the end of the path.
If backticks happen to be in those positions, use backslash to escape them.
2. You're right, that's a typo. Fixing it.
3. I don't think so, I have added a scenario to clarify this.

KIP is updated. Hopefully the rules and scenarios help to close any open
gap. Let me know if you see any cases that is not considered to address it.

Cheers,
Jorge.

On Wed, 31 Aug 2022 at 20:02, Chris Egerton <ch...@aiven.io.invalid> wrote:

> Hi Robert and Jorge,
>
> I think the backtick/backslash proposal works, but I'm a little unclear on
> some of the details:
>
> 1. Are backticks only given special treatment when they immediately follow
> a non-escaped dot? E.g., "foo.b`ar.ba`z" would refer to "foo" -> "b`ar" ->
> "ba`z" instead of "foo" -> "bar.baz"? Based on the example where the name
> "a.b`.c" refers to "a" -> "b`" -> "c", it seems like this is the case, but
> I'm worried this might cause confusion since the role of the backtick and
> the need to escape it becomes context-dependent.
>
> 2. In the example table near the beginning of the KIP, the name "a.`b\`.c`"
> refers to "a" -> "b`c". What happened to the dot in the second part of the
> name? Should it refer to "a" -> "b`.c" instead?
>
> 3. Is it ever necessary to escape backslashes themselves? If so, when?
>
> Overall, I wish we could come up with a prettier/simpler approach, but the
> benefits provided by the dual backtick/dot syntax are too great to deny:
> there are no correctness issues like the ones posed with double-dot
> escaping that would lead to ambiguity, the most common cases are still very
> simple to work with, and there's no risk of interfering with JSON escape
> mechanisms (in most cases) or single-quote shell quoting (which may be
> relevant when connector configurations are defined on the command line).
> Thanks for the suggestion, Robert!
>
> Cheers,
>
> Chris
>

Re: Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

[Jorge Esteban Quilcate Otoya <qu...@gmail.com>]

Hi there,

Bumping this thread for visibility.

Cheers,
Jorge

On Fri, 2 Sep 2022, 18:01 Chris Egerton, <ch...@aiven.io.invalid> wrote:

> Hi Jorge,
>
> One tiny nit, but LGTM otherwise:
>
> The KIP mentions backslashes as "(/)"; shouldn't this be "(\)"?
>
> I'll cast a +1 on the vote thread anyways; I'm sure this won't block us.
>
> Cheers, and thanks for all your hard work on this!
>
> Chris
>
> On Thu, Sep 1, 2022 at 1:33 PM Jorge Esteban Quilcate Otoya <
> quilcate.jorge@gmail.com> wrote:
>
> > Hi Chris,
> >
> > Thanks for your feedback!
> >
> > 1. Yes, it will be context-dependent. I have added rules and scenarios to
> > the nested notation to cover the happy path and edge cases. In short,
> > backticks will be not be considered as part of the field name when they
> are
> > wrapping a field name: first backtick at the beginning of the path or
> after
> > a dot, and closing backtick before a dot or at the end of the path.
> > If backticks happen to be in those positions, use backslash to escape
> them.
> > 2. You're right, that's a typo. Fixing it.
> > 3. I don't think so, I have added a scenario to clarify this.
> >
> > KIP is updated. Hopefully the rules and scenarios help to close any open
> > gap. Let me know if you see any cases that is not considered to address
> it.
> >
> > Cheers,
> > Jorge.
> >
> > On Wed, 31 Aug 2022 at 20:02, Chris Egerton <ch...@aiven.io.invalid>
> > wrote:
> >
> > > Hi Robert and Jorge,
> > >
> > > I think the backtick/backslash proposal works, but I'm a little unclear
> > on
> > > some of the details:
> > >
> > > 1. Are backticks only given special treatment when they immediately
> > follow
> > > a non-escaped dot? E.g., "foo.b`ar.ba`z" would refer to "foo" ->
> "b`ar"
> > ->
> > > "ba`z" instead of "foo" -> "bar.baz"? Based on the example where the
> name
> > > "a.b`.c" refers to "a" -> "b`" -> "c", it seems like this is the case,
> > but
> > > I'm worried this might cause confusion since the role of the backtick
> and
> > > the need to escape it becomes context-dependent.
> > >
> > > 2. In the example table near the beginning of the KIP, the name
> > "a.`b\`.c`"
> > > refers to "a" -> "b`c". What happened to the dot in the second part of
> > the
> > > name? Should it refer to "a" -> "b`.c" instead?
> > >
> > > 3. Is it ever necessary to escape backslashes themselves? If so, when?
> > >
> > > Overall, I wish we could come up with a prettier/simpler approach, but
> > the
> > > benefits provided by the dual backtick/dot syntax are too great to
> deny:
> > > there are no correctness issues like the ones posed with double-dot
> > > escaping that would lead to ambiguity, the most common cases are still
> > very
> > > simple to work with, and there's no risk of interfering with JSON
> escape
> > > mechanisms (in most cases) or single-quote shell quoting (which may be
> > > relevant when connector configurations are defined on the command
> line).
> > > Thanks for the suggestion, Robert!
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> >
>

Re: Re: [DISCUSS] KIP-821: Connect Transforms support for nested structures

[Chris Egerton <ch...@aiven.io.INVALID>]

Hi Jorge,

One tiny nit, but LGTM otherwise:

The KIP mentions backslashes as "(/)"; shouldn't this be "(\)"?

I'll cast a +1 on the vote thread anyways; I'm sure this won't block us.

Cheers, and thanks for all your hard work on this!

Chris

On Thu, Sep 1, 2022 at 1:33 PM Jorge Esteban Quilcate Otoya <
quilcate.jorge@gmail.com> wrote:

> Hi Chris,
>
> Thanks for your feedback!
>
> 1. Yes, it will be context-dependent. I have added rules and scenarios to
> the nested notation to cover the happy path and edge cases. In short,
> backticks will be not be considered as part of the field name when they are
> wrapping a field name: first backtick at the beginning of the path or after
> a dot, and closing backtick before a dot or at the end of the path.
> If backticks happen to be in those positions, use backslash to escape them.
> 2. You're right, that's a typo. Fixing it.
> 3. I don't think so, I have added a scenario to clarify this.
>
> KIP is updated. Hopefully the rules and scenarios help to close any open
> gap. Let me know if you see any cases that is not considered to address it.
>
> Cheers,
> Jorge.
>
> On Wed, 31 Aug 2022 at 20:02, Chris Egerton <ch...@aiven.io.invalid>
> wrote:
>
> > Hi Robert and Jorge,
> >
> > I think the backtick/backslash proposal works, but I'm a little unclear
> on
> > some of the details:
> >
> > 1. Are backticks only given special treatment when they immediately
> follow
> > a non-escaped dot? E.g., "foo.b`ar.ba`z" would refer to "foo" -> "b`ar"
> ->
> > "ba`z" instead of "foo" -> "bar.baz"? Based on the example where the name
> > "a.b`.c" refers to "a" -> "b`" -> "c", it seems like this is the case,
> but
> > I'm worried this might cause confusion since the role of the backtick and
> > the need to escape it becomes context-dependent.
> >
> > 2. In the example table near the beginning of the KIP, the name
> "a.`b\`.c`"
> > refers to "a" -> "b`c". What happened to the dot in the second part of
> the
> > name? Should it refer to "a" -> "b`.c" instead?
> >
> > 3. Is it ever necessary to escape backslashes themselves? If so, when?
> >
> > Overall, I wish we could come up with a prettier/simpler approach, but
> the
> > benefits provided by the dual backtick/dot syntax are too great to deny:
> > there are no correctness issues like the ones posed with double-dot
> > escaping that would lead to ambiguity, the most common cases are still
> very
> > simple to work with, and there's no risk of interfering with JSON escape
> > mechanisms (in most cases) or single-quote shell quoting (which may be
> > relevant when connector configurations are defined on the command line).
> > Thanks for the suggestion, Robert!
> >
> > Cheers,
> >
> > Chris
> >
>