You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Erik Goldman <er...@hunters.ai> on 2022/06/01 13:43:36 UTC
Help adding Snowflake variant syntax
I'm Erik <https://www.linkedin.com/in/erik-goldman/> and I'm currently
working on a project that requires a patch to Calcite as described in this
email <https://www.mail-archive.com/dev@calcite.apache.org/msg17417.html> (a
related Jira task is here
<https://issues.apache.org/jira/browse/CALCITE-4919>)
My current work is here
<https://github.com/ErikGoldman/calcite-snowflake/commits/snowflake-progress>
but
it needs much more work and understanding of the system to be successful.
I'm very early to Calcite and I think it would be better to work with
someone who understands the system before doing more work on this.
Would anyone be open to offering some advice or helping with the work? I am
happy to pay a competitive hourly rate for the assistance.
Thank you!
Re: Help adding Snowflake variant syntax
Posted by Walaa Eldin Moustafa <wa...@gmail.com>.
Hi Julian,
Thanks for asking those questions! I was providing this material as an
educational reference since the use case is very similar to the resources I
shared. I found those references to help enrich the Calcite community in
the past. For example, Gavin Ray has written an article [1] on converting
GraphQL to Calcite RelNodes after sharing similar resources, which I think
is a pretty nice contribution to Calcite (while not implemented in either
code base).
I might not be the best person to answer the IP question since there is
dedicated staff to address this type of questions at LinkedIn, but I'm
happy to route that question to the appropriate people.
In addition to the IP aspect, forking might not be productive since Coral
is quickly evolving, especially around its core APIs and workflows to
translate between input and output dialects, but I am sure there are ways
to achieve mutual leverage, starting from sharing knowledge and experiences.
[1] https://datalore.jetbrains.com/view/notebook/JYTVfn90xYSmv6U5f2NIQR
Thanks,
Walaa.
On Wed, Jun 1, 2022 at 12:36 PM Julian Hyde <jh...@gmail.com> wrote:
> Let’s suppose that there is code/tests in Coral would allow Calcite to
> implement this feature. Under the terms of Coral’s license (BSD2) some
> COULD copy code from Coral to Calcite, but SHOULD they?
>
> As an Apache project, and as individuals participating in an open source
> community beyond the ASF, we must be respectful of other projects’ IP and
> the “do not fork” principle.
>
> I would love to hear from Walaa or other Coral committers suggestions how
> we can share functionality between the projects.
>
> Julian
>
>
> > On Jun 1, 2022, at 7:47 AM, Walaa Eldin Moustafa <wa...@gmail.com>
> wrote:
> >
> > Hi Erik,
> >
> > Coral [1, 2] which is based on Calcite might be a good starting point for
> > this type of work.There are a couple of dialects it supports right now:
> > Hive (or Spark) and Trino (or Presto). You can check out the
> corresponding
> > test cases [3, 4] for these two dialects. coral-common [5] is a module
> that
> > contains all Calcite-related infra required to support any dialect. It is
> > extended by both coral-hive and coral-trino.
> >
> > [1] https://github.com/linkedin/coral
> > [2] https://engineering.linkedin.com/blog/2020/coral
> > [3]
> >
> https://github.com/linkedin/coral/blob/master/coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/HiveToRelConverterTest.java
> > [4]
> >
> https://github.com/linkedin/coral/blob/master/coral-trino/src/test/java/com/linkedin/coral/trino/trino2rel/TrinoToRelConverterTest.java
> > [5] https://github.com/linkedin/coral/tree/master/coral-common
> >
> > Thanks,
> > Walaa.
> >
> >
> > On Wed, Jun 1, 2022 at 6:44 AM Erik Goldman <er...@hunters.ai>
> wrote:
> >
> >> I'm Erik <https://www.linkedin.com/in/erik-goldman/> and I'm currently
> >> working on a project that requires a patch to Calcite as described in
> this
> >> email <
> https://www.mail-archive.com/dev@calcite.apache.org/msg17417.html>
> >> (a
> >> related Jira task is here
> >> <https://issues.apache.org/jira/browse/CALCITE-4919>)
> >>
> >> My current work is here
> >> <
> >>
> https://github.com/ErikGoldman/calcite-snowflake/commits/snowflake-progress
> >>>
> >> but
> >> it needs much more work and understanding of the system to be
> successful.
> >>
> >> I'm very early to Calcite and I think it would be better to work with
> >> someone who understands the system before doing more work on this.
> >>
> >> Would anyone be open to offering some advice or helping with the work?
> I am
> >> happy to pay a competitive hourly rate for the assistance.
> >>
> >> Thank you!
> >>
>
>
Re: Help adding Snowflake variant syntax
Posted by Julian Hyde <jh...@gmail.com>.
Let’s suppose that there is code/tests in Coral would allow Calcite to implement this feature. Under the terms of Coral’s license (BSD2) some COULD copy code from Coral to Calcite, but SHOULD they?
As an Apache project, and as individuals participating in an open source community beyond the ASF, we must be respectful of other projects’ IP and the “do not fork” principle.
I would love to hear from Walaa or other Coral committers suggestions how we can share functionality between the projects.
Julian
> On Jun 1, 2022, at 7:47 AM, Walaa Eldin Moustafa <wa...@gmail.com> wrote:
>
> Hi Erik,
>
> Coral [1, 2] which is based on Calcite might be a good starting point for
> this type of work.There are a couple of dialects it supports right now:
> Hive (or Spark) and Trino (or Presto). You can check out the corresponding
> test cases [3, 4] for these two dialects. coral-common [5] is a module that
> contains all Calcite-related infra required to support any dialect. It is
> extended by both coral-hive and coral-trino.
>
> [1] https://github.com/linkedin/coral
> [2] https://engineering.linkedin.com/blog/2020/coral
> [3]
> https://github.com/linkedin/coral/blob/master/coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/HiveToRelConverterTest.java
> [4]
> https://github.com/linkedin/coral/blob/master/coral-trino/src/test/java/com/linkedin/coral/trino/trino2rel/TrinoToRelConverterTest.java
> [5] https://github.com/linkedin/coral/tree/master/coral-common
>
> Thanks,
> Walaa.
>
>
> On Wed, Jun 1, 2022 at 6:44 AM Erik Goldman <er...@hunters.ai> wrote:
>
>> I'm Erik <https://www.linkedin.com/in/erik-goldman/> and I'm currently
>> working on a project that requires a patch to Calcite as described in this
>> email <https://www.mail-archive.com/dev@calcite.apache.org/msg17417.html>
>> (a
>> related Jira task is here
>> <https://issues.apache.org/jira/browse/CALCITE-4919>)
>>
>> My current work is here
>> <
>> https://github.com/ErikGoldman/calcite-snowflake/commits/snowflake-progress
>>>
>> but
>> it needs much more work and understanding of the system to be successful.
>>
>> I'm very early to Calcite and I think it would be better to work with
>> someone who understands the system before doing more work on this.
>>
>> Would anyone be open to offering some advice or helping with the work? I am
>> happy to pay a competitive hourly rate for the assistance.
>>
>> Thank you!
>>
Re: Help adding Snowflake variant syntax
Posted by Walaa Eldin Moustafa <wa...@gmail.com>.
Hi Erik,
Coral [1, 2] which is based on Calcite might be a good starting point for
this type of work.There are a couple of dialects it supports right now:
Hive (or Spark) and Trino (or Presto). You can check out the corresponding
test cases [3, 4] for these two dialects. coral-common [5] is a module that
contains all Calcite-related infra required to support any dialect. It is
extended by both coral-hive and coral-trino.
[1] https://github.com/linkedin/coral
[2] https://engineering.linkedin.com/blog/2020/coral
[3]
https://github.com/linkedin/coral/blob/master/coral-hive/src/test/java/com/linkedin/coral/hive/hive2rel/HiveToRelConverterTest.java
[4]
https://github.com/linkedin/coral/blob/master/coral-trino/src/test/java/com/linkedin/coral/trino/trino2rel/TrinoToRelConverterTest.java
[5] https://github.com/linkedin/coral/tree/master/coral-common
Thanks,
Walaa.
On Wed, Jun 1, 2022 at 6:44 AM Erik Goldman <er...@hunters.ai> wrote:
> I'm Erik <https://www.linkedin.com/in/erik-goldman/> and I'm currently
> working on a project that requires a patch to Calcite as described in this
> email <https://www.mail-archive.com/dev@calcite.apache.org/msg17417.html>
> (a
> related Jira task is here
> <https://issues.apache.org/jira/browse/CALCITE-4919>)
>
> My current work is here
> <
> https://github.com/ErikGoldman/calcite-snowflake/commits/snowflake-progress
> >
> but
> it needs much more work and understanding of the system to be successful.
>
> I'm very early to Calcite and I think it would be better to work with
> someone who understands the system before doing more work on this.
>
> Would anyone be open to offering some advice or helping with the work? I am
> happy to pay a competitive hourly rate for the assistance.
>
> Thank you!
>