You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Austin Bennett <wh...@gmail.com> on 2020/11/02 02:42:37 UTC
BeamSQL and Beam equivalent -- examples?
Hi All,
For something I am currently writing -- I am seeking any examples of
BeamSQL and Beam that take the same input and produce the same output. I
can't recall, off head, any examples/slides/writeups. Do any exist?
I would like to show:
(a) that BeamSQL is a real thing :-)
(b) that Beam can express the same as BeamSQL
(c) that Beam can be more expressive than just SQL concepts.
Imagining such examples can help with points a and b.
Thanks,
Austin
Re: BeamSQL and Beam equivalent -- examples?
Posted by Rui Wang <ru...@google.com>.
+1 to the Nexmark query 0 as it has two implementations for both Beam and
BeamSQL.
For (c), I think a good example will be stateful API. The state concept and
manipulation on states are not a concept of SQL.
-Rui
On Mon, Nov 2, 2020 at 9:25 AM Brian Hulette <bh...@google.com> wrote:
> Those examples may help with (c) as well, it's notable that there are
> fewer SqlQuery* implementations than Query*.
>
> Brian
>
> On Mon, Nov 2, 2020 at 2:00 AM Kamil Wasilewski <
> kamil.wasilewski@polidea.com> wrote:
>
>> Hi Austin,
>>
>> Did you take a look at Nexmark tests? Some of them have two versions:
>> Beam and BeamSQL. It sounds like this is what you are looking for.
>> For example:
>>
>>
>> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
>>
>> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
>>
>> Kamil
>>
>>
>> On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <
>> whatwouldaustindo@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> For something I am currently writing -- I am seeking any examples of
>>> BeamSQL and Beam that take the same input and produce the same output. I
>>> can't recall, off head, any examples/slides/writeups. Do any exist?
>>>
>>> I would like to show:
>>>
>>> (a) that BeamSQL is a real thing :-)
>>> (b) that Beam can express the same as BeamSQL
>>> (c) that Beam can be more expressive than just SQL concepts.
>>>
>>> Imagining such examples can help with points a and b.
>>>
>>> Thanks,
>>> Austin
>>>
>>
Re: BeamSQL and Beam equivalent -- examples?
Posted by Rui Wang <ru...@google.com>.
+1 to the Nexmark query 0 as it has two implementations for both Beam and
BeamSQL.
For (c), I think a good example will be stateful API. The state concept and
manipulation on states are not a concept of SQL.
-Rui
On Mon, Nov 2, 2020 at 9:25 AM Brian Hulette <bh...@google.com> wrote:
> Those examples may help with (c) as well, it's notable that there are
> fewer SqlQuery* implementations than Query*.
>
> Brian
>
> On Mon, Nov 2, 2020 at 2:00 AM Kamil Wasilewski <
> kamil.wasilewski@polidea.com> wrote:
>
>> Hi Austin,
>>
>> Did you take a look at Nexmark tests? Some of them have two versions:
>> Beam and BeamSQL. It sounds like this is what you are looking for.
>> For example:
>>
>>
>> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
>>
>> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
>>
>> Kamil
>>
>>
>> On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <
>> whatwouldaustindo@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> For something I am currently writing -- I am seeking any examples of
>>> BeamSQL and Beam that take the same input and produce the same output. I
>>> can't recall, off head, any examples/slides/writeups. Do any exist?
>>>
>>> I would like to show:
>>>
>>> (a) that BeamSQL is a real thing :-)
>>> (b) that Beam can express the same as BeamSQL
>>> (c) that Beam can be more expressive than just SQL concepts.
>>>
>>> Imagining such examples can help with points a and b.
>>>
>>> Thanks,
>>> Austin
>>>
>>
Re: BeamSQL and Beam equivalent -- examples?
Posted by Brian Hulette <bh...@google.com>.
Those examples may help with (c) as well, it's notable that there are fewer
SqlQuery* implementations than Query*.
Brian
On Mon, Nov 2, 2020 at 2:00 AM Kamil Wasilewski <
kamil.wasilewski@polidea.com> wrote:
> Hi Austin,
>
> Did you take a look at Nexmark tests? Some of them have two versions: Beam
> and BeamSQL. It sounds like this is what you are looking for. For example:
>
>
> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
>
> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
>
> Kamil
>
>
> On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <wh...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> For something I am currently writing -- I am seeking any examples of
>> BeamSQL and Beam that take the same input and produce the same output. I
>> can't recall, off head, any examples/slides/writeups. Do any exist?
>>
>> I would like to show:
>>
>> (a) that BeamSQL is a real thing :-)
>> (b) that Beam can express the same as BeamSQL
>> (c) that Beam can be more expressive than just SQL concepts.
>>
>> Imagining such examples can help with points a and b.
>>
>> Thanks,
>> Austin
>>
>
Re: BeamSQL and Beam equivalent -- examples?
Posted by Brian Hulette <bh...@google.com>.
Those examples may help with (c) as well, it's notable that there are fewer
SqlQuery* implementations than Query*.
Brian
On Mon, Nov 2, 2020 at 2:00 AM Kamil Wasilewski <
kamil.wasilewski@polidea.com> wrote:
> Hi Austin,
>
> Did you take a look at Nexmark tests? Some of them have two versions: Beam
> and BeamSQL. It sounds like this is what you are looking for. For example:
>
>
> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
>
> https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
>
> Kamil
>
>
> On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <wh...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> For something I am currently writing -- I am seeking any examples of
>> BeamSQL and Beam that take the same input and produce the same output. I
>> can't recall, off head, any examples/slides/writeups. Do any exist?
>>
>> I would like to show:
>>
>> (a) that BeamSQL is a real thing :-)
>> (b) that Beam can express the same as BeamSQL
>> (c) that Beam can be more expressive than just SQL concepts.
>>
>> Imagining such examples can help with points a and b.
>>
>> Thanks,
>> Austin
>>
>
Re: BeamSQL and Beam equivalent -- examples?
Posted by Kamil Wasilewski <ka...@polidea.com>.
Hi Austin,
Did you take a look at Nexmark tests? Some of them have two versions: Beam
and BeamSQL. It sounds like this is what you are looking for. For example:
https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
Kamil
On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <wh...@gmail.com>
wrote:
> Hi All,
>
> For something I am currently writing -- I am seeking any examples of
> BeamSQL and Beam that take the same input and produce the same output. I
> can't recall, off head, any examples/slides/writeups. Do any exist?
>
> I would like to show:
>
> (a) that BeamSQL is a real thing :-)
> (b) that Beam can express the same as BeamSQL
> (c) that Beam can be more expressive than just SQL concepts.
>
> Imagining such examples can help with points a and b.
>
> Thanks,
> Austin
>
Re: BeamSQL and Beam equivalent -- examples?
Posted by Kamil Wasilewski <ka...@polidea.com>.
Hi Austin,
Did you take a look at Nexmark tests? Some of them have two versions: Beam
and BeamSQL. It sounds like this is what you are looking for. For example:
https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/Query0.java
https://github.com/kamilwu/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery0.java
Kamil
On Mon, Nov 2, 2020 at 3:42 AM Austin Bennett <wh...@gmail.com>
wrote:
> Hi All,
>
> For something I am currently writing -- I am seeking any examples of
> BeamSQL and Beam that take the same input and produce the same output. I
> can't recall, off head, any examples/slides/writeups. Do any exist?
>
> I would like to show:
>
> (a) that BeamSQL is a real thing :-)
> (b) that Beam can express the same as BeamSQL
> (c) that Beam can be more expressive than just SQL concepts.
>
> Imagining such examples can help with points a and b.
>
> Thanks,
> Austin
>