First steps: Beam SQL over ProtoBuf

[Alex Van Boxel <al...@vanboxel.be>]

Hi all,

I got my first test pipeline running with *Beam SQL over ProtoBuf*. I was
so excited I need to shout this to the world ;-)

https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java

A lot of work is planned but this is the first step. If you're invested in
ProtoBuf and use Beam, follow the repo:
https://github.com/anemos-io/proto-beam
<https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java>

 _/
_/ Alex Van Boxel

Re: First steps: Beam SQL over ProtoBuf

[Alex Van Boxel <al...@gmail.com>]

I'm open to contributing the implementation back to Beam. But for now I'm
going to keep it in a separate repo because then the iteration can go very
fast, and no low level API changes are required anyway. The reason it needs
to go fast is that we need it internally in the company.

But I will certainly keep you all posted and then we can discuss the how
and what to contribute back

On Fri, Feb 1, 2019, 20:44 Anton Kedin <kedin@google.com wrote:

> I think this is awesome!
>
> When you feel it is in a good state please consider contributing it to
> Beam, I think it would be a valuable addition.
>
> Thank you,
> Anton
>
> On Fri, Feb 1, 2019 at 9:59 AM Alex Van Boxel <al...@vanboxel.be> wrote:
>
>> Hi all,
>>
>> I got my first test pipeline running with *Beam SQL over ProtoBuf*. I
>> was so excited I need to shout this to the world ;-)
>>
>>
>> https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java
>>
>> A lot of work is planned but this is the first step. If you're invested
>> in ProtoBuf and use Beam, follow the repo:
>> https://github.com/anemos-io/proto-beam
>> <https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java>
>>
>>  _/
>> _/ Alex Van Boxel
>>
>

Re: First steps: Beam SQL over ProtoBuf

[Anton Kedin <ke...@google.com>]

I think this is awesome!

When you feel it is in a good state please consider contributing it to
Beam, I think it would be a valuable addition.

Thank you,
Anton

On Fri, Feb 1, 2019 at 9:59 AM Alex Van Boxel <al...@vanboxel.be> wrote:

> Hi all,
>
> I got my first test pipeline running with *Beam SQL over ProtoBuf*. I was
> so excited I need to shout this to the world ;-)
>
>
> https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java
>
> A lot of work is planned but this is the first step. If you're invested in
> ProtoBuf and use Beam, follow the repo:
> https://github.com/anemos-io/proto-beam
> <https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java>
>
>  _/
> _/ Alex Van Boxel
>

Re: First steps: Beam SQL over ProtoBuf

[Rui Wang <ru...@google.com>]

It's awesome! Proto to/from Row will be a great utility!

-Rui

On Fri, Feb 1, 2019 at 9:59 AM Alex Van Boxel <al...@vanboxel.be> wrote:

> Hi all,
>
> I got my first test pipeline running with *Beam SQL over ProtoBuf*. I was
> so excited I need to shout this to the world ;-)
>
>
> https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java
>
> A lot of work is planned but this is the first step. If you're invested in
> ProtoBuf and use Beam, follow the repo:
> https://github.com/anemos-io/proto-beam
> <https://github.com/anemos-io/proto-beam/blob/master/transform/src/test/java/io/anemos/protobeam/transform/beamsql/BeamSqlPipelineTest.java>
>
>  _/
> _/ Alex Van Boxel
>