SQL support to Gearpump

[venkat lingala <ma...@gmail.com>]

Hello everyone,

I am venkat, I would like to work on adding SQL support for Gearpump for
this Google Summer of Code. Currently I am preparing my project proposal.

I am not pretty clear about converting logical plan of apache calcite in to
jobs on Gearpump?

Anyone has any ideas on this topic.

Looking forward to hear from you guys.

Regards,
Venkat

Re: SQL support to Gearpump

[Darion Yaphet <da...@gmail.com>]

Hi venkat:

SQL on Gearpump is very interesting .

I try to write a simple calcite example and found that each connection need
a Scheam structure to describe the input data type .

I'm learning Spark SQL running principle and It has a set of rules and
catalyst as a optimizer .



2017-03-25 22:47 GMT+08:00 venkat lingala <ma...@gmail.com>:

> Hello everyone,
>
> I am venkat, I would like to work on adding SQL support for Gearpump for
> this Google Summer of Code. Currently I am preparing my project proposal.
>
> I am not pretty clear about converting logical plan of apache calcite in to
> jobs on Gearpump?
>
> Anyone has any ideas on this topic.
>
> Looking forward to hear from you guys.
>
> Regards,
> Venkat
>



-- 

long is the way and hard  that out of Hell leads up to light

Re: SQL support to Gearpump

[Manu Zhang <ow...@gmail.com>]

Hi Venkat,

Thanks for your participation. This is Manu, mentoring this project.
Frankly, I'm new in this area myself and mostly I've been following how
other streaming framework has done so. For example,
https://github.com/milinda/samza-sql.

These incomplete codes,
https://github.com/manuzhang/incubator-gearpump/tree/sql/experiments/sql/src/main/scala/org/apache/gearpump/experiments/sql
are
what I've tried for this task some while ago, which could be a start point
for you.
  - In QueryExecutor.scala
<https://github.com/manuzhang/incubator-gearpump/blob/sql/experiments/sql/src/main/scala/org/apache/gearpump/experiments/sql/QueryExecutor.scala>,
you can see how a sql string is parsed and translated into tree of
`RelNode` in Calcite.
  - StreamRel
<https://github.com/manuzhang/incubator-gearpump/blob/sql/experiments/sql/src/main/scala/org/apache/gearpump/experiments/sql/plan/physical/StreamRel.scala>
and
its subclass should `translate` the node into corresponding Gearpump
`Stream`.

Looking forward to your proposal ;)

Thanks,
Manu Zhang



On Sat, Mar 25, 2017 at 10:47 PM venkat lingala <ma...@gmail.com>
wrote:

Hello everyone,

I am venkat, I would like to work on adding SQL support for Gearpump for
this Google Summer of Code. Currently I am preparing my project proposal.

I am not pretty clear about converting logical plan of apache calcite in to
jobs on Gearpump?

Anyone has any ideas on this topic.

Looking forward to hear from you guys.

Regards,
Venkat