Ask Advices on Calcite Streaming SQL

["邓小勇(静行)" <xi...@alibaba-inc.com>]

Hi there,

We're working on a streaming engine used internally at Alibaba, and would like to support some sort of streaming SQL on top of it.

The current plan is to follow Calcite Streaming SQL, though we might need to extend the current implementation in certain ways (e.g., allowing creating streams connecting to external sources/sink, multi-insert, the support for multiple statements in a script, different kinds of window functions, etc).

Since we'd like to have it ready for production use within the next 3-6 months, I'm wondering if you have any suggestions or comments on how we should proceed. Thank you very much!

Best,
Xiaoyong
Alibaba Cloud

Re: Ask Advices on Calcite Streaming SQL

["钱正平(布民)" <zh...@alibaba-inc.com>]

Hi Albert,

 

Thanks a lot for your reply.  Sorry for that we were not very clear.

 

We have studied the document very carefully and have sort of come up with a plan.  It includes things to add to the language for our scenarios, the runtime (= operator implementations) and how things should be put together with our existing engine…

 

As this is for real production use and the current spec seems to be still evolving, it would be of great help to have discussions on various parts with the community during the course.  Also, we are very happy to contribute some of our code back if possible.

 

One constraint is that we have a fixed/tight schedule, and so could have to move quite fast.  I’m wondering if you have any advice for us in this case, say on how to make that happen if possible.

 

Thank you!

 

Best,

Zhengping

 

From: Albert <zi...@gmail.com>
Date: Tuesday, December 13, 2016 at 10:10 AM
To: <de...@calcite.apache.org>, "邓小勇(静行)" <xi...@alibaba-inc.com>
Cc: "钱正平(布民)" <zh...@alibaba-inc.com>, 巴真 <ba...@taobao.com>, "洪鲛(洪鲛)" <ho...@alibaba-inc.com>
Subject: Re: Ask Advices on Calcite Streaming SQL

 

Hi,

    the question seems too broad and not well framed. but in general if you want to understand more about streaming within `calcite`.

 

- Firstly:  The document  

- Secondly: Julian also led several discussions in the dev thread, you might be able to found them within mail archive. 

- Last but not least: code is always there for you to reference. I suggest you start from the CSV adapter, you might have a clearer view regarding what to proceed after you look into that. 

 

anyways, discussion is always welcomed here. 

 

 

 

On Mon, Dec 12, 2016 at 5:38 PM, 邓小勇(静行) <xi...@alibaba-inc.com> wrote:

Hi there,

We're working on a streaming engine used internally at Alibaba, and would like to support some sort of streaming SQL on top of it.

The current plan is to follow Calcite Streaming SQL, though we might need to extend the current implementation in certain ways (e.g., allowing creating streams connecting to external sources/sink, multi-insert, the support for multiple statements in a script, different kinds of window functions, etc).

Since we'd like to have it ready for production use within the next 3-6 months, I'm wondering if you have any suggestions or comments on how we should proceed. Thank you very much!

Best,
Xiaoyong
Alibaba Cloud



 

-- 

~~~~~~~~~~~~~~~
no mistakes
~~~~~~~~~~~~~~~~~~

Re: Ask Advices on Calcite Streaming SQL

[Albert <zi...@gmail.com>]

Hi,
    the question seems too broad and not well framed. but in general if you
want to understand more about streaming within `calcite`.

- Firstly:  The document <https://calcite.apache.org/docs/stream.html>
- Secondly: Julian also led several discussions in the dev thread, you
might be able to found them within mail archive.
- Last but not least: code is always there for you to reference. I suggest
you start from the CSV adapter, you might have a clearer view regarding
what to proceed after you look into that.

anyways, discussion is always welcomed here.



On Mon, Dec 12, 2016 at 5:38 PM, 邓小勇(静行) <xi...@alibaba-inc.com>
wrote:

> Hi there,
>
> We're working on a streaming engine used internally at
> Alibaba, and would like to support some sort of
> streaming SQL on top of it.
>
> The current plan is to follow Calcite Streaming SQL, though
> we might need to extend the current implementation in
> certain ways (e.g., allowing creating streams connecting
> to external sources/sink, multi-insert, the support for
> multiple statements in a script, different kinds of window functions,
> etc).
>
> Since we'd like to have it ready for production use
> within the next 3-6 months, I'm wondering if you have any
> suggestions or comments on how we should proceed. Thank you very much!
>
> Best,
> Xiaoyong
> Alibaba Cloud




-- 
~~~~~~~~~~~~~~~
no mistakes
~~~~~~~~~~~~~~~~~~