[jira] [Commented] (CALCITE-4568) Tempura: extending Calcite into an incremental query optimizer

["Julian Hyde (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-4568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316765#comment-17316765 ] 

Julian Hyde commented on CALCITE-4568:
--------------------------------------

The code is in [alibaba/cost-based-incremental-optimizer|https://github.com/alibaba/cost-based-incremental-optimizer].

I would ask Calcite committers to review the high-level design rather than the lines of code. (For example, what capabilities are missing, and would need to be added before this is a generally usable mode of operation for Calcite?)

> Tempura: extending Calcite into an incremental query optimizer
> --------------------------------------------------------------
>
>                 Key: CALCITE-4568
>                 URL: https://issues.apache.org/jira/browse/CALCITE-4568
>             Project: Calcite
>          Issue Type: New Feature
>            Reporter: Botong Huang
>            Priority: Major
>
> As discussed in the email thread, this is an attempt to extend the Calcite optimizer into a general incremental query optimizer, based on our research paper published in VLDB 2021:
> Tempura: a general cost-based optimizer framework for incremental data processing
> To our best knowledge, this is the first general cost-based incremental optimizer that can find the best plan across multiple families of incremental computing methods, including IVM, Streaming, DBToaster, etc. Experiments (in the paper) shows that the generated best plan is consistently much better than the plans from each individual method alone.
> In general, incremental query planning is central to database view maintenance and stream processing systems, and are being adopted in active databases, resumable query execution, approximate query processing, etc. We are hoping that this feature can help widening the spectrum of Calcite, solicit more use cases and adoption of Calcite.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)