You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Lai Zhou (JIRA)" <ji...@apache.org> on 2019/05/16 02:57:00 UTC
[jira] [Updated] (CALCITE-3071) Cache the whole sql plan to reduce
the latency and improve the performance
[ https://issues.apache.org/jira/browse/CALCITE-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lai Zhou updated CALCITE-3071:
------------------------------
Description:
In real business, when sql queries become complex, the overhead of sql plan will increase quickly , and many of the sql queries are duplicates.
We already have some caching issue about improving the performance, such as the issue
https://issues.apache.org/jira/browse/CALCITE-2703,
which reduce code generation and class loading overhead when executing queries in the EnumerableConvention, but I think it's not enough.
I propose to cache the whole sql plan to reduce the latency ，for the same sql
, ignoring the cost optimizing based on statistics here, we can cache the generated code for it.
I use the FrameworkConfig API to execute sql queries, in this way I can easily do this job .
but it's not easy to make a whole sql execution plan(that says code-gen) cache in the sql processing flow based on JDBC Connection, because there're many intermediate state in this processing flow.
Let's discuss this feature and the probable solutions.
was:
In real business, when sql queries become complex, the overhead of sql plan will increase quickly , and many of the sql queries are duplicates.
We already do something on cacheing to improve the performance, such as the issue
https://issues.apache.org/jira/browse/CALCITE-2703,
which reduce code generation and class loading overhead when executing queries in the EnumerableConvention, but I think it's not enough.
I propose to cache the whole sql plan to reduce the latency ，for the same sql
, ignoring the cost optimizing based on statistics here, we can cache the generated code for it.
I use the FrameworkConfig API to execute sql queries, in this way I can easily do this job .
but it's not easy to make a whole sql execution plan(that says code-gen) cache in the sql processing flow based on JDBC Connection, because there're many intermediate state in this processing flow.
Let's discuss this feature and the probable solutions.
> Cache the whole sql plan to reduce the latency and improve the performance
> --------------------------------------------------------------------------
>
> Key: CALCITE-3071
> URL: https://issues.apache.org/jira/browse/CALCITE-3071
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.19.0
> Reporter: Lai Zhou
> Priority: Major
>
> In real business, when sql queries become complex, the overhead of sql plan will increase quickly , and many of the sql queries are duplicates.
> We already have some caching issue about improving the performance, such as the issue
> https://issues.apache.org/jira/browse/CALCITE-2703,
> which reduce code generation and class loading overhead when executing queries in the EnumerableConvention, but I think it's not enough.
> I propose to cache the whole sql plan to reduce the latency ，for the same sql
> , ignoring the cost optimizing based on statistics here, we can cache the generated code for it.
> I use the FrameworkConfig API to execute sql queries, in this way I can easily do this job .
> but it's not easy to make a whole sql execution plan(that says code-gen) cache in the sql processing flow based on JDBC Connection, because there're many intermediate state in this processing flow.
>
> Let's discuss this feature and the probable solutions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)