feature request / jdbc push-down / off-platform SQL rewrite

[Anthony Krinsky <dr...@gmail.com>]

Newbie question and perhaps feature request, apologies in advance if this
is well trod.

I've struggled to find a solution to these problems, which one would think
are common:

1) Full fidelity push-down to a JDBC adapted, traditional RDBMS, when only
that source is involved in a query.  Why not an optimization short-cut?

2) A plugin framework that would allow SQL rewrite outside of the
framework.  For example, I might have a redaction, row-level security, or
rewrite engine running on a separate application server.


I fully appreciate that these ideas are contrary to the spirit of a
cost-optimized, distributed and federated query middleware.  These
limitations exist in Trino and Calcite derivatives.

Thoughts?

Thank you so much,

Anthony

Re: feature request / jdbc push-down / off-platform SQL rewrite

[Stamatis Zampetakis <za...@gmail.com>]

Hi Anthony,

I don't completely understand what point 1 and 2 concretely mean.

1) Calcite does a pretty good job pushing computations to the underlying
DBMS when that is possible.
JdbcAdapterTest [1] has a few examples where the whole query is pushed to
the database.
Can you give more details on what is missing or what you wanted to achieve?

2) Calcite has many components which can be used independently from one
another.
If you use Calcite as a library then you can use whichever component you
need and perform rewrites exactly as you want.
Would you like to propose a new API for some particular use-case?

Best,
Stamatis

[1]
https://github.com/apache/calcite/blob/07e420bfc36e8f217f5c3459fdf6c1ede609cb80/core/src/test/java/org/apache/calcite/test/JdbcAdapterTest.java

On Fri, Nov 12, 2021 at 1:05 AM Anthony Krinsky <dr...@gmail.com>
wrote:

> Newbie question and perhaps feature request, apologies in advance if this
> is well trod.
>
> I've struggled to find a solution to these problems, which one would think
> are common:
>
> 1) Full fidelity push-down to a JDBC adapted, traditional RDBMS, when only
> that source is involved in a query.  Why not an optimization short-cut?
>
> 2) A plugin framework that would allow SQL rewrite outside of the
> framework.  For example, I might have a redaction, row-level security, or
> rewrite engine running on a separate application server.
>
>
> I fully appreciate that these ideas are contrary to the spirit of a
> cost-optimized, distributed and federated query middleware.  These
> limitations exist in Trino and Calcite derivatives.
>
> Thoughts?
>
> Thank you so much,
>
> Anthony
>