You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Forward Xu <fo...@gmail.com> on 2020/01/08 13:44:02 UTC
[discuss] Implement Presto adapter
Hi everybody,
I'd like to kick off a discussion on Implement Presto adapter in calcite.
We know that presto is also a powerful SQL query tool, which is widely used
with the olap of big data. So I want to add an adapter for presto to calite.
Would love to hear your thoughts.
Best,
Forward
Re: [discuss] Implement Elasticsearch jdbc driver
Posted by Michael Mior <mm...@apache.org>.
Is there any reason you can't use Calcite's existing JDBC driver for
this purpose?
--
Michael Mior
mmior@apache.org
Le jeu. 7 mai 2020 à 21:40, Forward Xu <fo...@gmail.com> a écrit :
>
> Hi everybody,
>
> I'd like to kick off a discussion on Implement Elasticsearch jdbc driver in
> calcite.
>
> Calcite has already implemented the elasticsearch adapter. I want to add
> standard jdbc driver support based on the adapter. The advantage is that it
> can be more easily used for visualization through the jdbc driver-based bi
> system. It can also enhance the DDL and DML of the current elasticsearch
> adapter. Implement es jdbc driver similar to aws open source [1]
>
> Would love to hear your thoughts.
>
> Best,
> Forward
>
> [1]https://github.com/opendistro-for-elasticsearch/sql
[discuss] Implement Elasticsearch jdbc driver
Posted by Forward Xu <fo...@gmail.com>.
Hi everybody,
I'd like to kick off a discussion on Implement Elasticsearch jdbc driver in
calcite.
Calcite has already implemented the elasticsearch adapter. I want to add
standard jdbc driver support based on the adapter. The advantage is that it
can be more easily used for visualization through the jdbc driver-based bi
system. It can also enhance the DDL and DML of the current elasticsearch
adapter. Implement es jdbc driver similar to aws open source [1]
Would love to hear your thoughts.
Best,
Forward
[1]https://github.com/opendistro-for-elasticsearch/sql
Re: [discuss] Implement Presto adapter
Posted by Julian Hyde <jh...@apache.org>.
I see. You intend to generate SQL. So this would be a dialect for the
JDBC adapter.
On Thu, Jan 9, 2020 at 10:50 PM Forward Xu <fo...@gmail.com> wrote:
>
> Hi Julian,
>
> In addition, we can also implement a PrestoSqlDialect.
>
>
> Best,
>
> Forward
>
> Forward Xu <fo...@gmail.com> 于2020年1月10日周五 下午2:23写道：
>
> > Hi Julian,
> >
> > The current idea is through com.facebook.presto.spi.
> > 1. Can be achieved through plugs provided by presto. For example [1].
> > 2. You can construct Connector directly through
> > com.facebook.presto.spi.ConnectorFactory [2].
> >
> > Best,
> > Forward
> >
> > [1]
> > https://www.codota.com/code/java/methods/com.facebook.presto.spi.Plugin/getConnectorFactories
> > [2]
> > https://www.programcreek.com/java-api-examples/?code=y-lan/presto/presto-master/presto-raptor/src/main/java/com/facebook/presto/raptor/RaptorPlugin.java
> >
> > Julian Hyde <jh...@apache.org> 于2020年1月10日周五 上午2:03写道：
> >
> >> Sounds like you are planning to target Presto’s internal engine rather
> >> than Presto the database. To clarify, can you give an example of the code
> >> that you would generate and send to Presto for a simple SQL query?
> >>
> >> Julian
> >>
> >>
> >> > On Jan 9, 2020, at 5:07 AM, Forward Xu <fo...@gmail.com> wrote:
> >> >
> >> > Hi, Stamatis
> >> >
> >> > Because we have a number of internal clusters already using and
> >> installing
> >> > presto. And we want use calcite to optimize the parser and push it down
> >> to
> >> > presto, spark and other calculation engines or olaps. Utilize the sql
> >> > optimization function of calite in combination with distributed
> >> computing
> >> > engines or distributed olaps such as presto. In this way, each
> >> performance
> >> > can be exerted.
> >> >
> >> > Best,
> >> > Forward
> >> >
> >> > Stamatis Zampetakis <za...@gmail.com> 于2020年1月9日周四 下午8:51写道：
> >> >
> >> >> Hi,
> >> >>
> >> >> Presto seems to have many similarities with Calcite itself.
> >> >> As you mentioned, Presto is not really a database but a distributed SQL
> >> >> framework to query heterogeneous data sources.
> >> >> Presto connectors are in some sense similar to Calcite adapters and if
> >> the
> >> >> Enumerable convention was capable of performing distributed joins etc.,
> >> >> then the similarities would be even bigger.
> >> >>
> >> >> To the best of my knowledge, the majority of our users adopt Calcite
> >> either
> >> >> to query multiple data sources in a unified way or to provide SQL query
> >> >> processing capabilities to data sources that do not already have.
> >> Presto
> >> >> seems to have similar goals, thus, I am not sure who will benefit from
> >> this
> >> >> new adapter.
> >> >>
> >> >> I never used Presto myself so I may have missed some important points.
> >> Can
> >> >> you share some use-cases about who may use this adapter and how?
> >> >>
> >> >> Best,
> >> >> Stamatis
> >> >>
> >> >>
> >> >> On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com>
> >> wrote:
> >> >>
> >> >>> Hi everybody,
> >> >>>
> >> >>> I'd like to kick off a discussion on Implement Presto adapter in
> >> calcite.
> >> >>>
> >> >>> We know that presto is also a powerful SQL query tool, which is widely
> >> >> used
> >> >>> with the olap of big data. So I want to add an adapter for presto to
> >> >>> calite.
> >> >>>
> >> >>> Would love to hear your thoughts.
> >> >>>
> >> >>> Best,
> >> >>> Forward
> >> >>>
> >> >>
> >>
> >>
Re: [discuss] Implement Presto adapter
Posted by Forward Xu <fo...@gmail.com>.
Hi Julian,
In addition, we can also implement a PrestoSqlDialect.
Best,
Forward
Forward Xu <fo...@gmail.com> 于2020年1月10日周五 下午2:23写道：
> Hi Julian,
>
> The current idea is through com.facebook.presto.spi.
> 1. Can be achieved through plugs provided by presto. For example [1].
> 2. You can construct Connector directly through
> com.facebook.presto.spi.ConnectorFactory [2].
>
> Best,
> Forward
>
> [1]
> https://www.codota.com/code/java/methods/com.facebook.presto.spi.Plugin/getConnectorFactories
> [2]
> https://www.programcreek.com/java-api-examples/?code=y-lan/presto/presto-master/presto-raptor/src/main/java/com/facebook/presto/raptor/RaptorPlugin.java
>
> Julian Hyde <jh...@apache.org> 于2020年1月10日周五 上午2:03写道：
>
>> Sounds like you are planning to target Presto’s internal engine rather
>> than Presto the database. To clarify, can you give an example of the code
>> that you would generate and send to Presto for a simple SQL query?
>>
>> Julian
>>
>>
>> > On Jan 9, 2020, at 5:07 AM, Forward Xu <fo...@gmail.com> wrote:
>> >
>> > Hi, Stamatis
>> >
>> > Because we have a number of internal clusters already using and
>> installing
>> > presto. And we want use calcite to optimize the parser and push it down
>> to
>> > presto, spark and other calculation engines or olaps. Utilize the sql
>> > optimization function of calite in combination with distributed
>> computing
>> > engines or distributed olaps such as presto. In this way, each
>> performance
>> > can be exerted.
>> >
>> > Best,
>> > Forward
>> >
>> > Stamatis Zampetakis <za...@gmail.com> 于2020年1月9日周四 下午8:51写道：
>> >
>> >> Hi,
>> >>
>> >> Presto seems to have many similarities with Calcite itself.
>> >> As you mentioned, Presto is not really a database but a distributed SQL
>> >> framework to query heterogeneous data sources.
>> >> Presto connectors are in some sense similar to Calcite adapters and if
>> the
>> >> Enumerable convention was capable of performing distributed joins etc.,
>> >> then the similarities would be even bigger.
>> >>
>> >> To the best of my knowledge, the majority of our users adopt Calcite
>> either
>> >> to query multiple data sources in a unified way or to provide SQL query
>> >> processing capabilities to data sources that do not already have.
>> Presto
>> >> seems to have similar goals, thus, I am not sure who will benefit from
>> this
>> >> new adapter.
>> >>
>> >> I never used Presto myself so I may have missed some important points.
>> Can
>> >> you share some use-cases about who may use this adapter and how?
>> >>
>> >> Best,
>> >> Stamatis
>> >>
>> >>
>> >> On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com>
>> wrote:
>> >>
>> >>> Hi everybody,
>> >>>
>> >>> I'd like to kick off a discussion on Implement Presto adapter in
>> calcite.
>> >>>
>> >>> We know that presto is also a powerful SQL query tool, which is widely
>> >> used
>> >>> with the olap of big data. So I want to add an adapter for presto to
>> >>> calite.
>> >>>
>> >>> Would love to hear your thoughts.
>> >>>
>> >>> Best,
>> >>> Forward
>> >>>
>> >>
>>
>>
Re: [discuss] Implement Presto adapter
Posted by Forward Xu <fo...@gmail.com>.
Hi Julian,
The current idea is through com.facebook.presto.spi.
1. Can be achieved through plugs provided by presto. For example [1].
2. You can construct Connector directly through
com.facebook.presto.spi.ConnectorFactory [2].
Best,
Forward
[1]
https://www.codota.com/code/java/methods/com.facebook.presto.spi.Plugin/getConnectorFactories
[2]
https://www.programcreek.com/java-api-examples/?code=y-lan/presto/presto-master/presto-raptor/src/main/java/com/facebook/presto/raptor/RaptorPlugin.java
Julian Hyde <jh...@apache.org> 于2020年1月10日周五 上午2:03写道：
> Sounds like you are planning to target Presto’s internal engine rather
> than Presto the database. To clarify, can you give an example of the code
> that you would generate and send to Presto for a simple SQL query?
>
> Julian
>
>
> > On Jan 9, 2020, at 5:07 AM, Forward Xu <fo...@gmail.com> wrote:
> >
> > Hi, Stamatis
> >
> > Because we have a number of internal clusters already using and
> installing
> > presto. And we want use calcite to optimize the parser and push it down
> to
> > presto, spark and other calculation engines or olaps. Utilize the sql
> > optimization function of calite in combination with distributed computing
> > engines or distributed olaps such as presto. In this way, each
> performance
> > can be exerted.
> >
> > Best,
> > Forward
> >
> > Stamatis Zampetakis <za...@gmail.com> 于2020年1月9日周四 下午8:51写道：
> >
> >> Hi,
> >>
> >> Presto seems to have many similarities with Calcite itself.
> >> As you mentioned, Presto is not really a database but a distributed SQL
> >> framework to query heterogeneous data sources.
> >> Presto connectors are in some sense similar to Calcite adapters and if
> the
> >> Enumerable convention was capable of performing distributed joins etc.,
> >> then the similarities would be even bigger.
> >>
> >> To the best of my knowledge, the majority of our users adopt Calcite
> either
> >> to query multiple data sources in a unified way or to provide SQL query
> >> processing capabilities to data sources that do not already have. Presto
> >> seems to have similar goals, thus, I am not sure who will benefit from
> this
> >> new adapter.
> >>
> >> I never used Presto myself so I may have missed some important points.
> Can
> >> you share some use-cases about who may use this adapter and how?
> >>
> >> Best,
> >> Stamatis
> >>
> >>
> >> On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com>
> wrote:
> >>
> >>> Hi everybody,
> >>>
> >>> I'd like to kick off a discussion on Implement Presto adapter in
> calcite.
> >>>
> >>> We know that presto is also a powerful SQL query tool, which is widely
> >> used
> >>> with the olap of big data. So I want to add an adapter for presto to
> >>> calite.
> >>>
> >>> Would love to hear your thoughts.
> >>>
> >>> Best,
> >>> Forward
> >>>
> >>
>
>
Re: [discuss] Implement Presto adapter
Posted by Julian Hyde <jh...@apache.org>.
Sounds like you are planning to target Presto’s internal engine rather than Presto the database. To clarify, can you give an example of the code that you would generate and send to Presto for a simple SQL query?
Julian
> On Jan 9, 2020, at 5:07 AM, Forward Xu <fo...@gmail.com> wrote:
>
> Hi, Stamatis
>
> Because we have a number of internal clusters already using and installing
> presto. And we want use calcite to optimize the parser and push it down to
> presto, spark and other calculation engines or olaps. Utilize the sql
> optimization function of calite in combination with distributed computing
> engines or distributed olaps such as presto. In this way, each performance
> can be exerted.
>
> Best,
> Forward
>
> Stamatis Zampetakis <za...@gmail.com> 于2020年1月9日周四 下午8:51写道：
>
>> Hi,
>>
>> Presto seems to have many similarities with Calcite itself.
>> As you mentioned, Presto is not really a database but a distributed SQL
>> framework to query heterogeneous data sources.
>> Presto connectors are in some sense similar to Calcite adapters and if the
>> Enumerable convention was capable of performing distributed joins etc.,
>> then the similarities would be even bigger.
>>
>> To the best of my knowledge, the majority of our users adopt Calcite either
>> to query multiple data sources in a unified way or to provide SQL query
>> processing capabilities to data sources that do not already have. Presto
>> seems to have similar goals, thus, I am not sure who will benefit from this
>> new adapter.
>>
>> I never used Presto myself so I may have missed some important points. Can
>> you share some use-cases about who may use this adapter and how?
>>
>> Best,
>> Stamatis
>>
>>
>> On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com> wrote:
>>
>>> Hi everybody,
>>>
>>> I'd like to kick off a discussion on Implement Presto adapter in calcite.
>>>
>>> We know that presto is also a powerful SQL query tool, which is widely
>> used
>>> with the olap of big data. So I want to add an adapter for presto to
>>> calite.
>>>
>>> Would love to hear your thoughts.
>>>
>>> Best,
>>> Forward
>>>
>>
Re: [discuss] Implement Presto adapter
Posted by Forward Xu <fo...@gmail.com>.
Hi, Stamatis
Because we have a number of internal clusters already using and installing
presto. And we want use calcite to optimize the parser and push it down to
presto, spark and other calculation engines or olaps. Utilize the sql
optimization function of calite in combination with distributed computing
engines or distributed olaps such as presto. In this way, each performance
can be exerted.
Best,
Forward
Stamatis Zampetakis <za...@gmail.com> 于2020年1月9日周四 下午8:51写道：
> Hi,
>
> Presto seems to have many similarities with Calcite itself.
> As you mentioned, Presto is not really a database but a distributed SQL
> framework to query heterogeneous data sources.
> Presto connectors are in some sense similar to Calcite adapters and if the
> Enumerable convention was capable of performing distributed joins etc.,
> then the similarities would be even bigger.
>
> To the best of my knowledge, the majority of our users adopt Calcite either
> to query multiple data sources in a unified way or to provide SQL query
> processing capabilities to data sources that do not already have. Presto
> seems to have similar goals, thus, I am not sure who will benefit from this
> new adapter.
>
> I never used Presto myself so I may have missed some important points. Can
> you share some use-cases about who may use this adapter and how?
>
> Best,
> Stamatis
>
>
> On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com> wrote:
>
> > Hi everybody,
> >
> > I'd like to kick off a discussion on Implement Presto adapter in calcite.
> >
> > We know that presto is also a powerful SQL query tool, which is widely
> used
> > with the olap of big data. So I want to add an adapter for presto to
> > calite.
> >
> > Would love to hear your thoughts.
> >
> > Best,
> > Forward
> >
>
Re: [discuss] Implement Presto adapter
Posted by Stamatis Zampetakis <za...@gmail.com>.
Hi,
Presto seems to have many similarities with Calcite itself.
As you mentioned, Presto is not really a database but a distributed SQL
framework to query heterogeneous data sources.
Presto connectors are in some sense similar to Calcite adapters and if the
Enumerable convention was capable of performing distributed joins etc.,
then the similarities would be even bigger.
To the best of my knowledge, the majority of our users adopt Calcite either
to query multiple data sources in a unified way or to provide SQL query
processing capabilities to data sources that do not already have. Presto
seems to have similar goals, thus, I am not sure who will benefit from this
new adapter.
I never used Presto myself so I may have missed some important points. Can
you share some use-cases about who may use this adapter and how?
Best,
Stamatis
On Wed, Jan 8, 2020 at 2:44 PM Forward Xu <fo...@gmail.com> wrote:
> Hi everybody,
>
> I'd like to kick off a discussion on Implement Presto adapter in calcite.
>
> We know that presto is also a powerful SQL query tool, which is widely used
> with the olap of big data. So I want to add an adapter for presto to
> calite.
>
> Would love to hear your thoughts.
>
> Best,
> Forward
>