Federated Query

[Yogendra Sharma <sy...@live.com>]

Hi Team,

I am exploring Apache Calcite to run federated queries on three different databases. I could not find a working example anywhere on internet.

 Can you guys confirm if it is possible and if yes, is there an example that you can point me to?

Thanks,
Yogendra

Re: Federated Query

[Walaa Eldin Moustafa <wa...@gmail.com>]

Coral [1] is based on Calcite and can be used to run HiveQL on Spark and
Presto/Trino. We are in the process of adding Presto/Trino SQL as an input
language so it can run on Hive/Spark as well. You can read more on how it
interacts with Calcite in this article [2].

[1] https://github.com/linkedin/coral
[2] https://engineering.linkedin.com/blog/2020/coral

Thanks,
Walaa.


On Mon, Nov 1, 2021 at 12:38 PM Yogendra Sharma <sy...@live.com> wrote:

> Hi Team,
>
> I am exploring Apache Calcite to run federated queries on three different
> databases. I could not find a working example anywhere on internet.
>
>  Can you guys confirm if it is possible and if yes, is there an example
> that you can point me to?
>
> Thanks,
> Yogendra
>
>

Re: Federated Query

[Yogendra Sharma <sy...@live.com>]

I have not but i will explore it right away.

Get Outlook for Android<https://aka.ms/ghei36>
________________________________
From: Charles Givre <cg...@gmail.com>
Sent: Tuesday, November 9, 2021 6:12:01 PM
To: dev@calcite.apache.org <de...@calcite.apache.org>
Subject: Re: Federated Query

Yogendra,
Out of curiosity, have you tried Apache Drill?  Drill uses Calcite for query planning, but does exactly what you are describing.
Best,
-- C

> On Nov 9, 2021, at 5:31 AM, JiaTao Tao <ta...@gmail.com> wrote:
>
> Yes, you can.
> I've done this, twice.
> Once you get the query plan, you can translate the calcite's plan to
> other engines.
> The main work is to define a SQL standard and align type system.
>
> Regards!
>
> Aron Tao
>
>
> Yogendra Sharma <sy...@live.com> 于2021年11月2日周二 上午3:38写道：
>
>> Hi Team,
>>
>> I am exploring Apache Calcite to run federated queries on three different
>> databases. I could not find a working example anywhere on internet.
>>
>> Can you guys confirm if it is possible and if yes, is there an example
>> that you can point me to?
>>
>> Thanks,
>> Yogendra
>>
>>

Re: Federated Query

[Charles Givre <cg...@gmail.com>]

Yogendra, 
Out of curiosity, have you tried Apache Drill?  Drill uses Calcite for query planning, but does exactly what you are describing.  
Best,
-- C

> On Nov 9, 2021, at 5:31 AM, JiaTao Tao <ta...@gmail.com> wrote:
> 
> Yes, you can.
> I've done this, twice.
> Once you get the query plan, you can translate the calcite's plan to
> other engines.
> The main work is to define a SQL standard and align type system.
> 
> Regards!
> 
> Aron Tao
> 
> 
> Yogendra Sharma <sy...@live.com> 于2021年11月2日周二 上午3:38写道：
> 
>> Hi Team,
>> 
>> I am exploring Apache Calcite to run federated queries on three different
>> databases. I could not find a working example anywhere on internet.
>> 
>> Can you guys confirm if it is possible and if yes, is there an example
>> that you can point me to?
>> 
>> Thanks,
>> Yogendra
>> 
>> 

Re: Federated Query

[JiaTao Tao <ta...@gmail.com>]

Yes, you can.
I've done this, twice.
Once you get the query plan, you can translate the calcite's plan to
other engines.
The main work is to define a SQL standard and align type system.

Regards!

Aron Tao


Yogendra Sharma <sy...@live.com> 于2021年11月2日周二 上午3:38写道：

> Hi Team,
>
> I am exploring Apache Calcite to run federated queries on three different
> databases. I could not find a working example anywhere on internet.
>
>  Can you guys confirm if it is possible and if yes, is there an example
> that you can point me to?
>
> Thanks,
> Yogendra
>
>

Re: Federated Query

[Michael Mior <mm...@apache.org>]

To run federated queries, you simply need to define each data source
in your model.json. You can see examples of model.json files for
various adapters in the test code for those adapters.

For example:

File - https://github.com/apache/calcite/blob/master/file/src/test/resources/model.json
Cassandra - https://github.com/apache/calcite/blob/master/cassandra/src/test/resources/model.json

You can add as many schemas as you want from different adapters into
the "schemas" array in model.json and all those schemas will be
available for you to write queries including joins across schemas.

--
Michael Mior
mmior@apache.org

Le lun. 1 nov. 2021 à 15:38, Yogendra Sharma <sy...@live.com> a écrit :
>
> Hi Team,
>
> I am exploring Apache Calcite to run federated queries on three different databases. I could not find a working example anywhere on internet.
>
>  Can you guys confirm if it is possible and if yes, is there an example that you can point me to?
>
> Thanks,
> Yogendra
>