Calcite JDBC-Spark Integration

[Prabhat Singh <pr...@gmail.com>]

Hi,

I have a requirement to query the multiple data sources in a single SQL
query. I saw Calcite provides this with a JDBC adapter. While running this,
I encountered the following challenges:

   1.

   Execution was happening in memory which causes OOM on big data.
   2.

   I saw there is an option for Spark. If this option is enabled, will the
   query be executed using Spark?
   3.

   If the answer to the 2nd question is yes, when I am trying to execute
   the query using the Spark option, I am getting the following exception:
   [image: spark exception.PNG]When I debug this it seems like the slim
   data context is null because of that it throws null pointer exception.

Please let me know the response.thanks in advance.


Best Regards
Prabhat Singh
+919028246725

Re: Calcite JDBC-Spark Integration

[Julian Hyde <jh...@apache.org>]

Your image did not come through. Please paste the stack trace as text.

Julian

On Tue, Oct 27, 2020 at 12:02 PM Prabhat Singh
<pr...@gmail.com> wrote:
>
> Hi,
>
> I have a requirement to query the multiple data sources in a single SQL query. I saw Calcite provides this with a JDBC adapter. While running this, I encountered the following challenges:
>
> Execution was happening in memory which causes OOM on big data.
>
> I saw there is an option for Spark. If this option is enabled, will the query be executed using Spark?
>
> If the answer to the 2nd question is yes, when I am trying to execute the query using the Spark option, I am getting the following exception:
>
> When I debug this it seems like the slim data context is null because of that it throws null pointer exception.
>
> Please let me know the response.thanks in advance.
>
>
> Best Regards
> Prabhat Singh
> +919028246725