You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Brandon White <bw...@gmail.com> on 2015/07/15 02:25:58 UTC
How do you access a cached Spark SQL Table from a JBDC connection?
Hello there,
I have a JBDC connection setup to my Spark cluster but I cannot see the
tables that I cache in memory. The only tables I can see are those that are
in my Hive instance. I use a HiveContext to register a table and cache it
in memory. How can I enable my JBDC connection to query this in memory
table?
Brandon
RE: How do you access a cached Spark SQL Table from a JBDC
connection?
Posted by "Cheng, Hao" <ha...@intel.com>.
So you’re with different HiveContext instances for the caching. We are not expected to see the cached tables cached with the other HiveContext instance.
From: Brandon White [mailto:bwwinthehouse@gmail.com]
Sent: Wednesday, July 15, 2015 8:48 AM
To: Cheng, Hao
Cc: user
Subject: Re: How do you access a cached Spark SQL Table from a JBDC connection?
I cache the tell with
hiveContext.cacheTable("tableName")
On Tue, Jul 14, 2015 at 5:43 PM, Cheng, Hao <ha...@intel.com>> wrote:
Can you describe how did you cache the tables? In another HiveContext? AFAIK, cached table only be visible within the same HiveContext, you probably need to execute the sql query like
“cache table mytable as SELECT xxx” in the JDBC connection also.
Cheng Hao
From: Brandon White [mailto:bwwinthehouse@gmail.com<ma...@gmail.com>]
Sent: Wednesday, July 15, 2015 8:26 AM
To: user
Subject: How do you access a cached Spark SQL Table from a JBDC connection?
Hello there,
I have a JBDC connection setup to my Spark cluster but I cannot see the tables that I cache in memory. The only tables I can see are those that are in my Hive instance. I use a HiveContext to register a table and cache it in memory. How can I enable my JBDC connection to query this in memory table?
Brandon
Re: How do you access a cached Spark SQL Table from a JBDC connection?
Posted by Brandon White <bw...@gmail.com>.
I cache the tell with
hiveContext.cacheTable("tableName")
On Tue, Jul 14, 2015 at 5:43 PM, Cheng, Hao <ha...@intel.com> wrote:
> Can you describe how did you cache the tables? In another HiveContext?
> AFAIK, cached table only be visible within the same HiveContext, you
> probably need to execute the sql query like
>
> “cache table mytable as SELECT xxx” in the JDBC connection also.
>
>
>
> Cheng Hao
>
>
>
> *From:* Brandon White [mailto:bwwinthehouse@gmail.com]
> *Sent:* Wednesday, July 15, 2015 8:26 AM
> *To:* user
> *Subject:* How do you access a cached Spark SQL Table from a JBDC
> connection?
>
>
>
> Hello there,
>
>
>
> I have a JBDC connection setup to my Spark cluster but I cannot see the
> tables that I cache in memory. The only tables I can see are those that are
> in my Hive instance. I use a HiveContext to register a table and cache it
> in memory. How can I enable my JBDC connection to query this in memory
> table?
>
>
>
> Brandon
>
RE: How do you access a cached Spark SQL Table from a JBDC
connection?
Posted by "Cheng, Hao" <ha...@intel.com>.
Can you describe how did you cache the tables? In another HiveContext? AFAIK, cached table only be visible within the same HiveContext, you probably need to execute the sql query like
“cache table mytable as SELECT xxx” in the JDBC connection also.
Cheng Hao
From: Brandon White [mailto:bwwinthehouse@gmail.com]
Sent: Wednesday, July 15, 2015 8:26 AM
To: user
Subject: How do you access a cached Spark SQL Table from a JBDC connection?
Hello there,
I have a JBDC connection setup to my Spark cluster but I cannot see the tables that I cache in memory. The only tables I can see are those that are in my Hive instance. I use a HiveContext to register a table and cache it in memory. How can I enable my JBDC connection to query this in memory table?
Brandon