You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by infa elance <in...@gmail.com> on 2017/04/06 05:06:16 UTC

Spark and Hive connection

Hi all,
When using spark-shell my understanding is spark connects to hive through
metastore.
The question i have is does spark connect to metastore , is it JDBC?

Thanks and Regards,
Ajay.

Re: Spark and Hive connection

Posted by Gourav Sengupta <go...@gmail.com>.

Hi,

the connection is made to HIVE thrift server.

Regards,
Gourav

On Thu, Apr 6, 2017 at 6:06 AM, infa elance <in...@gmail.com> wrote:

> Hi all,
> When using spark-shell my understanding is spark connects to hive through
> metastore.
> The question i have is does spark connect to metastore , is it JDBC?
>
> Thanks and Regards,
> Ajay.
>

Re: Spark and Hive connection

Posted by Nicholas Hakobian <ni...@rallyhealth.com>.

Spark connects directly to the Hive metastore service in order to manage
table definitions and locations and such. If you are using the CLI
interfaces and turn on INFO level logging, you can see when you instantiate
a HiveContext that it is connecting to the Hive Metastore and the URL its
using for the connection.

Hope this helps,
Nick

Nicholas Szandor Hakobian, Ph.D.
Senior Data Scientist
Rally Health
nicholas.hakobian@rallyhealth.com

On Wed, Apr 5, 2017 at 10:06 PM, infa elance <in...@gmail.com> wrote:

> Hi all,
> When using spark-shell my understanding is spark connects to hive through
> metastore.
> The question i have is does spark connect to metastore , is it JDBC?
>
> Thanks and Regards,
> Ajay.
>