You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "angela.whelan" <an...@synchronoss.com> on 2016/01/13 12:37:08 UTC
Is it possible to use SparkSQL JDBC ThriftServer without Hive
hi,
I'm wondering if it is possible to use the SparkSQL JDBC ThriftServer
without Hive?
The reason I'm asking is that we are unsure about the speed of Hive with
SparkSQL JDBC connectivity.
I can't find any article online about using SparkSQL JDBC ThriftServer
without Hive.
Many thanks in advance for any help on this.
Thanks, Angela
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-use-SparkSQL-JDBC-ThriftServer-without-Hive-tp25959.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Posted by Antonio Piccolboni <an...@piccolboni.info>.
Thriftserver creates a HiveContext hence the hive libs, but you don't need
to have Hive running at all. Depending on what you mean by "without Hive"
that could be a positive or negative answer to your question. You need the
dependencies, but you don't need a running instance.
On Wed, Jan 13, 2016 at 3:37 AM angela.whelan <an...@synchronoss.com>
wrote:
> hi,
> I'm wondering if it is possible to use the SparkSQL JDBC ThriftServer
> without Hive?
>
> The reason I'm asking is that we are unsure about the speed of Hive with
> SparkSQL JDBC connectivity.
>
> I can't find any article online about using SparkSQL JDBC ThriftServer
> without Hive.
>
> Many thanks in advance for any help on this.
>
> Thanks, Angela
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-use-SparkSQL-JDBC-ThriftServer-without-Hive-tp25959.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Posted by Mohammed Guller <mo...@glassbeam.com>.
Sambit - I believe the default Derby-based metastore allows only one active user at a time. You can replace it with MySQL or Postgres.
Using the Hive Metastore enables Spark SQL to be compatible with Hive. If you have an existing Hive setup, you can Spark SQL to process data in your Hive tables.
Mohammed
-----Original Message-----
From: Sambit Tripathy (RBEI/EDS1) [mailto:Sambit.Tripathy@in.bosch.com]
Sent: Friday, January 15, 2016 11:30 AM
To: Mohammed Guller; angela.whelan; user@spark.apache.org
Subject: RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Hi Mohammed,
I think this is something you can do at the Thrift server startup. So this would run an instance of Derby and act as a Metastore. Any idea if this Debry Metastore will have distributed access and why do we use the Hive Metastore then?
@Angela: I would also be happy to have a metastore owned by Spark Thrift Server. What are you trying to achieve by using the Thrift server without Hive?
Regards,
Sambit.
-----Original Message-----
From: Mohammed Guller [mailto:mohammed@glassbeam.com]
Sent: Wednesday, January 13, 2016 2:54 PM
To: angela.whelan <an...@synchronoss.com>; user@spark.apache.org
Subject: RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Hi Angela,
Yes, you can use Spark SQL JDBC/ThriftServer without Hive.
Mohammed
-----Original Message-----
From: angela.whelan [mailto:angela.whelan@synchronoss.com]
Sent: Wednesday, January 13, 2016 3:37 AM
To: user@spark.apache.org
Subject: Is it possible to use SparkSQL JDBC ThriftServer without Hive
hi,
I'm wondering if it is possible to use the SparkSQL JDBC ThriftServer without Hive?
The reason I'm asking is that we are unsure about the speed of Hive with SparkSQL JDBC connectivity.
I can't find any article online about using SparkSQL JDBC ThriftServer without Hive.
Many thanks in advance for any help on this.
Thanks, Angela
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-use-SparkSQL-JDBC-ThriftServer-without-Hive-tp25959.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Posted by "Sambit Tripathy (RBEI/EDS1)" <Sa...@in.bosch.com>.
Hi Mohammed,
I think this is something you can do at the Thrift server startup. So this would run an instance of Derby and act as a Metastore. Any idea if this Debry Metastore will have distributed access and why do we use the Hive Metastore then?
@Angela: I would also be happy to have a metastore owned by Spark Thrift Server. What are you trying to achieve by using the Thrift server without Hive?
Regards,
Sambit.
-----Original Message-----
From: Mohammed Guller [mailto:mohammed@glassbeam.com]
Sent: Wednesday, January 13, 2016 2:54 PM
To: angela.whelan <an...@synchronoss.com>; user@spark.apache.org
Subject: RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Hi Angela,
Yes, you can use Spark SQL JDBC/ThriftServer without Hive.
Mohammed
-----Original Message-----
From: angela.whelan [mailto:angela.whelan@synchronoss.com]
Sent: Wednesday, January 13, 2016 3:37 AM
To: user@spark.apache.org
Subject: Is it possible to use SparkSQL JDBC ThriftServer without Hive
hi,
I'm wondering if it is possible to use the SparkSQL JDBC ThriftServer without Hive?
The reason I'm asking is that we are unsure about the speed of Hive with SparkSQL JDBC connectivity.
I can't find any article online about using SparkSQL JDBC ThriftServer without Hive.
Many thanks in advance for any help on this.
Thanks, Angela
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-use-SparkSQL-JDBC-ThriftServer-without-Hive-tp25959.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: Is it possible to use SparkSQL JDBC ThriftServer without Hive
Posted by Mohammed Guller <mo...@glassbeam.com>.
Hi Angela,
Yes, you can use Spark SQL JDBC/ThriftServer without Hive.
Mohammed
-----Original Message-----
From: angela.whelan [mailto:angela.whelan@synchronoss.com]
Sent: Wednesday, January 13, 2016 3:37 AM
To: user@spark.apache.org
Subject: Is it possible to use SparkSQL JDBC ThriftServer without Hive
hi,
I'm wondering if it is possible to use the SparkSQL JDBC ThriftServer without Hive?
The reason I'm asking is that we are unsure about the speed of Hive with SparkSQL JDBC connectivity.
I can't find any article online about using SparkSQL JDBC ThriftServer without Hive.
Many thanks in advance for any help on this.
Thanks, Angela
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-possible-to-use-SparkSQL-JDBC-ThriftServer-without-Hive-tp25959.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org