You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Patrik Medvedev <pa...@gmail.com> on 2017/06/07 09:15:07 UTC
[Spark JDBC] Does spark support read from remote Hive server via JDBC
Hello guys,
I need to execute hive queries on remote hive server from spark, but for
some reasons i receive only column names(without data).
Data available in table, i checked it via HUE and java jdbc connection.
Here is my code example:
val test = spark.read
.option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
.option("user", "user")
.option("password", "password")
.option("dbtable", "some_table_with_data")
.option("driver", "org.apache.hive.jdbc.HiveDriver")
.format("jdbc")
.load()
test.show()
Scala version: 2.11
Spark version: 2.1.0, i also tried 2.1.1
Hive version: CDH 5.7 Hive 1.1.1
Hive JDBC version: 1.1.1
But this problem available on Hive with later versions, too.
Could you help me with this issue, because i didn't find anything in mail
group answers and StackOverflow.
Or could you help me find correct solution how to query remote hive from
spark?
--
*Cheers,*
*Patrick*
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Patrik Medvedev <pa...@gmail.com>.
Hello, we tested this, but still nothing.
пн, 12 июн. 2017 г. в 23:01, Thakrar, Jayesh <jt...@conversantmedia.com>:
> Could this be due to https://issues.apache.org/jira/browse/HIVE-11116 ?
>
>
>
> *From: *Patrik Medvedev <pa...@gmail.com>
> *Date: *Monday, June 12, 2017 at 2:31 AM
> *To: *Jörn Franke <jo...@gmail.com>, vaquar khan <
> vaquar.khan@gmail.com>
> *Cc: *Jean Georges Perrin <jg...@jgp.net>, User <us...@spark.apache.org>
> *Subject: *Re: [Spark JDBC] Does spark support read from remote Hive
> server via JDBC
>
>
>
> Hello,
>
>
>
> All security checkings disabled, but i still don't have any info in result.
>
>
>
>
>
> вс, 11 июн. 2017 г. в 14:24, Jörn Franke <jo...@gmail.com>:
>
> Is sentry preventing the access?
>
>
> On 11. Jun 2017, at 01:55, vaquar khan <va...@gmail.com> wrote:
>
> Hi ,
>
> Pleaae check your firewall security setting sharing link one good link.
>
>
>
>
> http://belablotski.blogspot.in/2016/01/access-hive-tables-from-spark-using.html?m=1
>
>
>
>
>
>
>
> Regards,
>
> Vaquar khan
>
>
>
> On Jun 8, 2017 1:53 AM, "Patrik Medvedev" <pa...@gmail.com>
> wrote:
>
> Hello guys,
>
>
>
> Can somebody help me with my problem?
>
> Let me know, if you need more details.
>
>
>
>
>
> ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>:
>
> No, I don't.
>
>
>
> ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
>
> Do you have some other security in place like Kerberos or impersonation?
> It may affect your access.
>
>
>
> jg
>
>
>
>
> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>
> wrote:
>
> Hello guys,
>
>
>
> I need to execute hive queries on remote hive server from spark, but for
> some reasons i receive only column names(without data).
>
> Data available in table, i checked it via HUE and java jdbc connection.
>
>
>
> Here is my code example:
>
> val test = spark.read
>
> .option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
>
> .option("user", "user")
>
> .option("password", "password")
>
> .option("dbtable", "some_table_with_data")
>
> .option("driver", "org.apache.hive.jdbc.HiveDriver")
>
> .format("jdbc")
>
> .load()
>
> test.show()
>
>
>
>
>
> Scala version: 2.11
>
> Spark version: 2.1.0, i also tried 2.1.1
>
> Hive version: CDH 5.7 Hive 1.1.1
>
> Hive JDBC version: 1.1.1
>
>
>
> But this problem available on Hive with later versions, too.
>
> Could you help me with this issue, because i didn't find anything in mail
> group answers and StackOverflow.
>
> Or could you help me find correct solution how to query remote hive from
> spark?
>
>
>
> --
>
> *Cheers,*
>
> *Patrick*
>
>
Re: [Spark JDBC] Does spark support read from remote Hive server via
JDBC
Posted by "Thakrar, Jayesh" <jt...@conversantmedia.com>.
Could this be due to https://issues.apache.org/jira/browse/HIVE-11116 ?
From: Patrik Medvedev <pa...@gmail.com>
Date: Monday, June 12, 2017 at 2:31 AM
To: Jörn Franke <jo...@gmail.com>, vaquar khan <va...@gmail.com>
Cc: Jean Georges Perrin <jg...@jgp.net>, User <us...@spark.apache.org>
Subject: Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Hello,
All security checkings disabled, but i still don't have any info in result.
вс, 11 июн. 2017 г. в 14:24, Jörn Franke <jo...@gmail.com>>:
Is sentry preventing the access?
On 11. Jun 2017, at 01:55, vaquar khan <va...@gmail.com>> wrote:
Hi ,
Pleaae check your firewall security setting sharing link one good link.
http://belablotski.blogspot.in/2016/01/access-hive-tables-from-spark-using.html?m=1
Regards,
Vaquar khan
On Jun 8, 2017 1:53 AM, "Patrik Medvedev" <pa...@gmail.com>> wrote:
Hello guys,
Can somebody help me with my problem?
Let me know, if you need more details.
ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>>:
No, I don't.
ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>>:
Do you have some other security in place like Kerberos or impersonation? It may affect your access.
jg
On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>> wrote:
Hello guys,
I need to execute hive queries on remote hive server from spark, but for some reasons i receive only column names(without data).
Data available in table, i checked it via HUE and java jdbc connection.
Here is my code example:
val test = spark.read
.option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
.option("user", "user")
.option("password", "password")
.option("dbtable", "some_table_with_data")
.option("driver", "org.apache.hive.jdbc.HiveDriver")
.format("jdbc")
.load()
test.show()
Scala version: 2.11
Spark version: 2.1.0, i also tried 2.1.1
Hive version: CDH 5.7 Hive 1.1.1
Hive JDBC version: 1.1.1
But this problem available on Hive with later versions, too.
Could you help me with this issue, because i didn't find anything in mail group answers and StackOverflow.
Or could you help me find correct solution how to query remote hive from spark?
--
Cheers,
Patrick
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Patrik Medvedev <pa...@gmail.com>.
Hello,
All security checkings disabled, but i still don't have any info in result.
вс, 11 июн. 2017 г. в 14:24, Jörn Franke <jo...@gmail.com>:
> Is sentry preventing the access?
>
> On 11. Jun 2017, at 01:55, vaquar khan <va...@gmail.com> wrote:
>
> Hi ,
> Pleaae check your firewall security setting sharing link one good link.
>
>
> http://belablotski.blogspot.in/2016/01/access-hive-tables-from-spark-using.html?m=1
>
>
>
> Regards,
> Vaquar khan
>
> On Jun 8, 2017 1:53 AM, "Patrik Medvedev" <pa...@gmail.com>
> wrote:
>
>> Hello guys,
>>
>> Can somebody help me with my problem?
>> Let me know, if you need more details.
>>
>>
>> ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>:
>>
>>> No, I don't.
>>>
>>> ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
>>>
>>>> Do you have some other security in place like Kerberos or
>>>> impersonation? It may affect your access.
>>>>
>>>>
>>>> jg
>>>>
>>>>
>>>> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>
>>>> wrote:
>>>>
>>>> Hello guys,
>>>>
>>>> I need to execute hive queries on remote hive server from spark, but
>>>> for some reasons i receive only column names(without data).
>>>> Data available in table, i checked it via HUE and java jdbc connection.
>>>>
>>>> Here is my code example:
>>>> val test = spark.read
>>>> .option("url",
>>>> "jdbc:hive2://remote.hive.server:10000/work_base")
>>>> .option("user", "user")
>>>> .option("password", "password")
>>>> .option("dbtable", "some_table_with_data")
>>>> .option("driver", "org.apache.hive.jdbc.HiveDriver")
>>>> .format("jdbc")
>>>> .load()
>>>> test.show()
>>>>
>>>>
>>>> Scala version: 2.11
>>>> Spark version: 2.1.0, i also tried 2.1.1
>>>> Hive version: CDH 5.7 Hive 1.1.1
>>>> Hive JDBC version: 1.1.1
>>>>
>>>> But this problem available on Hive with later versions, too.
>>>> Could you help me with this issue, because i didn't find anything in
>>>> mail group answers and StackOverflow.
>>>> Or could you help me find correct solution how to query remote hive
>>>> from spark?
>>>>
>>>> --
>>>> *Cheers,*
>>>> *Patrick*
>>>>
>>>>
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Jörn Franke <jo...@gmail.com>.
Is sentry preventing the access?
> On 11. Jun 2017, at 01:55, vaquar khan <va...@gmail.com> wrote:
>
> Hi ,
> Pleaae check your firewall security setting sharing link one good link.
>
> http://belablotski.blogspot.in/2016/01/access-hive-tables-from-spark-using.html?m=1
>
>
>
> Regards,
> Vaquar khan
>
>> On Jun 8, 2017 1:53 AM, "Patrik Medvedev" <pa...@gmail.com> wrote:
>> Hello guys,
>>
>> Can somebody help me with my problem?
>> Let me know, if you need more details.
>>
>>
>> ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>:
>>> No, I don't.
>>>
>>> ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
>>>> Do you have some other security in place like Kerberos or impersonation? It may affect your access.
>>>>
>>>>
>>>> jg
>>>>
>>>>
>>>>> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com> wrote:
>>>>>
>>>>> Hello guys,
>>>>>
>>>>> I need to execute hive queries on remote hive server from spark, but for some reasons i receive only column names(without data).
>>>>> Data available in table, i checked it via HUE and java jdbc connection.
>>>>>
>>>>> Here is my code example:
>>>>> val test = spark.read
>>>>> .option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
>>>>> .option("user", "user")
>>>>> .option("password", "password")
>>>>> .option("dbtable", "some_table_with_data")
>>>>> .option("driver", "org.apache.hive.jdbc.HiveDriver")
>>>>> .format("jdbc")
>>>>> .load()
>>>>> test.show()
>>>>>
>>>>>
>>>>> Scala version: 2.11
>>>>> Spark version: 2.1.0, i also tried 2.1.1
>>>>> Hive version: CDH 5.7 Hive 1.1.1
>>>>> Hive JDBC version: 1.1.1
>>>>>
>>>>> But this problem available on Hive with later versions, too.
>>>>> Could you help me with this issue, because i didn't find anything in mail group answers and StackOverflow.
>>>>> Or could you help me find correct solution how to query remote hive from spark?
>>>>>
>>>>> --
>>>>> Cheers,
>>>>> Patrick
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by vaquar khan <va...@gmail.com>.
Hi ,
Pleaae check your firewall security setting sharing link one good link.
http://belablotski.blogspot.in/2016/01/access-hive-tables-from-spark-using.html?m=1
Regards,
Vaquar khan
On Jun 8, 2017 1:53 AM, "Patrik Medvedev" <pa...@gmail.com> wrote:
> Hello guys,
>
> Can somebody help me with my problem?
> Let me know, if you need more details.
>
>
> ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>:
>
>> No, I don't.
>>
>> ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
>>
>>> Do you have some other security in place like Kerberos or impersonation?
>>> It may affect your access.
>>>
>>>
>>> jg
>>>
>>>
>>> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>
>>> wrote:
>>>
>>> Hello guys,
>>>
>>> I need to execute hive queries on remote hive server from spark, but for
>>> some reasons i receive only column names(without data).
>>> Data available in table, i checked it via HUE and java jdbc connection.
>>>
>>> Here is my code example:
>>> val test = spark.read
>>> .option("url", "jdbc:hive2://remote.hive.
>>> server:10000/work_base")
>>> .option("user", "user")
>>> .option("password", "password")
>>> .option("dbtable", "some_table_with_data")
>>> .option("driver", "org.apache.hive.jdbc.HiveDriver")
>>> .format("jdbc")
>>> .load()
>>> test.show()
>>>
>>>
>>> Scala version: 2.11
>>> Spark version: 2.1.0, i also tried 2.1.1
>>> Hive version: CDH 5.7 Hive 1.1.1
>>> Hive JDBC version: 1.1.1
>>>
>>> But this problem available on Hive with later versions, too.
>>> Could you help me with this issue, because i didn't find anything in
>>> mail group answers and StackOverflow.
>>> Or could you help me find correct solution how to query remote hive from
>>> spark?
>>>
>>> --
>>> *Cheers,*
>>> *Patrick*
>>>
>>>
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Patrik Medvedev <pa...@gmail.com>.
Hello guys,
Can somebody help me with my problem?
Let me know, if you need more details.
ср, 7 июн. 2017 г. в 16:43, Patrik Medvedev <pa...@gmail.com>:
> No, I don't.
>
> ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
>
>> Do you have some other security in place like Kerberos or impersonation?
>> It may affect your access.
>>
>>
>> jg
>>
>>
>> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>
>> wrote:
>>
>> Hello guys,
>>
>> I need to execute hive queries on remote hive server from spark, but for
>> some reasons i receive only column names(without data).
>> Data available in table, i checked it via HUE and java jdbc connection.
>>
>> Here is my code example:
>> val test = spark.read
>> .option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
>> .option("user", "user")
>> .option("password", "password")
>> .option("dbtable", "some_table_with_data")
>> .option("driver", "org.apache.hive.jdbc.HiveDriver")
>> .format("jdbc")
>> .load()
>> test.show()
>>
>>
>> Scala version: 2.11
>> Spark version: 2.1.0, i also tried 2.1.1
>> Hive version: CDH 5.7 Hive 1.1.1
>> Hive JDBC version: 1.1.1
>>
>> But this problem available on Hive with later versions, too.
>> Could you help me with this issue, because i didn't find anything in mail
>> group answers and StackOverflow.
>> Or could you help me find correct solution how to query remote hive from
>> spark?
>>
>> --
>> *Cheers,*
>> *Patrick*
>>
>>
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Patrik Medvedev <pa...@gmail.com>.
No, I don't.
ср, 7 июн. 2017 г. в 16:42, Jean Georges Perrin <jg...@jgp.net>:
> Do you have some other security in place like Kerberos or impersonation?
> It may affect your access.
>
>
> jg
>
>
> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com>
> wrote:
>
> Hello guys,
>
> I need to execute hive queries on remote hive server from spark, but for
> some reasons i receive only column names(without data).
> Data available in table, i checked it via HUE and java jdbc connection.
>
> Here is my code example:
> val test = spark.read
> .option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
> .option("user", "user")
> .option("password", "password")
> .option("dbtable", "some_table_with_data")
> .option("driver", "org.apache.hive.jdbc.HiveDriver")
> .format("jdbc")
> .load()
> test.show()
>
>
> Scala version: 2.11
> Spark version: 2.1.0, i also tried 2.1.1
> Hive version: CDH 5.7 Hive 1.1.1
> Hive JDBC version: 1.1.1
>
> But this problem available on Hive with later versions, too.
> Could you help me with this issue, because i didn't find anything in mail
> group answers and StackOverflow.
> Or could you help me find correct solution how to query remote hive from
> spark?
>
> --
> *Cheers,*
> *Patrick*
>
>
Re: [Spark JDBC] Does spark support read from remote Hive server via JDBC
Posted by Jean Georges Perrin <jg...@jgp.net>.
Do you have some other security in place like Kerberos or impersonation? It may affect your access.
jg
> On Jun 7, 2017, at 02:15, Patrik Medvedev <pa...@gmail.com> wrote:
>
> Hello guys,
>
> I need to execute hive queries on remote hive server from spark, but for some reasons i receive only column names(without data).
> Data available in table, i checked it via HUE and java jdbc connection.
>
> Here is my code example:
> val test = spark.read
> .option("url", "jdbc:hive2://remote.hive.server:10000/work_base")
> .option("user", "user")
> .option("password", "password")
> .option("dbtable", "some_table_with_data")
> .option("driver", "org.apache.hive.jdbc.HiveDriver")
> .format("jdbc")
> .load()
> test.show()
>
>
> Scala version: 2.11
> Spark version: 2.1.0, i also tried 2.1.1
> Hive version: CDH 5.7 Hive 1.1.1
> Hive JDBC version: 1.1.1
>
> But this problem available on Hive with later versions, too.
> Could you help me with this issue, because i didn't find anything in mail group answers and StackOverflow.
> Or could you help me find correct solution how to query remote hive from spark?
>
> --
> Cheers,
> Patrick