You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Doug Balog <do...@dugos.com> on 2015/03/27 09:48:56 UTC
Support for Hive 0.14 in secure mode on hadoop 2.6.0
Hi,
I'm just wondering if anybody is working on supporting Hive 0.14 in secure mode on hadoop 2.6.0 ?
I see once Jira referring to it https://issues.apache.org/jira/browse/SPARK-5111
but it mentions no effort to move to 0.14.
Thanks,
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Support for Hive 0.14 in secure mode on hadoop 2.6.0
Posted by Zhan Zhang <zz...@hortonworks.com>.
Hi Doug,
Spark-5111 is to make spark work with security hadoop cluster in 2.6. There is some compatibility issue which need the fix Spark-5111 patch.
In insecure cluster, current spark can connect to hive-0.14 without problems.
By the way, I am really glad to hear that "an adaption layer in Spark SQL in 1.4” and "allows Spark SQL to connect to arbitrary Hive version”
Thanks.
Zhan Zhang
On Mar 27, 2015, at 12:57 PM, Doug Balog <do...@dugos.com> wrote:
> Is there a JIRA for this adaption layer ? It sounds like a better long term solution.
>
> If anybody knows what is require to get the current Shim layer working with Hive 0.14, please post what you know.
> I’m willing to spend some time on it, but I’m still learning how things fit together and it might take me a while.
> I’ve been looking at the pr associated with SPARK-5111 for hints.
>
> Thanks,
>
> Doug
>
>
>> On Mar 27, 2015, at 6:13 AM, Cheng Lian <li...@gmail.com> wrote:
>>
>> We're planning to replace the current Hive version profiles and shim layer with an adaption layer in Spark SQL in 1.4. This adaption layer allows Spark SQL to connect to arbitrary Hive version greater than or equal to 0.12.0 (or maybe 0.13.1, not decided yet).
>>
>> However, it's not a promise yet, since this requires major refactoring of the current Spark SQL Hive support.
>>
>> Cheng
>>
>> On 3/27/15 4:48 PM, Doug Balog wrote:
>>> Hi,
>>> I'm just wondering if anybody is working on supporting Hive 0.14 in secure mode on hadoop 2.6.0 ?
>>> I see once Jira referring to it https://issues.apache.org/jira/browse/SPARK-5111
>>> but it mentions no effort to move to 0.14.
>>>
>>> Thanks,
>>>
>>> Doug
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Support for Hive 0.14 in secure mode on hadoop 2.6.0
Posted by Doug Balog <do...@dugos.com>.
Is there a JIRA for this adaption layer ? It sounds like a better long term solution.
If anybody knows what is require to get the current Shim layer working with Hive 0.14, please post what you know.
I’m willing to spend some time on it, but I’m still learning how things fit together and it might take me a while.
I’ve been looking at the pr associated with SPARK-5111 for hints.
Thanks,
Doug
> On Mar 27, 2015, at 6:13 AM, Cheng Lian <li...@gmail.com> wrote:
>
> We're planning to replace the current Hive version profiles and shim layer with an adaption layer in Spark SQL in 1.4. This adaption layer allows Spark SQL to connect to arbitrary Hive version greater than or equal to 0.12.0 (or maybe 0.13.1, not decided yet).
>
> However, it's not a promise yet, since this requires major refactoring of the current Spark SQL Hive support.
>
> Cheng
>
> On 3/27/15 4:48 PM, Doug Balog wrote:
>> Hi,
>> I'm just wondering if anybody is working on supporting Hive 0.14 in secure mode on hadoop 2.6.0 ?
>> I see once Jira referring to it https://issues.apache.org/jira/browse/SPARK-5111
>> but it mentions no effort to move to 0.14.
>>
>> Thanks,
>>
>> Doug
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Support for Hive 0.14 in secure mode on hadoop 2.6.0
Posted by Cheng Lian <li...@gmail.com>.
We're planning to replace the current Hive version profiles and shim
layer with an adaption layer in Spark SQL in 1.4. This adaption layer
allows Spark SQL to connect to arbitrary Hive version greater than or
equal to 0.12.0 (or maybe 0.13.1, not decided yet).
However, it's not a promise yet, since this requires major refactoring
of the current Spark SQL Hive support.
Cheng
On 3/27/15 4:48 PM, Doug Balog wrote:
> Hi,
> I'm just wondering if anybody is working on supporting Hive 0.14 in secure mode on hadoop 2.6.0 ?
> I see once Jira referring to it https://issues.apache.org/jira/browse/SPARK-5111
> but it mentions no effort to move to 0.14.
>
> Thanks,
>
> Doug
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org