You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jing Dong <ji...@qubitdigital.com> on 2014/12/12 13:26:35 UTC

Spark CDH5 packages

Hi,

    I'm new to this list, so please excuse if I'm asking simple questions.

   We are experimenting spark deployment on existing CDH clusters.
However the spark package come with CDH are very out of date (v1.0.0).

    Has anyone had experience with custom Spark upgrade for CDH5? Any
installation or packaging recommendation would be appreciate.

    The download page and documentation site only mention about CDH4
pre build package.


Thanks
Jing

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark CDH5 packages

Posted by Jing Dong <ji...@qubitdigital.com>.
Hi Sowen,

Thanks for the tip. When will CDH 5.3 be released? Some sort of
timeline would be helpful.

Thanks,
Jing


> On 12 Dec 2014, at 12:34, Sean Owen <so...@cloudera.com> wrote:
>
> No, CDH 5.2 includes Spark 1.1 actually, which is the latest released
> minor version; 5.3 will include 1.2, which not released yet.
>
> You can make a build of just about any version of Spark for CDH5, and
> manually install it yourself, sure, but easier would be to just update
> CDH. The instructions are the ones you've found; all you need to know
> is to use the hadoop-2.4 profile (which is for 2.5 and 2.6 too) and
> set hadoop.version appropriately.
>
>> On Fri, Dec 12, 2014 at 12:26 PM, Jing Dong <ji...@qubitdigital.com> wrote:
>> Hi,
>>
>>    I'm new to this list, so please excuse if I'm asking simple questions.
>>
>>   We are experimenting spark deployment on existing CDH clusters.
>> However the spark package come with CDH are very out of date (v1.0.0).
>>
>>    Has anyone had experience with custom Spark upgrade for CDH5? Any
>> installation or packaging recommendation would be appreciate.
>>
>>    The download page and documentation site only mention about CDH4
>> pre build package.
>>
>>
>> Thanks
>> Jing
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark CDH5 packages

Posted by Sean Owen <so...@cloudera.com>.
No, CDH 5.2 includes Spark 1.1 actually, which is the latest released
minor version; 5.3 will include 1.2, which not released yet.

You can make a build of just about any version of Spark for CDH5, and
manually install it yourself, sure, but easier would be to just update
CDH. The instructions are the ones you've found; all you need to know
is to use the hadoop-2.4 profile (which is for 2.5 and 2.6 too) and
set hadoop.version appropriately.

On Fri, Dec 12, 2014 at 12:26 PM, Jing Dong <ji...@qubitdigital.com> wrote:
> Hi,
>
>     I'm new to this list, so please excuse if I'm asking simple questions.
>
>    We are experimenting spark deployment on existing CDH clusters.
> However the spark package come with CDH are very out of date (v1.0.0).
>
>     Has anyone had experience with custom Spark upgrade for CDH5? Any
> installation or packaging recommendation would be appreciate.
>
>     The download page and documentation site only mention about CDH4
> pre build package.
>
>
> Thanks
> Jing
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org