How to upgrade Spark version in CDH 5.4

[Ashish Dutt <as...@gmail.com>]

Hi,
I need to upgrade spark version 1.3 to version 1.4 on CDH 5.4.
I checked the documentation here
<http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v5-1-x/Cloudera-Manager-Managing-Clusters/cm5mc_upgrade_cdh5_using_parcels.html#xd_583c10bfdbd326ba--6eed2fb8-14349d04bee--7dc6__section_zd5_1yz_l4>
but
I do not see any thing relevant

Any suggestions directing to a solution are welcome.

Thanks,
Ashish

Re: How to upgrade Spark version in CDH 5.4

[David Sabater Dinter <da...@gmail.com>]

As Sean suggested you can actually build Spark 1.4 for CDH 5.4.x and also
include Hive libraries for 0.13.1, but *this will be completely unsupported
by Cloudera*.
I would suggest to do that only if you just want to experiment with new
features from Spark 1.4. I.e. Run SparkSQL with sort-merge join to run
expensive BI types of queries over a Hadoop YARN cluster.
You just need to keep the build in your user folder across the cluster,
then include the Path for the binaries (spark-shell, spark-subimt,
pyspark,etc...) in your user profile and make sure you include CDH
Classpath to load all dependencies.

On Sun, Jul 12, 2015 at 8:19 PM, Sean Owen <so...@cloudera.com> wrote:

> Yeah, it won't technically be supported, and you shouldn't go
> modifying the actual installation, but if you just make your own build
> of 1.4 for CDH 5.4 and use that build to launch YARN-based apps, I
> imagine it will Just Work for most any use case.
>
> On Sun, Jul 12, 2015 at 7:34 PM, Ruslan Dautkhanov <da...@gmail.com>
> wrote:
> > Good question. I'd like to know the same. Although I think you'll loose
> > supportability.
> >
> >
> >
> > --
> > Ruslan Dautkhanov
> >
> > On Wed, Jul 8, 2015 at 2:03 AM, Ashish Dutt <as...@gmail.com>
> wrote:
> >>
> >>
> >> Hi,
> >> I need to upgrade spark version 1.3 to version 1.4 on CDH 5.4.
> >> I checked the documentation here but I do not see any thing relevant
> >>
> >> Any suggestions directing to a solution are welcome.
> >>
> >> Thanks,
> >> Ashish
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: How to upgrade Spark version in CDH 5.4

[Sean Owen <so...@cloudera.com>]

Yeah, it won't technically be supported, and you shouldn't go
modifying the actual installation, but if you just make your own build
of 1.4 for CDH 5.4 and use that build to launch YARN-based apps, I
imagine it will Just Work for most any use case.

On Sun, Jul 12, 2015 at 7:34 PM, Ruslan Dautkhanov <da...@gmail.com> wrote:
> Good question. I'd like to know the same. Although I think you'll loose
> supportability.
>
>
>
> --
> Ruslan Dautkhanov
>
> On Wed, Jul 8, 2015 at 2:03 AM, Ashish Dutt <as...@gmail.com> wrote:
>>
>>
>> Hi,
>> I need to upgrade spark version 1.3 to version 1.4 on CDH 5.4.
>> I checked the documentation here but I do not see any thing relevant
>>
>> Any suggestions directing to a solution are welcome.
>>
>> Thanks,
>> Ashish
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: How to upgrade Spark version in CDH 5.4

[Ruslan Dautkhanov <da...@gmail.com>]

Good question. I'd like to know the same. Although I think you'll loose
supportability.



-- 
Ruslan Dautkhanov

On Wed, Jul 8, 2015 at 2:03 AM, Ashish Dutt <as...@gmail.com> wrote:

>
> Hi,
> I need to upgrade spark version 1.3 to version 1.4 on CDH 5.4.
> I checked the documentation here
> <http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v5-1-x/Cloudera-Manager-Managing-Clusters/cm5mc_upgrade_cdh5_using_parcels.html#xd_583c10bfdbd326ba--6eed2fb8-14349d04bee--7dc6__section_zd5_1yz_l4> but
> I do not see any thing relevant
>
> Any suggestions directing to a solution are welcome.
>
> Thanks,
> Ashish
>