You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@zeppelin.apache.org by IT CTO <go...@gmail.com> on 2015/07/16 18:35:51 UTC

Zeppelin on CDH

Hi,
As much as I understand CDH does not support spark 1.4 (correct me if I am
wrong) and only support spark 1.3 (CDH 5.3.1)
Will I have problems using zeppelin with pyspark on spark 1.3? or is it
just setting the right spark version in the build options?

Thanks
Eran

Re: Zeppelin on CDH

Posted by IT CTO <go...@gmail.com>.

Thanks all for the help. Will go with spark 1.3 and pyspark d option

בתאריך יום ו׳, 17 ביולי 2015, 12:51 מאת <fe...@hotmail.com>:

>  CDH does not yet have support for Spark 1.4.*x*
>
>
>
> On Thu, Jul 16, 2015 at 9:49 AM -0700, "Vadla, Karthik" <
> karthik.vadla@intel.com> wrote:
>
>   Nope. Right now I have built my binaries with CDH 5.4 and spark 1.3.
> I’m not sure of CDH support for spark 1.4.
> Instead of configuring pyspark manually. To enable pyspark configuration
> during build, just provide profile name “Ppyspark” while building your
> zeppelin binaries.
>
>
>
> Example: If you want to build binaries for CDH 5.4 , Hadoop 2.6 and Spark
> 1.3  with pyspark profile
>
> *mvn clean package -Pspark-1.3 -Ppyspark -Dhadoop.version=2.6.0-cdh5.4.2
> -Phadoop-2.6 –DskipTests*
>
>
>
> In above command, make sure you provide correct version numbers.
>
>
>
> Hope this helps.
>
>
>
> Thanks
>
> Karthik
>
> *From:* IT CTO [mailto:goi.cto@gmail.com]
> *Sent:* Thursday, July 16, 2015 9:36 AM
> *To:* users
> *Subject:* Zeppelin on CDH
>
>
>
> Hi,
>
> As much as I understand CDH does not support spark 1.4 (correct me if I am
> wrong) and only support spark 1.3 (CDH 5.3.1)
>
> Will I have problems using zeppelin with pyspark on spark 1.3? or is it
> just setting the right spark version in the build options?
>
>
>
> Thanks
>
> Eran
>

RE: Zeppelin on CDH

Posted by fe...@hotmail.com.

CDH does not yet have support for Spark 1.4.x





On Thu, Jul 16, 2015 at 9:49 AM -0700, "Vadla, Karthik" <ka...@intel.com> wrote:
Nope. Right now I have built my binaries with CDH 5.4 and spark 1.3. I’m not sure of CDH support for spark 1.4.
Instead of configuring pyspark manually. To enable pyspark configuration during build, just provide profile name “Ppyspark” while building your zeppelin binaries.


Example: If you want to build binaries for CDH 5.4 , Hadoop 2.6 and Spark 1.3  with pyspark profile
mvn clean package -Pspark-1.3 -Ppyspark -Dhadoop.version=2.6.0-cdh5.4.2 -Phadoop-2.6 –DskipTests

In above command, make sure you provide correct version numbers.

Hope this helps.

Thanks
Karthik
From: IT CTO [mailto:goi.cto@gmail.com]
Sent: Thursday, July 16, 2015 9:36 AM
To: users
Subject: Zeppelin on CDH

Hi,
As much as I understand CDH does not support spark 1.4 (correct me if I am wrong) and only support spark 1.3 (CDH 5.3.1)
Will I have problems using zeppelin with pyspark on spark 1.3? or is it just setting the right spark version in the build options?

Thanks
Eran

RE: Zeppelin on CDH

Posted by "Vadla, Karthik" <ka...@intel.com>.

Nope. Right now I have built my binaries with CDH 5.4 and spark 1.3. I’m not sure of CDH support for spark 1.4.
Instead of configuring pyspark manually. To enable pyspark configuration during build, just provide profile name “Ppyspark” while building your zeppelin binaries.


Example: If you want to build binaries for CDH 5.4 , Hadoop 2.6 and Spark 1.3  with pyspark profile
mvn clean package -Pspark-1.3 -Ppyspark -Dhadoop.version=2.6.0-cdh5.4.2 -Phadoop-2.6 –DskipTests

In above command, make sure you provide correct version numbers.

Hope this helps.

Thanks
Karthik
From: IT CTO [mailto:goi.cto@gmail.com]
Sent: Thursday, July 16, 2015 9:36 AM
To: users
Subject: Zeppelin on CDH

Hi,
As much as I understand CDH does not support spark 1.4 (correct me if I am wrong) and only support spark 1.3 (CDH 5.3.1)
Will I have problems using zeppelin with pyspark on spark 1.3? or is it just setting the right spark version in the build options?

Thanks
Eran