cannot build Spark 1.6 with Pyspark

[enzo <en...@smartinsightsfromdata.com>]

I’m trying to build Spark 1.6 with Pyspark but somewhere in Zeppelin the old dependency (py4j-0.8.2.1-src.zip) is listed, instead of the new py4j-0.9-src.zip

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (download-and-zip-pyspark-files) on project zeppelin-spark-dependencies: An Ant BuildException has occured: Warning: Could not find file /opt/zeppelin/spark-dependencies/target/spark-dist/spark-1.6.0/python/lib/py4j-0.8.2.1-src.zip to copy.
[ERROR] around Ant part ...<copy file="/opt/zeppelin/spark-dependencies/target/spark-dist/spark-1.6.0/python/lib/py4j-0.8.2.1-src.zip" todir="../interpreter/spark/pyspark"/>... @ 5:149 in /opt/zeppelin/spark-dependencies/target/antrun/build-main.xml

Eliminating -Pyspark Zeppelin builds without problems.



Enzo
enzo@smartinsightsfromdata.com

Re: cannot build Spark 1.6 with Pyspark

[moon soo Lee <mo...@apache.org>]

Hi Enzo,

Could you make sure you're in latest master branch and specified
-Pspark-1.6 flag, too?

Thanks,
moon

On Fri, Jan 8, 2016 at 12:00 PM enzo <en...@smartinsightsfromdata.com> wrote:

> I’m trying to build Spark 1.6 with Pyspark but somewhere in Zeppelin the
> old dependency (py4j-0.8.2.1-src.zip) is listed, instead of the new
> py4j-0.9-src.zip
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run
> (download-and-zip-pyspark-files) on project zeppelin-spark-dependencies: An
> Ant BuildException has occured: Warning: Could not find file
> /opt/zeppelin/spark-dependencies/target/spark-dist/spark-1.6.0/python/lib/py4j-0.8.2.1-src.zip
> to copy.
> [ERROR] around Ant part ...<copy
> file="/opt/zeppelin/spark-dependencies/target/spark-dist/spark-1.6.0/python/lib/py4j-0.8.2.1-src.zip"
> todir="../interpreter/spark/pyspark"/>... @ 5:149 in
> /opt/zeppelin/spark-dependencies/target/antrun/build-main.xml
>
> Eliminating -Pyspark Zeppelin builds without problems.
>
>
>
> Enzo
> enzo@smartinsightsfromdata.com
>
>
>
>