You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Marco Mistroni <mm...@gmail.com> on 2016/10/01 20:24:19 UTC
Re: Pls assist: Spark 2.0 build failure on Ubuntu 16.06
Hi guys
sorry to annoy you on this but i am getting nowhere. So far i have tried
to build spark 2.0 on my local laptop with no success.... so i blamed my
laptop poor performance
So today i fired off an EC2 Ubuntu 16.06 Instance and installed the
following (i copy paste commands here)
ubuntu@ip-172-31-40-104:~/spark$ history
sudo apt-get install -y python-software-properties
sudo apt-get install -y software-properties-common
sudo add-apt-repository -y ppa:webupd8team/java
sudo apt-get install -y oracle-java8-installer
sudo apt-get install -y git
git clone git://github.com/apache/spark.git
cd spark
Then launched the following commands:
First thi sone, with only yarn, as i dont need hadoop
./build/mvn -Pyarn -DskipTests clean package
This failed. after kicking off same command with -X i got this
ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
on project spark-core_2.11: Execution scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
-> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
(scala-compile-first) on project spark-core_2.11: Execution
scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at
sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at
sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at
scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at
scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at
scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
... 21 more
Then i tried to use hadoop even though i dont need hadoop for my code
./build/mvn -Pyarn -Phadoop-2.4 -DskipTests clean package
This failed again, at exactly the same point, with the same error
Then i thought maybe i used an old version of hadoop, so i tried to use 2.7
./build/mvn -Pyarn -Phadoop-2.7 -DskipTests clean package
Nada. same error.
I have tried sbt, no luck
Could anyone suggest me a mvn proflie to use to build spark 2.0? i am
starting to suspect (before arguing that there's something wrong with spark
git) that i am using somehow wrong parameters..... or perhaps i should
install scala 2.11 before i install spark? or Maven ?
kr
marco
On Fri, Sep 30, 2016 at 8:23 PM, Marco Mistroni <mm...@gmail.com> wrote:
> Hi all
> this problem is still bothering me.
> Here's my setup
> - Ubuntu 16.06
> - Java 8
> - Spark 2.0
> - have launched following command: ./build/mvn -X -Pyarn -Phadoop-2.7
> -DskipTests clean package
> and i am gettign this exception:
>
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
> (scala-test-compile-first) on project spark-mllib_2.10: Execution
> scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
> failed.
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:212)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:153)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:145)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.
> buildProject(LifecycleModuleBuilder.java:116)
> at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.
> buildProject(LifecycleModuleBuilder.java:80)
> at org.apache.maven.lifecycle.internal.builder.singlethreaded.
> SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.
> execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.
> launchEnhanced(Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.
> launch(Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.
> mainWithExitCode(Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.
> main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
> failed.
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:145)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:207)
> ... 20 more
> Caused by: Compile failed via zinc server
> at sbt_inc.SbtIncrementalCompiler.zincCompile(
> SbtIncrementalCompiler.java:136)
> at sbt_inc.SbtIncrementalCompiler.compile(
> SbtIncrementalCompiler.java:86)
> at scala_maven.ScalaCompilerSupport.incrementalCompile(
> ScalaCompilerSupport.java:303)
> at scala_maven.ScalaCompilerSupport.compile(
> ScalaCompilerSupport.java:119)
> at scala_maven.ScalaCompilerSupport.doExecute(
> ScalaCompilerSupport.java:99)
> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
> at scala_maven.ScalaTestCompileMojo.execute(
> ScalaTestCompileMojo.java:48)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:134)
>
> anyone found a similar error?
>
> kr
>
>
>
>
> On Sat, Sep 3, 2016 at 2:54 PM, Marco Mistroni <mm...@gmail.com>
> wrote:
>
>> hi all
>>
>> i am getting failures when building spark 2.0 on Ubuntu 16.06
>> Here's details of what i have installed on the ubuntu host
>> - java 8
>> - scala 2.11
>> - git
>>
>> When i launch the command
>>
>> ./build/mvn -Pyarn -Phadoop-2.7 -DskipTests clean package
>>
>> everything compiles sort of fine and at the end i get this exception
>>
>> INFO] BUILD FAILURE
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Total time: 01:05 h
>> [INFO] Finished at: 2016-09-03T13:25:27+00:00
>> [INFO] Final Memory: 57M/208M
>> [INFO] ------------------------------------------------------------
>> ------------
>> [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
>> (scala-test-compile-first) on project spark-streaming_2.11: Execution
>> scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
>> failed. CompileFailed -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1] http://cwiki.apache.org/conflu
>> ence/display/MAVEN/PluginExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :spark-streaming_2.11
>>
>> could anyone help?
>>
>> kr
>>
>
>
Re: Pls assist: Spark 2.0 build failure on Ubuntu 16.06
Posted by Marco Mistroni <mm...@gmail.com>.
Hi Sean
thanks. I managed to build Spark 2 (it was actually 2.1, not 2.0...i m
sourcing it from here (git clone git://github.com/apache/spark.git))
Now, i managed to build it but i had to
- use Java 1.7 along with MAVEN_OPTS (using java1.8 send the whole process
into insufficient memory for JVM while compiling spark-core....)
- use Scala 2.10, installed on my host (when i tried with Scala 2.11, i
got same results as with java 1.8)
.
kr
marco
On Sat, Oct 1, 2016 at 9:29 PM, Sean Owen <so...@cloudera.com> wrote:
> "Compile failed via zinc server"
>
> Try shutting down zinc. Something's funny about your compile server.
> It's not required anyway.
>
> On Sat, Oct 1, 2016 at 3:24 PM, Marco Mistroni <mm...@gmail.com>
> wrote:
> > Hi guys
> > sorry to annoy you on this but i am getting nowhere. So far i have
> tried to
> > build spark 2.0 on my local laptop with no success.... so i blamed my
> > laptop poor performance
> > So today i fired off an EC2 Ubuntu 16.06 Instance and installed the
> > following (i copy paste commands here)
> >
> > ubuntu@ip-172-31-40-104:~/spark$ history
> > sudo apt-get install -y python-software-properties
> > sudo apt-get install -y software-properties-common
> > sudo add-apt-repository -y ppa:webupd8team/java
> > sudo apt-get install -y oracle-java8-installer
> > sudo apt-get install -y git
> > git clone git://github.com/apache/spark.git
> > cd spark
> >
> > Then launched the following commands:
> > First thi sone, with only yarn, as i dont need hadoop
> >
> > ./build/mvn -Pyarn -DskipTests clean package
> >
> > This failed. after kicking off same command with -X i got this
> >
> > ERROR] Failed to execute goal
> > net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> (scala-compile-first) on
> > project spark-core_2.11: Execution scala-compile-first of goal
> > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> CompileFailed ->
> > [Help 1]
> > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute
> > goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> > (scala-compile-first) on project spark-core_2.11: Execution
> > scala-compile-first of goal
> > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:212)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:153)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:145)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(
> LifecycleModuleBuilder.java:116)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(
> LifecycleModuleBuilder.java:80)
> > at
> > org.apache.maven.lifecycle.internal.builder.singlethreaded.
> SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> > at
> > org.apache.maven.lifecycle.internal.LifecycleStarter.
> execute(LifecycleStarter.java:128)
> > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
> 307)
> > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
> 193)
> > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> > at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.
> launchEnhanced(Launcher.java:289)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.
> launch(Launcher.java:229)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.
> mainWithExitCode(Launcher.java:415)
> > at
> > org.codehaus.plexus.classworlds.launcher.Launcher.
> main(Launcher.java:356)
> > Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> > scala-compile-first of goal
> > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> > at
> > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:145)
> > at
> > org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:207)
> > ... 20 more
> > Caused by: Compile failed via zinc server
> > at
> > sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:
> 136)
> > at
> > sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
> > at
> > scala_maven.ScalaCompilerSupport.incrementalCompile(
> ScalaCompilerSupport.java:303)
> > at
> > scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
> > at
> > scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
> > at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:
> 482)
> > at
> > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:134)
> > ... 21 more
> >
> >
> >
> > Then i tried to use hadoop even though i dont need hadoop for my code
> >
> > ./build/mvn -Pyarn -Phadoop-2.4 -DskipTests clean package
> >
> > This failed again, at exactly the same point, with the same error
> >
> > Then i thought maybe i used an old version of hadoop, so i tried to use
> 2.7
> >
> > ./build/mvn -Pyarn -Phadoop-2.7 -DskipTests clean package
> >
> > Nada. same error.
> >
> >
> > I have tried sbt, no luck
> >
> > Could anyone suggest me a mvn proflie to use to build spark 2.0? i am
> > starting to suspect (before arguing that there's something wrong with
> spark
> > git) that i am using somehow wrong parameters..... or perhaps i should
> > install scala 2.11 before i install spark? or Maven ?
> >
> > kr
> > marco
> >
> >
> > On Fri, Sep 30, 2016 at 8:23 PM, Marco Mistroni <mm...@gmail.com>
> wrote:
> >>
> >> Hi all
> >> this problem is still bothering me.
> >> Here's my setup
> >> - Ubuntu 16.06
> >> - Java 8
> >> - Spark 2.0
> >> - have launched following command: ./build/mvn -X -Pyarn -Phadoop-2.7
> >> -DskipTests clean package
> >> and i am gettign this exception:
> >>
> >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute
> >> goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
> >> (scala-test-compile-first) on project spark-mllib_2.10: Execution
> >> scala-test-compile-first of goal
> >> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
> >>
> >> at
> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:212)
> >> at
> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:153)
> >> at
> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:145)
> >> at
> >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.
> buildProject(LifecycleModuleBuilder.java:116)
> >> at
> >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.
> buildProject(LifecycleModuleBuilder.java:80)
> >> at
> >> org.apache.maven.lifecycle.internal.builder.singlethreaded.
> SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> >> at
> >> org.apache.maven.lifecycle.internal.LifecycleStarter.
> execute(LifecycleStarter.java:128)
> >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
> 307)
> >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
> 193)
> >> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> >> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> >> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> >> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> >> at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:498)
> >> at
> >> org.codehaus.plexus.classworlds.launcher.Launcher.
> launchEnhanced(Launcher.java:289)
> >> at
> >> org.codehaus.plexus.classworlds.launcher.Launcher.
> launch(Launcher.java:229)
> >> at
> >> org.codehaus.plexus.classworlds.launcher.Launcher.
> mainWithExitCode(Launcher.java:415)
> >> at
> >> org.codehaus.plexus.classworlds.launcher.Launcher.
> main(Launcher.java:356)
> >> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> >> scala-test-compile-first of goal
> >> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
> >> at
> >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:145)
> >> at
> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:207)
> >> ... 20 more
> >> Caused by: Compile failed via zinc server
> >> at
> >> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:
> 136)
> >> at
> >> sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
> >> at
> >> scala_maven.ScalaCompilerSupport.incrementalCompile(
> ScalaCompilerSupport.java:303)
> >> at
> >> scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
> >> at
> >> scala_maven.ScalaCompilerSupport.doExecute(
> ScalaCompilerSupport.java:99)
> >> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:
> 482)
> >> at
> >> scala_maven.ScalaTestCompileMojo.execute(ScalaTestCompileMojo.java:48)
> >> at
> >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:134)
> >>
> >> anyone found a similar error?
> >>
> >> kr
>
Re: Pls assist: Spark 2.0 build failure on Ubuntu 16.06
Posted by Sean Owen <so...@cloudera.com>.
"Compile failed via zinc server"
Try shutting down zinc. Something's funny about your compile server.
It's not required anyway.
On Sat, Oct 1, 2016 at 3:24 PM, Marco Mistroni <mm...@gmail.com> wrote:
> Hi guys
> sorry to annoy you on this but i am getting nowhere. So far i have tried to
> build spark 2.0 on my local laptop with no success.... so i blamed my
> laptop poor performance
> So today i fired off an EC2 Ubuntu 16.06 Instance and installed the
> following (i copy paste commands here)
>
> ubuntu@ip-172-31-40-104:~/spark$ history
> sudo apt-get install -y python-software-properties
> sudo apt-get install -y software-properties-common
> sudo add-apt-repository -y ppa:webupd8team/java
> sudo apt-get install -y oracle-java8-installer
> sudo apt-get install -y git
> git clone git://github.com/apache/spark.git
> cd spark
>
> Then launched the following commands:
> First thi sone, with only yarn, as i dont need hadoop
>
> ./build/mvn -Pyarn -DskipTests clean package
>
> This failed. after kicking off same command with -X i got this
>
> ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on
> project spark-core_2.11: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed ->
> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> (scala-compile-first) on project spark-core_2.11: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> ... 20 more
> Caused by: Compile failed via zinc server
> at
> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
> at
> sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
> at
> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
> at
> scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
> at
> scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> ... 21 more
>
>
>
> Then i tried to use hadoop even though i dont need hadoop for my code
>
> ./build/mvn -Pyarn -Phadoop-2.4 -DskipTests clean package
>
> This failed again, at exactly the same point, with the same error
>
> Then i thought maybe i used an old version of hadoop, so i tried to use 2.7
>
> ./build/mvn -Pyarn -Phadoop-2.7 -DskipTests clean package
>
> Nada. same error.
>
>
> I have tried sbt, no luck
>
> Could anyone suggest me a mvn proflie to use to build spark 2.0? i am
> starting to suspect (before arguing that there's something wrong with spark
> git) that i am using somehow wrong parameters..... or perhaps i should
> install scala 2.11 before i install spark? or Maven ?
>
> kr
> marco
>
>
> On Fri, Sep 30, 2016 at 8:23 PM, Marco Mistroni <mm...@gmail.com> wrote:
>>
>> Hi all
>> this problem is still bothering me.
>> Here's my setup
>> - Ubuntu 16.06
>> - Java 8
>> - Spark 2.0
>> - have launched following command: ./build/mvn -X -Pyarn -Phadoop-2.7
>> -DskipTests clean package
>> and i am gettign this exception:
>>
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
>> (scala-test-compile-first) on project spark-mllib_2.10: Execution
>> scala-test-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
>>
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>> at
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
>> scala-test-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>> ... 20 more
>> Caused by: Compile failed via zinc server
>> at
>> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
>> at
>> sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
>> at
>> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
>> at
>> scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
>> at
>> scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
>> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
>> at
>> scala_maven.ScalaTestCompileMojo.execute(ScalaTestCompileMojo.java:48)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>
>> anyone found a similar error?
>>
>> kr
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org