You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jade Liu <ja...@nor1.com> on 2016/01/06 01:54:41 UTC
problem building spark on centos
Hi, All:
I'm trying to build spark 1.5.2 from source using maven with the following command:
./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver -DskipTests
I got the following error:
+ VERSION='[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException'
When I try:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
I got the following error:
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): java.security.ProviderException: java.security.KeyException and 'parent.relativePath' points at wrong local POM @ line 22, column 11
Does anyone know how to change the settings in maven to fix this?
Thanks in advance,
Jade
Re: problem building spark on centos
Posted by Todd Nist <ts...@gmail.com>.
That should read "I think your missing the --name option". Sorry about
that.
On Wed, Jan 6, 2016 at 3:03 PM, Todd Nist <ts...@gmail.com> wrote:
> Hi Jade,
>
> I think you "--name" option. The makedistribution should look like this:
>
> ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6
> -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests.
>
> As for why it failed to build with scala 2.11, did you run the
> ./dev/change-scala-version.sh 2.11 script to set the version of the
> artifacts to 2.11? If you do that then issue the build like this I think
> you will be ok:
>
> ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn
> -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11
> -DskipTests
>
> HTH.
>
> -Todd
>
> On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <ja...@nor1.com> wrote:
>
>> I’ve changed the scala version to 2.10.
>>
>> With this command:
>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean
>> package
>> Build was successful.
>>
>> But make a runnable version:
>> /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>> -Phive -Phive-thriftserver –DskipTests
>> Still fails with the following error:
>> [ERROR] Failed to execute goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
>> on project spark-launcher_2.10: Execution scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
>> -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
>> (scala-compile-first) on project spark-launcher_2.10: Execution
>> scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>> at
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
>> scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>> ... 20 more
>> Caused by: Compile failed via zinc server
>> at
>> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
>> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
>> at
>> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
>> at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
>> at
>> scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
>> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>
>> Not sure what’s causing it. Does anyone have any idea?
>>
>> Thanks!
>>
>> Jade
>> From: Ted Yu <yu...@gmail.com>
>> Date: Wednesday, January 6, 2016 at 10:40 AM
>> To: Jade Liu <ja...@nor1.com>, user <us...@spark.apache.org>
>>
>> Subject: Re: problem building spark on centos
>>
>> w.r.t. the second error, have you read this ?
>>
>> http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
>>
>> On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com> wrote:
>>
>>> I’m using 3.3.9. Thanks!
>>>
>>> Jade
>>>
>>> From: Ted Yu <yu...@gmail.com>
>>> Date: Tuesday, January 5, 2016 at 4:57 PM
>>> To: Jade Liu <ja...@nor1.com>
>>> Cc: "user@spark.apache.org" <us...@spark.apache.org>
>>> Subject: Re: problem building spark on centos
>>>
>>> Which version of maven are you using ?
>>>
>>> It should be 3.3.3+
>>>
>>> On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com> wrote:
>>>
>>>> Hi, All:
>>>>
>>>> I’m trying to build spark 1.5.2 from source using maven with the
>>>> following command:
>>>>
>>>> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>>>> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
>>>>
>>>>
>>>> I got the following error:I'
>>>>
>>>> + VERSION='[ERROR] [Help 2]
>>>> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
>>>> '
>>>>
>>>>
>>>> When I try:
>>>>
>>>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests
>>>> clean package
>>>>
>>>>
>>>> I got the following error:
>>>>
>>>> [FATAL] Non-resolvable parent POM for
>>>> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact
>>>> org.apache:apache:pom:14 from/to central (
>>>> https://repo1.maven.org/maven2): java.security.ProviderException:
>>>> java.security.KeyException and 'parent.relativePath' points at wrong local
>>>> POM @ line 22, column 11
>>>>
>>>>
>>>> Does anyone know how to change the settings in maven to fix this?
>>>>
>>>>
>>>> Thanks in advance,
>>>>
>>>>
>>>> Jade
>>>>
>>>
>>>
>>
>
Re: problem building spark on centos
Posted by Jade Liu <ja...@nor1.com>.
Yes I’m using maven 3.3.9.
From: Todd Nist <ts...@gmail.com>>
Date: Wednesday, January 6, 2016 at 12:33 PM
To: Jade Liu <ja...@nor1.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Not sure, I just built it with java 8, but 7 is supported so that should be fine. Are you using maven 3.3.3 + ?
RADTech:spark-1.5.2 tnist$ mvn -version
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T07:57:37-04:00)
Maven home: /usr/local/maven
Java version: 1.8.0_51, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_51.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.10.5", arch: "x86_64", family: "mac"
On Wed, Jan 6, 2016 at 3:27 PM, Jade Liu <ja...@nor1.com>> wrote:
Hi, Todd:
Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh 2.11 script when using scala version 2.11.
I just tried this as you suggested:
./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
Still got the same error:
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-launcher_2.10
Do you think it’s java problem? I’m using oracle JDK 1.7. Should I update it to 1.8 instead? I just checked the conf and it says 1.7.
Thanks,
Jade
From: Todd Nist <ts...@gmail.com>>
Date: Wednesday, January 6, 2016 at 12:03 PM
To: Jade Liu <ja...@nor1.com>>
Cc: Ted Yu <yu...@gmail.com>>, "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Hi Jade,
I think you "--name" option. The makedistribution should look like this:
./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests.
As for why it failed to build with scala 2.11, did you run the ./dev/change-scala-version.sh 2.11 script to set the version of the artifacts to 2.11? If you do that then issue the build like this I think you will be ok:
./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11 -DskipTests
HTH.
-Todd
On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <ja...@nor1.com>> wrote:
I’ve changed the scala version to 2.10.
With this command:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
Build was successful.
But make a runnable version:
/make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
Still fails with the following error:
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
Not sure what’s causing it. Does anyone have any idea?
Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Wednesday, January 6, 2016 at 10:40 AM
To: Jade Liu <ja...@nor1.com>>, user <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
w.r.t. the second error, have you read this ?
http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com>> wrote:
I’m using 3.3.9. Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Tuesday, January 5, 2016 at 4:57 PM
To: Jade Liu <ja...@nor1.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Which version of maven are you using ?
It should be 3.3.3+
On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com>> wrote:
Hi, All:
I’m trying to build spark 1.5.2 from source using maven with the following command:
./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
I got the following error:I'
+ VERSION='[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException'
When I try:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
I got the following error:
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): java.security.ProviderException: java.security.KeyException and 'parent.relativePath' points at wrong local POM @ line 22, column 11
Does anyone know how to change the settings in maven to fix this?
Thanks in advance,
Jade
Re: problem building spark on centos
Posted by Todd Nist <ts...@gmail.com>.
Not sure, I just built it with java 8, but 7 is supported so that should be
fine. Are you using maven 3.3.3 + ?
RADTech:spark-1.5.2 tnist$ mvn -version
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=512m; support was removed in 8.0
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06;
2015-04-22T07:57:37-04:00)
Maven home: /usr/local/maven
Java version: 1.8.0_51, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.8.0_51.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.10.5", arch: "x86_64", family: "mac"
On Wed, Jan 6, 2016 at 3:27 PM, Jade Liu <ja...@nor1.com> wrote:
> Hi, Todd:
>
> Thanks for your suggestion. Yes I did run the
> ./dev/change-scala-version.sh 2.11 script when using scala version 2.11.
>
> I just tried this as you suggested:
> ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6
> -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
>
> Still got the same error:
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
> on project spark-launcher_2.10: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
> -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> (scala-compile-first) on project spark-launcher_2.10: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> ... 20 more
> Caused by: Compile failed via zinc server
> at
> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
> at
> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
> at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
> at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> ... 21 more
> [ERROR]
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :spark-launcher_2.10
>
> Do you think it’s java problem? I’m using oracle JDK 1.7. Should I update
> it to 1.8 instead? I just checked the conf and it says 1.7.
>
> Thanks,
>
> Jade
>
> From: Todd Nist <ts...@gmail.com>
> Date: Wednesday, January 6, 2016 at 12:03 PM
> To: Jade Liu <ja...@nor1.com>
> Cc: Ted Yu <yu...@gmail.com>, "user@spark.apache.org" <
> user@spark.apache.org>
>
> Subject: Re: problem building spark on centos
>
> Hi Jade,
>
> I think you "--name" option. The makedistribution should look like this:
>
> ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6
> -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests.
>
> As for why it failed to build with scala 2.11, did you run the
> ./dev/change-scala-version.sh 2.11 script to set the version of the
> artifacts to 2.11? If you do that then issue the build like this I think
> you will be ok:
>
> ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn
> -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11
> -DskipTests
>
> HTH.
>
> -Todd
>
> On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <ja...@nor1.com> wrote:
>
>> I’ve changed the scala version to 2.10.
>>
>> With this command:
>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean
>> package
>> Build was successful.
>>
>> But make a runnable version:
>> /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>> -Phive -Phive-thriftserver –DskipTests
>> Still fails with the following error:
>> [ERROR] Failed to execute goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
>> on project spark-launcher_2.10: Execution scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
>> -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
>> (scala-compile-first) on project spark-launcher_2.10: Execution
>> scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>> at
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
>> scala-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>> ... 20 more
>> Caused by: Compile failed via zinc server
>> at
>> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
>> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
>> at
>> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
>> at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
>> at
>> scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
>> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>
>> Not sure what’s causing it. Does anyone have any idea?
>>
>> Thanks!
>>
>> Jade
>> From: Ted Yu <yu...@gmail.com>
>> Date: Wednesday, January 6, 2016 at 10:40 AM
>> To: Jade Liu <ja...@nor1.com>, user <us...@spark.apache.org>
>>
>> Subject: Re: problem building spark on centos
>>
>> w.r.t. the second error, have you read this ?
>>
>> http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
>>
>> On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com> wrote:
>>
>>> I’m using 3.3.9. Thanks!
>>>
>>> Jade
>>>
>>> From: Ted Yu <yu...@gmail.com>
>>> Date: Tuesday, January 5, 2016 at 4:57 PM
>>> To: Jade Liu <ja...@nor1.com>
>>> Cc: "user@spark.apache.org" <us...@spark.apache.org>
>>> Subject: Re: problem building spark on centos
>>>
>>> Which version of maven are you using ?
>>>
>>> It should be 3.3.3+
>>>
>>> On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com> wrote:
>>>
>>>> Hi, All:
>>>>
>>>> I’m trying to build spark 1.5.2 from source using maven with the
>>>> following command:
>>>>
>>>> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>>>> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
>>>>
>>>>
>>>> I got the following error:I'
>>>>
>>>> + VERSION='[ERROR] [Help 2]
>>>> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
>>>> '
>>>>
>>>>
>>>> When I try:
>>>>
>>>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests
>>>> clean package
>>>>
>>>>
>>>> I got the following error:
>>>>
>>>> [FATAL] Non-resolvable parent POM for
>>>> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact
>>>> org.apache:apache:pom:14 from/to central (
>>>> https://repo1.maven.org/maven2): java.security.ProviderException:
>>>> java.security.KeyException and 'parent.relativePath' points at wrong local
>>>> POM @ line 22, column 11
>>>>
>>>>
>>>> Does anyone know how to change the settings in maven to fix this?
>>>>
>>>>
>>>> Thanks in advance,
>>>>
>>>>
>>>> Jade
>>>>
>>>
>>>
>>
>
Re: problem building spark on centos
Posted by Marcelo Vanzin <va...@cloudera.com>.
If you're trying to compile against Scala 2.11, you're missing
"-Dscala-2.11" in that command.
On Wed, Jan 6, 2016 at 12:27 PM, Jade Liu <ja...@nor1.com> wrote:
> Hi, Todd:
>
> Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh
> 2.11 script when using scala version 2.11.
>
> I just tried this as you suggested:
> ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6
> -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: problem building spark on centos
Posted by Jade Liu <ja...@nor1.com>.
Hi, Todd:
Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh 2.11 script when using scala version 2.11.
I just tried this as you suggested:
./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
Still got the same error:
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-launcher_2.10
Do you think it’s java problem? I’m using oracle JDK 1.7. Should I update it to 1.8 instead? I just checked the conf and it says 1.7.
Thanks,
Jade
From: Todd Nist <ts...@gmail.com>>
Date: Wednesday, January 6, 2016 at 12:03 PM
To: Jade Liu <ja...@nor1.com>>
Cc: Ted Yu <yu...@gmail.com>>, "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Hi Jade,
I think you "--name" option. The makedistribution should look like this:
./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests.
As for why it failed to build with scala 2.11, did you run the ./dev/change-scala-version.sh 2.11 script to set the version of the artifacts to 2.11? If you do that then issue the build like this I think you will be ok:
./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11 -DskipTests
HTH.
-Todd
On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <ja...@nor1.com>> wrote:
I’ve changed the scala version to 2.10.
With this command:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
Build was successful.
But make a runnable version:
/make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
Still fails with the following error:
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
Not sure what’s causing it. Does anyone have any idea?
Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Wednesday, January 6, 2016 at 10:40 AM
To: Jade Liu <ja...@nor1.com>>, user <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
w.r.t. the second error, have you read this ?
http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com>> wrote:
I’m using 3.3.9. Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Tuesday, January 5, 2016 at 4:57 PM
To: Jade Liu <ja...@nor1.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Which version of maven are you using ?
It should be 3.3.3+
On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com>> wrote:
Hi, All:
I’m trying to build spark 1.5.2 from source using maven with the following command:
./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
I got the following error:I'
+ VERSION='[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException'
When I try:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
I got the following error:
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): java.security.ProviderException: java.security.KeyException and 'parent.relativePath' points at wrong local POM @ line 22, column 11
Does anyone know how to change the settings in maven to fix this?
Thanks in advance,
Jade
Re: problem building spark on centos
Posted by Todd Nist <ts...@gmail.com>.
Hi Jade,
I think you "--name" option. The makedistribution should look like this:
./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6
-Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests.
As for why it failed to build with scala 2.11, did you run the
./dev/change-scala-version.sh 2.11 script to set the version of the
artifacts to 2.11? If you do that then issue the build like this I think
you will be ok:
./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn
-Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11
-DskipTests
HTH.
-Todd
On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <ja...@nor1.com> wrote:
> I’ve changed the scala version to 2.10.
>
> With this command:
> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean
> package
> Build was successful.
>
> But make a runnable version:
> /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
> -Phive -Phive-thriftserver –DskipTests
> Still fails with the following error:
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
> on project spark-launcher_2.10: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
> -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile
> (scala-compile-first) on project spark-launcher_2.10: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
> scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> ... 20 more
> Caused by: Compile failed via zinc server
> at
> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
> at
> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
> at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
> at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
> at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>
> Not sure what’s causing it. Does anyone have any idea?
>
> Thanks!
>
> Jade
> From: Ted Yu <yu...@gmail.com>
> Date: Wednesday, January 6, 2016 at 10:40 AM
> To: Jade Liu <ja...@nor1.com>, user <us...@spark.apache.org>
>
> Subject: Re: problem building spark on centos
>
> w.r.t. the second error, have you read this ?
>
> http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
>
> On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com> wrote:
>
>> I’m using 3.3.9. Thanks!
>>
>> Jade
>>
>> From: Ted Yu <yu...@gmail.com>
>> Date: Tuesday, January 5, 2016 at 4:57 PM
>> To: Jade Liu <ja...@nor1.com>
>> Cc: "user@spark.apache.org" <us...@spark.apache.org>
>> Subject: Re: problem building spark on centos
>>
>> Which version of maven are you using ?
>>
>> It should be 3.3.3+
>>
>> On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com> wrote:
>>
>>> Hi, All:
>>>
>>> I’m trying to build spark 1.5.2 from source using maven with the
>>> following command:
>>>
>>> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>>> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
>>>
>>>
>>> I got the following error:I'
>>>
>>> + VERSION='[ERROR] [Help 2]
>>> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
>>> '
>>>
>>>
>>> When I try:
>>>
>>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests
>>> clean package
>>>
>>>
>>> I got the following error:
>>>
>>> [FATAL] Non-resolvable parent POM for
>>> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact
>>> org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2):
>>> java.security.ProviderException: java.security.KeyException and
>>> 'parent.relativePath' points at wrong local POM @ line 22, column 11
>>>
>>>
>>> Does anyone know how to change the settings in maven to fix this?
>>>
>>>
>>> Thanks in advance,
>>>
>>>
>>> Jade
>>>
>>
>>
>
Re: problem building spark on centos
Posted by Jade Liu <ja...@nor1.com>.
I’ve changed the scala version to 2.10.
With this command:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
Build was successful.
But make a runnable version:
/make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests
Still fails with the following error:
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: Compile failed via zinc server
at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136)
at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86)
at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
Not sure what’s causing it. Does anyone have any idea?
Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Wednesday, January 6, 2016 at 10:40 AM
To: Jade Liu <ja...@nor1.com>>, user <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
w.r.t. the second error, have you read this ?
http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com>> wrote:
I’m using 3.3.9. Thanks!
Jade
From: Ted Yu <yu...@gmail.com>>
Date: Tuesday, January 5, 2016 at 4:57 PM
To: Jade Liu <ja...@nor1.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: problem building spark on centos
Which version of maven are you using ?
It should be 3.3.3+
On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com>> wrote:
Hi, All:
I’m trying to build spark 1.5.2 from source using maven with the following command:
./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
I got the following error:I'
+ VERSION='[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException'
When I try:
build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
I got the following error:
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): java.security.ProviderException: java.security.KeyException and 'parent.relativePath' points at wrong local POM @ line 22, column 11
Does anyone know how to change the settings in maven to fix this?
Thanks in advance,
Jade
Re: problem building spark on centos
Posted by Ted Yu <yu...@gmail.com>.
w.r.t. the second error, have you read this ?
http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo
On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <ja...@nor1.com> wrote:
> I’m using 3.3.9. Thanks!
>
> Jade
>
> From: Ted Yu <yu...@gmail.com>
> Date: Tuesday, January 5, 2016 at 4:57 PM
> To: Jade Liu <ja...@nor1.com>
> Cc: "user@spark.apache.org" <us...@spark.apache.org>
> Subject: Re: problem building spark on centos
>
> Which version of maven are you using ?
>
> It should be 3.3.3+
>
> On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com> wrote:
>
>> Hi, All:
>>
>> I’m trying to build spark 1.5.2 from source using maven with the
>> following command:
>>
>> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
>> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
>>
>>
>> I got the following error:I'
>>
>> + VERSION='[ERROR] [Help 2]
>> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
>> '
>>
>>
>> When I try:
>>
>> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean
>> package
>>
>>
>> I got the following error:
>>
>> [FATAL] Non-resolvable parent POM for
>> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact
>> org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2):
>> java.security.ProviderException: java.security.KeyException and
>> 'parent.relativePath' points at wrong local POM @ line 22, column 11
>>
>>
>> Does anyone know how to change the settings in maven to fix this?
>>
>>
>> Thanks in advance,
>>
>>
>> Jade
>>
>
>
Re: problem building spark on centos
Posted by Ted Yu <yu...@gmail.com>.
Which version of maven are you using ?
It should be 3.3.3+
On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <ja...@nor1.com> wrote:
> Hi, All:
>
> I’m trying to build spark 1.5.2 from source using maven with the following
> command:
>
> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0
> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests
>
>
> I got the following error:
>
> + VERSION='[ERROR] [Help 2]
> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
> '
>
>
> When I try:
>
> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean
> package
>
>
> I got the following error:
>
> [FATAL] Non-resolvable parent POM for
> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact
> org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2):
> java.security.ProviderException: java.security.KeyException and
> 'parent.relativePath' points at wrong local POM @ line 22, column 11
>
>
> Does anyone know how to change the settings in maven to fix this?
>
>
> Thanks in advance,
>
>
> Jade
>