You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jacek Laskowski <ja...@japila.pl> on 2016/04/09 21:01:43 UTC
[BUILD FAILURE] Spark Project ML Local Library - me or it's real?
Hi,
Is this me or the build is broken today? I'm looking for help as it looks scary.
$ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.2 -Phive
-Phive-thriftserver -DskipTests clean install
[INFO] --- scala-maven-plugin:3.2.2:testCompile
(scala-test-compile-first) @ spark-mllib-local_2.11 ---
[INFO] Using zinc server for incremental compilation
[warn] Pruning sources from previous analysis, due to incompatible CompileSetup.
[info] Compiling 1 Scala source to
/Users/jacek/dev/oss/spark/mllib-local/target/scala-2.11/test-classes...
[error] missing or invalid dependency detected while loading class
file 'SparkFunSuite.class'.
[error] Could not access type Logging in package org.apache.spark.internal,
[error] because it (or its dependencies) are missing. Check your build
definition for
[error] missing or conflicting dependencies. (Re-run with
`-Ylog-classpath` to see the problematic classpath.)
[error] A full rebuild may help if 'SparkFunSuite.class' was compiled
against an incompatible version of org.apache.spark.internal.
[error] one error found
[error] Compile failed at Apr 9, 2016 2:27:30 PM [0.475s]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 4.338 s]
[INFO] Spark Project Test Tags ............................ SUCCESS [ 5.238 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 6.158 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 10.397 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.263 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 10.448 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 11.028 s]
[INFO] Spark Project Core ................................. SUCCESS [02:04 min]
[INFO] Spark Project GraphX ............................... SUCCESS [ 16.973 s]
[INFO] Spark Project Streaming ............................ SUCCESS [ 38.458 s]
[INFO] Spark Project Catalyst ............................. SUCCESS [01:18 min]
[INFO] Spark Project SQL .................................. SUCCESS [01:24 min]
[INFO] Spark Project ML Local Library ..................... FAILURE [ 1.083 s]
[INFO] Spark Project ML Library ........................... SKIPPED
[INFO] Spark Project Tools ................................ SKIPPED
[INFO] Spark Project Hive ................................. SKIPPED
[INFO] Spark Project Docker Integration Tests ............. SKIPPED
[INFO] Spark Project REPL ................................. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] Spark Project YARN ................................. SKIPPED
[INFO] Spark Project Hive Thrift Server ................... SKIPPED
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] Spark Project Java 8 Tests ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:39 min
[INFO] Finished at: 2016-04-09T14:27:30-04:00
[INFO] Final Memory: 79M/893M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
(scala-test-compile-first) on project spark-mllib-local_2.11:
Execution scala-test-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
CompileFailed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with
the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-mllib-local_2.11
Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [BUILD FAILURE] Spark Project ML Local Library - me or it's real?
Posted by Jacek Laskowski <ja...@japila.pl>.
Hi Ted et al,
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:41 min
[INFO] Finished at: 2016-04-09T19:21:02-04:00
[INFO] Final Memory: 106M/961M
[INFO] ------------------------------------------------------------------------
Thank you so much for the prompt solution! And that's while I was
driving from Toronto to Mississauga. Thanks!
Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski
On Sat, Apr 9, 2016 at 4:16 PM, Ted Yu <yu...@gmail.com> wrote:
> Sent PR:
> https://github.com/apache/spark/pull/12276
>
> I was able to get build going past mllib-local module.
>
> FYI
>
> On Sat, Apr 9, 2016 at 12:40 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>> The broken build was caused by the following:
>>
>> [SPARK-14462][ML][MLLIB] add the mllib-local build to maven pom
>>
>> See
>> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/607/
>>
>> FYI
>>
>> On Sat, Apr 9, 2016 at 12:01 PM, Jacek Laskowski <ja...@japila.pl> wrote:
>>>
>>> Hi,
>>>
>>> Is this me or the build is broken today? I'm looking for help as it looks
>>> scary.
>>>
>>> $ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.2 -Phive
>>> -Phive-thriftserver -DskipTests clean install
>>> [INFO] --- scala-maven-plugin:3.2.2:testCompile
>>> (scala-test-compile-first) @ spark-mllib-local_2.11 ---
>>> [INFO] Using zinc server for incremental compilation
>>> [warn] Pruning sources from previous analysis, due to incompatible
>>> CompileSetup.
>>> [info] Compiling 1 Scala source to
>>> /Users/jacek/dev/oss/spark/mllib-local/target/scala-2.11/test-classes...
>>> [error] missing or invalid dependency detected while loading class
>>> file 'SparkFunSuite.class'.
>>> [error] Could not access type Logging in package
>>> org.apache.spark.internal,
>>> [error] because it (or its dependencies) are missing. Check your build
>>> definition for
>>> [error] missing or conflicting dependencies. (Re-run with
>>> `-Ylog-classpath` to see the problematic classpath.)
>>> [error] A full rebuild may help if 'SparkFunSuite.class' was compiled
>>> against an incompatible version of org.apache.spark.internal.
>>> [error] one error found
>>> [error] Compile failed at Apr 9, 2016 2:27:30 PM [0.475s]
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Reactor Summary:
>>> [INFO]
>>> [INFO] Spark Project Parent POM ........................... SUCCESS [
>>> 4.338 s]
>>> [INFO] Spark Project Test Tags ............................ SUCCESS [
>>> 5.238 s]
>>> [INFO] Spark Project Sketch ............................... SUCCESS [
>>> 6.158 s]
>>> [INFO] Spark Project Networking ........................... SUCCESS [
>>> 10.397 s]
>>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
>>> 7.263 s]
>>> [INFO] Spark Project Unsafe ............................... SUCCESS [
>>> 10.448 s]
>>> [INFO] Spark Project Launcher ............................. SUCCESS [
>>> 11.028 s]
>>> [INFO] Spark Project Core ................................. SUCCESS
>>> [02:04 min]
>>> [INFO] Spark Project GraphX ............................... SUCCESS [
>>> 16.973 s]
>>> [INFO] Spark Project Streaming ............................ SUCCESS [
>>> 38.458 s]
>>> [INFO] Spark Project Catalyst ............................. SUCCESS
>>> [01:18 min]
>>> [INFO] Spark Project SQL .................................. SUCCESS
>>> [01:24 min]
>>> [INFO] Spark Project ML Local Library ..................... FAILURE [
>>> 1.083 s]
>>> [INFO] Spark Project ML Library ........................... SKIPPED
>>> [INFO] Spark Project Tools ................................ SKIPPED
>>> [INFO] Spark Project Hive ................................. SKIPPED
>>> [INFO] Spark Project Docker Integration Tests ............. SKIPPED
>>> [INFO] Spark Project REPL ................................. SKIPPED
>>> [INFO] Spark Project YARN Shuffle Service ................. SKIPPED
>>> [INFO] Spark Project YARN ................................. SKIPPED
>>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
>>> [INFO] Spark Project Assembly ............................. SKIPPED
>>> [INFO] Spark Project External Flume Sink .................. SKIPPED
>>> [INFO] Spark Project External Flume ....................... SKIPPED
>>> [INFO] Spark Project External Flume Assembly .............. SKIPPED
>>> [INFO] Spark Project External Kafka ....................... SKIPPED
>>> [INFO] Spark Project Examples ............................. SKIPPED
>>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
>>> [INFO] Spark Project Java 8 Tests ......................... SKIPPED
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 06:39 min
>>> [INFO] Finished at: 2016-04-09T14:27:30-04:00
>>> [INFO] Final Memory: 79M/893M
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] Failed to execute goal
>>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
>>> (scala-test-compile-first) on project spark-mllib-local_2.11:
>>> Execution scala-test-compile-first of goal
>>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
>>> CompileFailed -> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>> the -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
>>> [ERROR]
>>> [ERROR] After correcting the problems, you can resume the build with the
>>> command
>>> [ERROR] mvn <goals> -rf :spark-mllib-local_2.11
>>>
>>> Pozdrawiam,
>>> Jacek Laskowski
>>> ----
>>> https://medium.com/@jaceklaskowski/
>>> Mastering Apache Spark http://bit.ly/mastering-apache-spark
>>> Follow me at https://twitter.com/jaceklaskowski
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [BUILD FAILURE] Spark Project ML Local Library - me or it's real?
Posted by Ted Yu <yu...@gmail.com>.
Sent PR:
https://github.com/apache/spark/pull/12276
I was able to get build going past mllib-local module.
FYI
On Sat, Apr 9, 2016 at 12:40 PM, Ted Yu <yu...@gmail.com> wrote:
> The broken build was caused by the following:
>
> [SPARK-14462][ML][MLLIB] add the mllib-local build to maven pom
>
> See
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/607/
>
> FYI
>
> On Sat, Apr 9, 2016 at 12:01 PM, Jacek Laskowski <ja...@japila.pl> wrote:
>
>> Hi,
>>
>> Is this me or the build is broken today? I'm looking for help as it looks
>> scary.
>>
>> $ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.2 -Phive
>> -Phive-thriftserver -DskipTests clean install
>> [INFO] --- scala-maven-plugin:3.2.2:testCompile
>> (scala-test-compile-first) @ spark-mllib-local_2.11 ---
>> [INFO] Using zinc server for incremental compilation
>> [warn] Pruning sources from previous analysis, due to incompatible
>> CompileSetup.
>> [info] Compiling 1 Scala source to
>> /Users/jacek/dev/oss/spark/mllib-local/target/scala-2.11/test-classes...
>> [error] missing or invalid dependency detected while loading class
>> file 'SparkFunSuite.class'.
>> [error] Could not access type Logging in package
>> org.apache.spark.internal,
>> [error] because it (or its dependencies) are missing. Check your build
>> definition for
>> [error] missing or conflicting dependencies. (Re-run with
>> `-Ylog-classpath` to see the problematic classpath.)
>> [error] A full rebuild may help if 'SparkFunSuite.class' was compiled
>> against an incompatible version of org.apache.spark.internal.
>> [error] one error found
>> [error] Compile failed at Apr 9, 2016 2:27:30 PM [0.475s]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Spark Project Parent POM ........................... SUCCESS [
>> 4.338 s]
>> [INFO] Spark Project Test Tags ............................ SUCCESS [
>> 5.238 s]
>> [INFO] Spark Project Sketch ............................... SUCCESS [
>> 6.158 s]
>> [INFO] Spark Project Networking ........................... SUCCESS [
>> 10.397 s]
>> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
>> 7.263 s]
>> [INFO] Spark Project Unsafe ............................... SUCCESS [
>> 10.448 s]
>> [INFO] Spark Project Launcher ............................. SUCCESS [
>> 11.028 s]
>> [INFO] Spark Project Core ................................. SUCCESS
>> [02:04 min]
>> [INFO] Spark Project GraphX ............................... SUCCESS [
>> 16.973 s]
>> [INFO] Spark Project Streaming ............................ SUCCESS [
>> 38.458 s]
>> [INFO] Spark Project Catalyst ............................. SUCCESS
>> [01:18 min]
>> [INFO] Spark Project SQL .................................. SUCCESS
>> [01:24 min]
>> [INFO] Spark Project ML Local Library ..................... FAILURE [
>> 1.083 s]
>> [INFO] Spark Project ML Library ........................... SKIPPED
>> [INFO] Spark Project Tools ................................ SKIPPED
>> [INFO] Spark Project Hive ................................. SKIPPED
>> [INFO] Spark Project Docker Integration Tests ............. SKIPPED
>> [INFO] Spark Project REPL ................................. SKIPPED
>> [INFO] Spark Project YARN Shuffle Service ................. SKIPPED
>> [INFO] Spark Project YARN ................................. SKIPPED
>> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
>> [INFO] Spark Project Assembly ............................. SKIPPED
>> [INFO] Spark Project External Flume Sink .................. SKIPPED
>> [INFO] Spark Project External Flume ....................... SKIPPED
>> [INFO] Spark Project External Flume Assembly .............. SKIPPED
>> [INFO] Spark Project External Kafka ....................... SKIPPED
>> [INFO] Spark Project Examples ............................. SKIPPED
>> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
>> [INFO] Spark Project Java 8 Tests ......................... SKIPPED
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 06:39 min
>> [INFO] Finished at: 2016-04-09T14:27:30-04:00
>> [INFO] Final Memory: 79M/893M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
>> (scala-test-compile-first) on project spark-mllib-local_2.11:
>> Execution scala-test-compile-first of goal
>> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
>> CompileFailed -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>> the -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :spark-mllib-local_2.11
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> ----
>> https://medium.com/@jaceklaskowski/
>> Mastering Apache Spark http://bit.ly/mastering-apache-spark
>> Follow me at https://twitter.com/jaceklaskowski
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: [BUILD FAILURE] Spark Project ML Local Library - me or it's real?
Posted by Ted Yu <yu...@gmail.com>.
The broken build was caused by the following:
[SPARK-14462][ML][MLLIB] add the mllib-local build to maven pom
See
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/607/
FYI
On Sat, Apr 9, 2016 at 12:01 PM, Jacek Laskowski <ja...@japila.pl> wrote:
> Hi,
>
> Is this me or the build is broken today? I'm looking for help as it looks
> scary.
>
> $ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.2 -Phive
> -Phive-thriftserver -DskipTests clean install
> [INFO] --- scala-maven-plugin:3.2.2:testCompile
> (scala-test-compile-first) @ spark-mllib-local_2.11 ---
> [INFO] Using zinc server for incremental compilation
> [warn] Pruning sources from previous analysis, due to incompatible
> CompileSetup.
> [info] Compiling 1 Scala source to
> /Users/jacek/dev/oss/spark/mllib-local/target/scala-2.11/test-classes...
> [error] missing or invalid dependency detected while loading class
> file 'SparkFunSuite.class'.
> [error] Could not access type Logging in package org.apache.spark.internal,
> [error] because it (or its dependencies) are missing. Check your build
> definition for
> [error] missing or conflicting dependencies. (Re-run with
> `-Ylog-classpath` to see the problematic classpath.)
> [error] A full rebuild may help if 'SparkFunSuite.class' was compiled
> against an incompatible version of org.apache.spark.internal.
> [error] one error found
> [error] Compile failed at Apr 9, 2016 2:27:30 PM [0.475s]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM ........................... SUCCESS [
> 4.338 s]
> [INFO] Spark Project Test Tags ............................ SUCCESS [
> 5.238 s]
> [INFO] Spark Project Sketch ............................... SUCCESS [
> 6.158 s]
> [INFO] Spark Project Networking ........................... SUCCESS [
> 10.397 s]
> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [
> 7.263 s]
> [INFO] Spark Project Unsafe ............................... SUCCESS [
> 10.448 s]
> [INFO] Spark Project Launcher ............................. SUCCESS [
> 11.028 s]
> [INFO] Spark Project Core ................................. SUCCESS [02:04
> min]
> [INFO] Spark Project GraphX ............................... SUCCESS [
> 16.973 s]
> [INFO] Spark Project Streaming ............................ SUCCESS [
> 38.458 s]
> [INFO] Spark Project Catalyst ............................. SUCCESS [01:18
> min]
> [INFO] Spark Project SQL .................................. SUCCESS [01:24
> min]
> [INFO] Spark Project ML Local Library ..................... FAILURE [
> 1.083 s]
> [INFO] Spark Project ML Library ........................... SKIPPED
> [INFO] Spark Project Tools ................................ SKIPPED
> [INFO] Spark Project Hive ................................. SKIPPED
> [INFO] Spark Project Docker Integration Tests ............. SKIPPED
> [INFO] Spark Project REPL ................................. SKIPPED
> [INFO] Spark Project YARN Shuffle Service ................. SKIPPED
> [INFO] Spark Project YARN ................................. SKIPPED
> [INFO] Spark Project Hive Thrift Server ................... SKIPPED
> [INFO] Spark Project Assembly ............................. SKIPPED
> [INFO] Spark Project External Flume Sink .................. SKIPPED
> [INFO] Spark Project External Flume ....................... SKIPPED
> [INFO] Spark Project External Flume Assembly .............. SKIPPED
> [INFO] Spark Project External Kafka ....................... SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
> [INFO] Spark Project Java 8 Tests ......................... SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 06:39 min
> [INFO] Finished at: 2016-04-09T14:27:30-04:00
> [INFO] Final Memory: 79M/893M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile
> (scala-test-compile-first) on project spark-mllib-local_2.11:
> Execution scala-test-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed.
> CompileFailed -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with
> the -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :spark-mllib-local_2.11
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://medium.com/@jaceklaskowski/
> Mastering Apache Spark http://bit.ly/mastering-apache-spark
> Follow me at https://twitter.com/jaceklaskowski
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>