You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Ricky <49...@qq.com> on 2015/11/09 15:57:02 UTC
回复： [VOTE] Release Apache Spark 1.5.2 (RC2)
Now I try the spark-1.5.2-rc2.zip from githup ,the result also has errors .
[root@ouyangshourui spark-1.5.2-rc2]# pwd
/SparkCode/spark-1.5.2-rc2
[root@ouyangshourui spark-1.5.2-rc2]# nohup mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive -Phive-thriftserver -DskipTests clean package &
The error log as following:
[INFO] Building jar: /SparkCode/spark-1.5.2-rc2/external/flume-assembly/target/spark-streaming-flume-assembly_2.10-1.5.2-test-sources.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project External MQTT 1.5.2
[INFO] ------------------------------------------------------------------------
Downloading: http://maven.oschina.net/content/groups/public/org/eclipse/paho/org.eclipse.paho.client.mqttv3/1.0.1/org.eclipse.paho.client.mqttv3-1.0.1.pom
[WARNING] The POM for org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 is missing, no dependency information available
Downloading: http://maven.oschina.net/content/groups/public/org/eclipse/paho/org.eclipse.paho.client.mqttv3/1.0.1/org.eclipse.paho.client.mqttv3-1.0.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 13.576 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 19.966 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 11.279 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 5.353 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 20.199 s]
[INFO] Spark Project Core ................................. SUCCESS [04:18 min]
[INFO] Spark Project Bagel ................................ SUCCESS [ 27.070 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:09 min]
[INFO] Spark Project Streaming ............................ SUCCESS [01:57 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [02:21 min]
[INFO] Spark Project SQL .................................. SUCCESS [02:50 min]
[INFO] Spark Project ML Library ........................... SUCCESS [03:01 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 13.731 s]
[INFO] Spark Project Hive ................................. SUCCESS [02:06 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 42.023 s]
[INFO] Spark Project YARN ................................. SUCCESS [ 56.501 s]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [ 53.986 s]
[INFO] Spark Project Assembly ............................. SUCCESS [01:58 min]
[INFO] Spark Project External Twitter ..................... SUCCESS [ 18.626 s]
[INFO] Spark Project External Flume Sink .................. SUCCESS [ 34.569 s]
[INFO] Spark Project External Flume ....................... SUCCESS [ 29.643 s]
[INFO] Spark Project External Flume Assembly .............. SUCCESS [ 4.430 s]
[INFO] Spark Project External MQTT ........................ FAILURE [ 5.822 s]
[INFO] Spark Project External MQTT Assembly ............... SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25:41 min
[INFO] Finished at: 2015-11-09T09:45:27-05:00
[INFO] Final Memory: 73M/1041M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project spark-streaming-mqtt_2.10: Could not resolve dependencies for project org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.2: Could not find artifact org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 in nexus-osc (http://maven.oschina.net/content/groups/public/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-streaming-mqtt_2.10
Best Regards
Ricky Yang(欧阳守锐)
------------------ 原始邮件 ------------------
发件人: "Sean Owen";<so...@cloudera.com>;
发送时间: 2015年11月9日(星期一) 下午2:39
收件人: "Ricky"<49...@qq.com>;
抄送: "Denny Lee"<de...@gmail.com>; "Mark Hamstra"<ma...@clearstorydata.com>; "Reynold Xin"<rx...@databricks.com>; "dev@spark.apache.org"<de...@spark.apache.org>;
主题: Re: [VOTE] Release Apache Spark 1.5.2 (RC2)
Looks like you are building a module without install-ing other
modules. That won't work in general in Maven. Also, it looks like you
are building a snapshot, not the release we are talking about.
On Mon, Nov 9, 2015 at 3:28 AM, 欧锐 <49...@qq.com> wrote:
>
> build spark-streaming-mqtt_2.10 failed!
>
> nohup mvn -X -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
> -Phive-thriftserver -DskipTests clean package -rf :spark-streaming-mqtt_2.10
> &
>
> [DEBUG] org.scala-tools.testing:test-interface:jar:0.5:test
> [DEBUG] org.apache.activemq:activemq-core:jar:5.7.0:test
> [DEBUG] org.apache.geronimo.specs:geronimo-jms_1.1_spec:jar:1.1.1:test
> [DEBUG] org.apache.activemq:kahadb:jar:5.7.0:test
> [DEBUG] org.apache.activemq.protobuf:activemq-protobuf:jar:1.1:test
> [DEBUG] org.fusesource.mqtt-client:mqtt-client:jar:1.3:test
> [DEBUG] org.fusesource.hawtdispatch:hawtdispatch-transport:jar:1.11:test
> [DEBUG] org.fusesource.hawtdispatch:hawtdispatch:jar:1.11:test
> [DEBUG] org.fusesource.hawtbuf:hawtbuf:jar:1.9:test
> [DEBUG]
> org.apache.geronimo.specs:geronimo-j2ee-management_1.1_spec:jar:1.0.1:test
> [DEBUG] org.springframework:spring-context:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-aop:jar:3.0.7.RELEASE:test
> [DEBUG] aopalliance:aopalliance:jar:1.0:test
> [DEBUG] org.springframework:spring-beans:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-core:jar:3.0.7.RELEASE:test
> [DEBUG] commons-logging:commons-logging:jar:1.1.1:test
> [DEBUG] org.springframework:spring-expression:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-asm:jar:3.0.7.RELEASE:test
> [DEBUG] org.jasypt:jasypt:jar:1.9.0:test
> [DEBUG] org.spark-project.spark:unused:jar:1.0.0:compile
> [DEBUG] org.scalatest:scalatest_2.10:jar:2.2.1:test
> [DEBUG] org.scala-lang:scala-reflect:jar:2.10.4:provided
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project External MQTT ........................ FAILURE [ 2.403
> s]
> [INFO] Spark Project External MQTT Assembly ............... SKIPPED
> [INFO] Spark Project External ZeroMQ ...................... SKIPPED
> [INFO] Spark Project External Kafka ....................... SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 4.471 s
> [INFO] Finished at: 2015-11-09T11:10:57+08:00
> [INFO] Final Memory: 31M/173M
> [INFO]
> ------------------------------------------------------------------------
> [WARNING] The requested profile "hive" could not be activated because it
> does not exist.
> [ERROR] Failed to execute goal on project spark-streaming-mqtt_2.10: Could
> not resolve dependencies for project
> org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.0-SNAPSHOT: The following
> artifacts could not be resolved:
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:tests:1.5.0-SNAPSHOT,
> org.apache.spark:spark-launcher_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-network-common_2.10:jar:1.5.0-SNAPSHOT,
> org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1: Failure to find
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-20150818.023902-334 in
> http://maven.cnsuning.com/content/groups/public/ was cached in the local
> repository, resolution will not be reattempted until the update interval of
> suning_maven_repo has elapsed or updates are forced -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project spark-streaming-mqtt_2.10: Could not resolve dependencies
> for project org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.0-SNAPSHOT:
> The following artifacts could not be resolved:
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:tests:1.5.0-SNAPSHOT,
> org.apache.spark:spark-launcher_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-network-common_2.10:jar:1.5.0-SNAPSHOT,
> org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1: Failure to find
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-20150818.023902-334 in
> http://maven.cnsuning.com/content/groups/public/ was cached in the local
> repository, resolution will not be reattempted until the update interval of
> suning_maven_repo has elapsed or updates are forced
> 发自我的iPhone
>
>
回复： [VOTE] Release Apache Spark 1.5.2 (RC2)
Posted by Ricky <49...@qq.com>.
thank for your help,do as you said,the problem is firewall issues,when changing to maven default repo (http://repo1.maven.org) from http://maven.oschina.net, spark-streaming-mqtt_2.10 module compiled Successfully.
------------------
Best Regards
------------------ 原始邮件 ------------------
发件人: "欧锐";<49...@qq.com>;
发送时间: 2015年11月9日(星期一) 晚上10:57
收件人: "Sean Owen"<so...@cloudera.com>; "Ted Yu"<yu...@gmail.com>; "Krishna Sankar"<ks...@gmail.com>;
抄送: "Denny Lee"<de...@gmail.com>; "Mark Hamstra"<ma...@clearstorydata.com>; "Reynold Xin"<rx...@databricks.com>; "dev@spark.apache.org"<de...@spark.apache.org>;
主题: 回复： [VOTE] Release Apache Spark 1.5.2 (RC2)
Now I try the spark-1.5.2-rc2.zip from githup ,the result also has errors .
[root@ouyangshourui spark-1.5.2-rc2]# pwd
/SparkCode/spark-1.5.2-rc2
[root@ouyangshourui spark-1.5.2-rc2]# nohup mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive -Phive-thriftserver -DskipTests clean package &
The error log as following:
[INFO] Building jar: /SparkCode/spark-1.5.2-rc2/external/flume-assembly/target/spark-streaming-flume-assembly_2.10-1.5.2-test-sources.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project External MQTT 1.5.2
[INFO] ------------------------------------------------------------------------
Downloading: http://maven.oschina.net/content/groups/public/org/eclipse/paho/org.eclipse.paho.client.mqttv3/1.0.1/org.eclipse.paho.client.mqttv3-1.0.1.pom
[WARNING] The POM for org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 is missing, no dependency information available
Downloading: http://maven.oschina.net/content/groups/public/org/eclipse/paho/org.eclipse.paho.client.mqttv3/1.0.1/org.eclipse.paho.client.mqttv3-1.0.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 13.576 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 19.966 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 11.279 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 5.353 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 20.199 s]
[INFO] Spark Project Core ................................. SUCCESS [04:18 min]
[INFO] Spark Project Bagel ................................ SUCCESS [ 27.070 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:09 min]
[INFO] Spark Project Streaming ............................ SUCCESS [01:57 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [02:21 min]
[INFO] Spark Project SQL .................................. SUCCESS [02:50 min]
[INFO] Spark Project ML Library ........................... SUCCESS [03:01 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 13.731 s]
[INFO] Spark Project Hive ................................. SUCCESS [02:06 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 42.023 s]
[INFO] Spark Project YARN ................................. SUCCESS [ 56.501 s]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [ 53.986 s]
[INFO] Spark Project Assembly ............................. SUCCESS [01:58 min]
[INFO] Spark Project External Twitter ..................... SUCCESS [ 18.626 s]
[INFO] Spark Project External Flume Sink .................. SUCCESS [ 34.569 s]
[INFO] Spark Project External Flume ....................... SUCCESS [ 29.643 s]
[INFO] Spark Project External Flume Assembly .............. SUCCESS [ 4.430 s]
[INFO] Spark Project External MQTT ........................ FAILURE [ 5.822 s]
[INFO] Spark Project External MQTT Assembly ............... SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project External Kafka ....................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project External Kafka Assembly .............. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25:41 min
[INFO] Finished at: 2015-11-09T09:45:27-05:00
[INFO] Final Memory: 73M/1041M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project spark-streaming-mqtt_2.10: Could not resolve dependencies for project org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.2: Could not find artifact org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1 in nexus-osc (http://maven.oschina.net/content/groups/public/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :spark-streaming-mqtt_2.10
Best Regards
Ricky Yang(欧阳守锐)
------------------ 原始邮件 ------------------
发件人: "Sean Owen";<so...@cloudera.com>;
发送时间: 2015年11月9日(星期一) 下午2:39
收件人: "Ricky"<49...@qq.com>;
抄送: "Denny Lee"<de...@gmail.com>; "Mark Hamstra"<ma...@clearstorydata.com>; "Reynold Xin"<rx...@databricks.com>; "dev@spark.apache.org"<de...@spark.apache.org>;
主题: Re: [VOTE] Release Apache Spark 1.5.2 (RC2)
Looks like you are building a module without install-ing other
modules. That won't work in general in Maven. Also, it looks like you
are building a snapshot, not the release we are talking about.
On Mon, Nov 9, 2015 at 3:28 AM, 欧锐 <49...@qq.com> wrote:
>
> build spark-streaming-mqtt_2.10 failed!
>
> nohup mvn -X -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive
> -Phive-thriftserver -DskipTests clean package -rf :spark-streaming-mqtt_2.10
> &
>
> [DEBUG] org.scala-tools.testing:test-interface:jar:0.5:test
> [DEBUG] org.apache.activemq:activemq-core:jar:5.7.0:test
> [DEBUG] org.apache.geronimo.specs:geronimo-jms_1.1_spec:jar:1.1.1:test
> [DEBUG] org.apache.activemq:kahadb:jar:5.7.0:test
> [DEBUG] org.apache.activemq.protobuf:activemq-protobuf:jar:1.1:test
> [DEBUG] org.fusesource.mqtt-client:mqtt-client:jar:1.3:test
> [DEBUG] org.fusesource.hawtdispatch:hawtdispatch-transport:jar:1.11:test
> [DEBUG] org.fusesource.hawtdispatch:hawtdispatch:jar:1.11:test
> [DEBUG] org.fusesource.hawtbuf:hawtbuf:jar:1.9:test
> [DEBUG]
> org.apache.geronimo.specs:geronimo-j2ee-management_1.1_spec:jar:1.0.1:test
> [DEBUG] org.springframework:spring-context:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-aop:jar:3.0.7.RELEASE:test
> [DEBUG] aopalliance:aopalliance:jar:1.0:test
> [DEBUG] org.springframework:spring-beans:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-core:jar:3.0.7.RELEASE:test
> [DEBUG] commons-logging:commons-logging:jar:1.1.1:test
> [DEBUG] org.springframework:spring-expression:jar:3.0.7.RELEASE:test
> [DEBUG] org.springframework:spring-asm:jar:3.0.7.RELEASE:test
> [DEBUG] org.jasypt:jasypt:jar:1.9.0:test
> [DEBUG] org.spark-project.spark:unused:jar:1.0.0:compile
> [DEBUG] org.scalatest:scalatest_2.10:jar:2.2.1:test
> [DEBUG] org.scala-lang:scala-reflect:jar:2.10.4:provided
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project External MQTT ........................ FAILURE [ 2.403
> s]
> [INFO] Spark Project External MQTT Assembly ............... SKIPPED
> [INFO] Spark Project External ZeroMQ ...................... SKIPPED
> [INFO] Spark Project External Kafka ....................... SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Project External Kafka Assembly .............. SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 4.471 s
> [INFO] Finished at: 2015-11-09T11:10:57+08:00
> [INFO] Final Memory: 31M/173M
> [INFO]
> ------------------------------------------------------------------------
> [WARNING] The requested profile "hive" could not be activated because it
> does not exist.
> [ERROR] Failed to execute goal on project spark-streaming-mqtt_2.10: Could
> not resolve dependencies for project
> org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.0-SNAPSHOT: The following
> artifacts could not be resolved:
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:tests:1.5.0-SNAPSHOT,
> org.apache.spark:spark-launcher_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-network-common_2.10:jar:1.5.0-SNAPSHOT,
> org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1: Failure to find
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-20150818.023902-334 in
> http://maven.cnsuning.com/content/groups/public/ was cached in the local
> repository, resolution will not be reattempted until the update interval of
> suning_maven_repo has elapsed or updates are forced -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal on project spark-streaming-mqtt_2.10: Could not resolve dependencies
> for project org.apache.spark:spark-streaming-mqtt_2.10:jar:1.5.0-SNAPSHOT:
> The following artifacts could not be resolved:
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-core_2.10:jar:tests:1.5.0-SNAPSHOT,
> org.apache.spark:spark-launcher_2.10:jar:1.5.0-SNAPSHOT,
> org.apache.spark:spark-network-common_2.10:jar:1.5.0-SNAPSHOT,
> org.eclipse.paho:org.eclipse.paho.client.mqttv3:jar:1.0.1: Failure to find
> org.apache.spark:spark-streaming_2.10:jar:1.5.0-20150818.023902-334 in
> http://maven.cnsuning.com/content/groups/public/ was cached in the local
> repository, resolution will not be reattempted until the update interval of
> suning_maven_repo has elapsed or updates are forced
> 发自我的iPhone
>
>