You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "shane knapp (JIRA)" <ji...@apache.org> on 2014/10/21 05:13:35 UTC
[jira] [Commented] (SPARK-4021) Issues observed after upgrading
Jenkins to JDK7u71
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177872#comment-14177872 ]
shane knapp commented on SPARK-4021:
------------------------------------
thing thing that seemed to trigger this failure was when i added a second JDK, via the jenkins management interface (7u71). after that, the JAVA_HOME env variable (in a jenkins job) disappeared and the spark jobs started failing. there were no job config changes, or anything related to this on the systems themselves. as well, i am absolutely, 100% certain that i did not click/check/configure/whatever anything related to java 5.
these failures may be isolated to spark, as SparkR passed a build after i added that second JDK and triggered it manually:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/
regarding the disappearing JAVA_HOME environment variables, compare:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21929/injectedEnvVars/ (passed, just before change)
to:
https://amplab.cs.berkeley.edu/jenkins/job/SparkR/74/injectedEnvVars/ (passed, right after change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21930/injectedEnvVars/ (failed, right after change)
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21941/injectedEnvVars/ (passed, after i removed 7u71)
i'll look in to this tomorrow and see if i can recreate. it seems that somehow the JAVA_HOME env got clobbered, and jenkins used some random java on the build agent. to test this, i'll reinstall 7u71, and create a PR that runs 'java -version' from run-tests-jenkins. once the build kicks off, i'll remove that JDK so that we don't mess anything else up (this is also a good impetus to start scrounging some resources up for a staging instance so i don't have to test this on prod).
if we're looking at an infrastructure problem, rather than code, i'll close this out.
> Issues observed after upgrading Jenkins to JDK7u71
> --------------------------------------------------
>
> Key: SPARK-4021
> URL: https://issues.apache.org/jira/browse/SPARK-4021
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Environment: JDK 7u71
> Reporter: Patrick Wendell
>
> The following compile failure was observed after adding JDK7u71 to Jenkins. However, this is likely a misconfiguration from Jenkins rather than an issue with Spark (these errors are specific to JDK5, in fact).
> {code}
> [error] ----------
> [error] 1. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 83)
> [error] private static final Logger logger = Logger.getLogger(JavaKinesisWordCountASL.class);
> [error] ^^^^^^
> [error] The field JavaKinesisWordCountASL.logger is never read locally
> [error] ----------
> [error] 2. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 151)
> [error] JavaDStream<String> words = unionStreams.flatMap(new FlatMapFunction<byte[], String>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field of type long
> [error] ----------
> [error] 3. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 153)
> [error] public Iterable<String> call(byte[] line) {
> [error] ^^^^^^^^^^^^^^^^^
> [error] The method call(byte[]) of type new FlatMapFunction<byte[],String>(){} must override a superclass method
> [error] ----------
> [error] 4. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 160)
> [error] new PairFunction<String, String, Integer>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field of type long
> [error] ----------
> [error] 5. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 162)
> [error] public Tuple2<String, Integer> call(String s) {
> [error] ^^^^^^^^^^^^^^
> [error] The method call(String) of type new PairFunction<String,String,Integer>(){} must override a superclass method
> [error] ----------
> [error] 6. WARNING in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 165)
> [error] }).reduceByKey(new Function2<Integer, Integer, Integer>() {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The serializable class does not declare a static final serialVersionUID field of type long
> [error] ----------
> [error] 7. ERROR in /home/jenkins/workspace/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE/hadoop1.0/label/centos/extras/kinesis-asl/src/main/java/org/apache/spark/examples/streaming/JavaKinesisWordCountASL.java (at line 167)
> [error] public Integer call(Integer i1, Integer i2) {
> [error] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [error] The method call(Integer, Integer) of type new Function2<Integer,Integer,Integer>(){} must override a superclass method
> [error] ----------
> [error] 7 problems (3 errors, 4 warnings)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org