You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by srowen <gi...@git.apache.org> on 2014/08/02 01:53:43 UTC
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/1726
SPARK-2798 [BUILD] Jenkins build failing due to missing scalatest in flume-sink module
The build is still not quite passing even after https://issues.apache.org/jira/browse/SPARK-2749
See for example:
https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-pre-YARN/lastFailedBuild/hadoop.version=1.0.4,label=centos/console
It's the Flume Sink module that's failing, and I couldn't see the error until I ran it for myself and it seems to be:
```
Error: Could not find or load main class org.scalatest.tools.Runner
```
scalatest is a test dependency of other modules but not this one.
This PR has a change to just add the scalatest dep, but, also one to touch up a few other potential issues with this POM, like an Avro version mismatch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-2798
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1726.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1726
----
commit 9d4f98bbec0238419dde0324ddb2f31f1076efeb
Author: Sean Owen <sr...@gmail.com>
Date: 2014-08-01T23:50:55Z
Add scalatest to let module tests work
commit f4ad2150a0ce6389bf5c8833ecdc4d3273a4453f
Author: Sean Owen <sr...@gmail.com>
Date: 2014-08-01T23:51:37Z
Standardize flume, avro version across the build too
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53140142
@srowen I left the other fixes to the pom.xml for you in this PR. Once https://github.com/apache/spark/pull/2101 is merged, can you updated this PR. I will merge it before 1.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-52906646
@pwendell Could you take a look? @tdas approves and I think this may resolve https://github.com/apache/spark/pull/2075 too
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51313961
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626770
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Seems like an open maven bug: https://jira.codehaus.org/browse/MNG-1378
Does not seem like there is scope for resolution on that one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51319235
QA results for PR 1726:<br>- This patch PASSES unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18017/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626588
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
The sink does not depend on streaming either. The streaming jar should not even be dependency - even for test only the streaming test jar is required:
[INFO] org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.1.0-SNAPSHOT
[INFO] +- org.apache.flume:flume-ng-sdk:jar:1.4.0:compile
[INFO] | +- org.apache.avro:avro:jar:1.7.6:compile
[INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] | | +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
[INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | | \- org.tukaani:xz:jar:1.0:compile
[INFO] | \- org.apache.avro:avro-ipc:jar:1.7.6:compile
[INFO] +- org.apache.flume:flume-ng-core:jar:1.4.0:compile
[INFO] | +- org.apache.flume:flume-ng-configuration:jar:1.4.0:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
[INFO] | +- com.google.guava:guava:jar:14.0.1:provided
[INFO] | +- commons-io:commons-io:jar:2.1:compile
[INFO] | +- commons-codec:commons-codec:jar:1.5:compile
[INFO] | +- log4j:log4j:jar:1.2.17:compile
[INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- commons-lang:commons-lang:jar:2.5:compile
[INFO] | +- joda-time:joda-time:jar:2.1:compile
[INFO] | +- org.mortbay.jetty:servlet-api:jar:2.5-20110124:compile
[INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.2:compile
[INFO] | \- org.apache.mina:mina-core:jar:2.0.4:compile
[INFO] +- org.scala-lang:scala-library:jar:2.10.4:compile
[INFO] +- org.scalatest:scalatest_2.10:jar:2.1.5:test
[INFO] | \- org.scala-lang:scala-reflect:jar:2.10.4:test
[INFO] \- org.apache.spark:spark-streaming_2.10:test-jar:tests:1.1.0-SNAPSHOT:test
[INFO] +- org.apache.spark:spark-core_2.10:jar:1.1.0-SNAPSHOT:test
[INFO] | +- org.apache.hadoop:hadoop-client:jar:1.0.4:test
[INFO] | | \- org.apache.hadoop:hadoop-core:jar:1.0.4:test
[INFO] | | +- xmlenc:xmlenc:jar:0.52:test
[INFO] | | +- org.apache.commons:commons-math:jar:2.1:test
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:test
[INFO] | | | +- commons-collections:commons-collections:jar:3.2.1:test
[INFO] | | | +- commons-digester:commons-digester:jar:1.8:test
[INFO] | | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:test
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:test
[INFO] | | +- commons-el:commons-el:jar:1.0:test
[INFO] | | +- hsqldb:hsqldb:jar:1.8.0.10:test
[INFO] | | \- oro:oro:jar:2.0.8:test
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:test
[INFO] | | \- commons-httpclient:commons-httpclient:jar:3.1:test
[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:test
[INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:test
[INFO] | | | \- org.apache.curator:curator-client:jar:2.4.0:test
[INFO] | | \- org.apache.zookeeper:zookeeper:jar:3.4.5:test
[INFO] | | \- jline:jline:jar:0.9.94:test
[INFO] | +- org.eclipse.jetty:jetty-plus:jar:8.1.14.v20131031:test
[INFO] | | +- org.eclipse.jetty.orbit:javax.transaction:jar:1.1.1.v201105210645:test
[INFO] | | +- org.eclipse.jetty:jetty-webapp:jar:8.1.14.v20131031:test
[INFO] | | | +- org.eclipse.jetty:jetty-xml:jar:8.1.14.v20131031:test
[INFO] | | | \- org.eclipse.jetty:jetty-servlet:jar:8.1.14.v20131031:test
[INFO] | | \- org.eclipse.jetty:jetty-jndi:jar:8.1.14.v20131031:test
[INFO] | | \- org.eclipse.jetty.orbit:javax.mail.glassfish:jar:1.4.1.v201005082020:test
[INFO] | | \- org.eclipse.jetty.orbit:javax.activation:jar:1.1.0.v201105071233:test
[INFO] | +- org.eclipse.jetty:jetty-security:jar:8.1.14.v20131031:test
[INFO] | +- org.eclipse.jetty:jetty-util:jar:8.1.14.v20131031:test
[INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:test
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:test
[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.5:test
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.5:test
[INFO] | +- com.ning:compress-lzf:jar:1.0.0:test
[INFO] | +- net.jpountz.lz4:lz4:jar:1.2.0:test
[INFO] | +- com.twitter:chill_2.10:jar:0.3.6:test
[INFO] | | \- com.esotericsoftware.kryo:kryo:jar:2.21:test
[INFO] | | +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:test
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:test
[INFO] | | \- org.objenesis:objenesis:jar:1.2:test
[INFO] | +- com.twitter:chill-java:jar:0.3.6:test
[INFO] | +- commons-net:commons-net:jar:2.2:test
[INFO] | +- org.spark-project.akka:akka-remote_2.10:jar:2.2.3-shaded-protobuf:test
[INFO] | | +- org.spark-project.akka:akka-actor_2.10:jar:2.2.3-shaded-protobuf:test
[INFO] | | | \- com.typesafe:config:jar:1.0.2:test
[INFO] | | +- io.netty:netty:jar:3.6.6.Final:test
[INFO] | | +- org.spark-project.protobuf:protobuf-java:jar:2.4.1-shaded:test
[INFO] | | \- org.uncommons.maths:uncommons-maths:jar:1.2.2a:test
[INFO] | +- org.spark-project.akka:akka-slf4j_2.10:jar:2.2.3-shaded-protobuf:test
[INFO] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:test
[INFO] | | +- org.json4s:json4s-core_2.10:jar:3.2.10:test
[INFO] | | | +- org.json4s:json4s-ast_2.10:jar:3.2.10:test
[INFO] | | | \- org.scala-lang:scalap:jar:2.10.4:test
[INFO] | | | \- org.scala-lang:scala-compiler:jar:2.10.4:test
[INFO] | | \- com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:test
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.3.0:test
[INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.3.1:test
[INFO] | +- colt:colt:jar:1.2.0:test
[INFO] | | \- concurrent:concurrent:jar:1.3.4:test
[INFO] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.18.1:test
[INFO] | +- io.netty:netty-all:jar:4.0.23.Final:test
[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:test
[INFO] | +- com.codahale.metrics:metrics-core:jar:3.0.0:test
[INFO] | +- com.codahale.metrics:metrics-jvm:jar:3.0.0:test
[INFO] | +- com.codahale.metrics:metrics-json:jar:3.0.0:test
[INFO] | +- com.codahale.metrics:metrics-graphite:jar:3.0.0:test
[INFO] | +- org.tachyonproject:tachyon-client:jar:0.5.0:test
[INFO] | | \- org.tachyonproject:tachyon:jar:0.5.0:test
[INFO] | +- org.spark-project:pyrolite:jar:2.0.1:test
[INFO] | \- net.sf.py4j:py4j:jar:0.8.2.1:test
[INFO] \- org.eclipse.jetty:jetty-server:jar:8.1.14.v20131031:test
[INFO] +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
[INFO] +- org.eclipse.jetty:jetty-continuation:jar:8.1.14.v20131031:test
[INFO] \- org.eclipse.jetty:jetty-http:jar:8.1.14.v20131031:test
[INFO] \- org.eclipse.jetty:jetty-io:jar:8.1.14.v20131031:test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51412557
@pwendell I think this is a good set of changes. Can you take a quick look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-52944078
@witgo Yes I see the same issue now after a rebase. I added your extra dependency, rebased, and it works. Let's see what Jenkins says.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51311768
QA results for PR 1726:<br>- This patch FAILED unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18015/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53218467
LGTM. Also spoke to @pwendell offline that its cool to add flume version (that is version of libraries used in external projects) in root pom.xml. Test this once more and merging this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by witgo <gi...@git.apache.org>.
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16541517
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,9 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
- </dependency>
- <dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
+ <scope>test</scope>
--- End diff --
Add the dependency. So `make-distribution.sh` can work.
```xml
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_${scala.binary.version}</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-50958177
QA tests have started for PR 1726. This patch merges cleanly. <br>View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17766/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51307848
QA tests have started for PR 1726. This patch merges cleanly. <br>View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18015/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-52944285
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19059/consoleFull) for PR 1726 at commit [`138523b`](https://github.com/apache/spark/commit/138523be794b0e0c75e0aca344fc79e9a97c5681).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626690
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Look at line 30 onwards in the gist. It lists core and a whole lot of others as deps, not streaming main code as dependency for the streaming-test code. That is what I am talking about.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626631
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Isn't that the problem? You can see the dependency on streaming test code,
but not streaming non-test code. But non-test streaming code is required at
runtime. Who are we to argue with the compiler?
On Sat, Aug 23, 2014 at 12:47 AM, Hari Shreedharan <notifications@github.com
> wrote:
> In external/flume-sink/pom.xml:
>
> > @@ -65,12 +66,15 @@
> > </exclusions>
> > </dependency>
> > <dependency>
> > - <groupId>org.scala-lang</groupId>
> > - <artifactId>scala-library</artifactId>
>
> The sink does not depend on streaming either. The streaming jar should not
> even be dependency - even for test only the streaming test jar is required:
>
> [INFO] org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.1.0-SNAPSHOT
> [INFO] +- org.apache.flume:flume-ng-sdk:jar:1.4.0:compile
> [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile
> [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
> [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
> [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
> [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
> [INFO] | | - org.apache.commons:commons-compress:jar:1.4.1:compile
> [INFO] | | - org.tukaani:xz:jar:1.0:compile
> [INFO] | - org.apache.avro:avro-ipc:jar:1.7.6:compile
> [INFO] +- org.apache.flume:flume-ng-core:jar:1.4.0:compile
> [INFO] | +- org.apache.flume:flume-ng-configuration:jar:1.4.0:compile
> [INFO] | +- org.slf4j:slf4j-api:jar:1.7.5:compile
> [INFO] | +- com.google.guava:guava:jar:14.0.1:provided
> [INFO] | +- commons-io:commons-io:jar:2.1:compile
> [INFO] | +- commons-codec:commons-codec:jar:1.5:compile
> [INFO] | +- log4j:log4j:jar:1.2.17:compile
> [INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] | +- commons-lang:commons-lang:jar:2.5:compile
> [INFO] | +- joda-time:joda-time:jar:2.1:compile
> [INFO] | +- org.mortbay.jetty:servlet-api:jar:2.5-20110124:compile
> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile
> [INFO] | +- com.google.code.gson:gson:jar:2.2.2:compile
> [INFO] | - org.apache.mina:mina-core:jar:2.0.4:compile
> [INFO] +- org.scala-lang:scala-library:jar:2.10.4:compile
> [INFO] +- org.scalatest:scalatest_2.10:jar:2.1.5:test
> [INFO] | - org.scala-lang:scala-reflect:jar:2.10.4:test
> [INFO] -
> org.apache.spark:spark-streaming_2.10:test-jar:tests:1.1.0-SNAPSHOT:test
> [INFO] +- org.apache.spark:spark-core_2.10:jar:1.1.0-SNAPSHOT:test
> [INFO] | +- org.apache.hadoop:hadoop-client:jar:1.0.4:test
> [INFO] | | - org.apache.hadoop:hadoop-core:jar:1.0.4:test
> [INFO] | | +- xmlenc:xmlenc:jar:0.52:test
> [INFO] | | +- org.apache.commons:commons-math:jar:2.1:test
> [INFO] | | +- commons-configuration:commons-configuration:jar:1.6:test
> [INFO] | | | +- commons-collections:commons-collections:jar:3.2.1:test
> [INFO] | | | +- commons-digester:commons-digester:jar:1.8:test
> [INFO] | | | | - commons-beanutils:commons-beanutils:jar:1.7.0:test
> [INFO] | | | - commons-beanutils:commons-beanutils-core:jar:1.8.0:test
> [INFO] | | +- commons-el:commons-el:jar:1.0:test
> [INFO] | | +- hsqldb:hsqldb:jar:1.8.0.10:test
> [INFO] | | - oro:oro:jar:2.0.8:test
> [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:test
> [INFO] | | - commons-httpclient:commons-httpclient:jar:3.1:test
> [INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:test
> [INFO] | | +- org.apache.curator:curator-framework:jar:2.4.0:test
> [INFO] | | | - org.apache.curator:curator-client:jar:2.4.0:test
> [INFO] | | - org.apache.zookeeper:zookeeper:jar:3.4.5:test
> [INFO] | | - jline:jline:jar:0.9.94:test
> [INFO] | +- org.eclipse.jetty:jetty-plus:jar:8.1.14.v20131031:test
> [INFO] | | +-
> org.eclipse.jetty.orbit:javax.transaction:jar:1.1.1.v201105210645:test
> [INFO] | | +- org.eclipse.jetty:jetty-webapp:jar:8.1.14.v20131031:test
> [INFO] | | | +- org.eclipse.jetty:jetty-xml:jar:8.1.14.v20131031:test
> [INFO] | | | - org.eclipse.jetty:jetty-servlet:jar:8.1.14.v20131031:test
> [INFO] | | - org.eclipse.jetty:jetty-jndi:jar:8.1.14.v20131031:test
> [INFO] | | -
> org.eclipse.jetty.orbit:javax.mail.glassfish:jar:1.4.1.v201005082020:test
> [INFO] | | -
> org.eclipse.jetty.orbit:javax.activation:jar:1.1.0.v201105071233:test
> [INFO] | +- org.eclipse.jetty:jetty-security:jar:8.1.14.v20131031:test
> [INFO] | +- org.eclipse.jetty:jetty-util:jar:8.1.14.v20131031:test
> [INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:test
> [INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:test
> [INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.5:test
> [INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.5:test
> [INFO] | +- com.ning:compress-lzf:jar:1.0.0:test
> [INFO] | +- net.jpountz.lz4:lz4:jar:1.2.0:test
> [INFO] | +- com.twitter:chill_2.10:jar:0.3.6:test
> [INFO] | | - com.esotericsoftware.kryo:kryo:jar:2.21:test
> [INFO] | | +-
> com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:test
> [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:test
> [INFO] | | - org.objenesis:objenesis:jar:1.2:test
> [INFO] | +- com.twitter:chill-java:jar:0.3.6:test
> [INFO] | +- commons-net:commons-net:jar:2.2:test
> [INFO] | +-
> org.spark-project.akka:akka-remote_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | | +-
> org.spark-project.akka:akka-actor_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | | | - com.typesafe:config:jar:1.0.2:test
> [INFO] | | +- io.netty:netty:jar:3.6.6.Final:test
> [INFO] | | +-
> org.spark-project.protobuf:protobuf-java:jar:2.4.1-shaded:test
> [INFO] | | - org.uncommons.maths:uncommons-maths:jar:1.2.2a:test
> [INFO] | +-
> org.spark-project.akka:akka-slf4j_2.10:jar:2.2.3-shaded-protobuf:test
> [INFO] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:test
> [INFO] | | +- org.json4s:json4s-core_2.10:jar:3.2.10:test
> [INFO] | | | +- org.json4s:json4s-ast_2.10:jar:3.2.10:test
> [INFO] | | | - org.scala-lang:scalap:jar:2.10.4:test
> [INFO] | | | - org.scala-lang:scala-compiler:jar:2.10.4:test
> [INFO] | | - com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:test
> [INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.3.0:test
> [INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.3.1:test
> [INFO] | +- colt:colt:jar:1.2.0:test
> [INFO] | | - concurrent:concurrent:jar:1.3.4:test
> [INFO] | +- org.apache.mesos:mesos:jar:shaded-protobuf:0.18.1:test
> [INFO] | +- io.netty:netty-all:jar:4.0.23.Final:test
> [INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:test
> [INFO] | +- com.codahale.metrics:metrics-core:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-jvm:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-json:jar:3.0.0:test
> [INFO] | +- com.codahale.metrics:metrics-graphite:jar:3.0.0:test
> [INFO] | +- org.tachyonproject:tachyon-client:jar:0.5.0:test
> [INFO] | | - org.tachyonproject:tachyon:jar:0.5.0:test
> [INFO] | +- org.spark-project:pyrolite:jar:2.0.1:test
> [INFO] | - net.sf.py4j:py4j:jar:0.8.2.1:test
> [INFO] - org.eclipse.jetty:jetty-server:jar:8.1.14.v20131031:test
> [INFO] +-
> org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
> [INFO] +- org.eclipse.jetty:jetty-continuation:jar:8.1.14.v20131031:test
> [INFO] - org.eclipse.jetty:jetty-http:jar:8.1.14.v20131031:test
> [INFO] - org.eclipse.jetty:jetty-io:jar:8.1.14.v20131031:test
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1726/files#r16626588>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53134963
Actually, the Spark Sink test code only depends on the test jar. The test jar seems to depend on the main jar. I am not sure why the sink's pom must also pull in the main jar even though it does not depend on it directly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-50959118
QA results for PR 1726:<br>- This patch PASSES unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17766/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16625685
--- Diff: pom.xml ---
@@ -125,6 +125,7 @@
<protobuf.version>2.4.1</protobuf.version>
<yarn.version>${hadoop.version}</yarn.version>
<hbase.version>0.94.6</hbase.version>
+ <flume.version>1.4.0</flume.version>
--- End diff --
We can bump this to 1.5.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-51314361
QA tests have started for PR 1726. This patch merges cleanly. <br>View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18017/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626447
--- Diff: pom.xml ---
@@ -125,6 +125,7 @@
<protobuf.version>2.4.1</protobuf.version>
<yarn.version>${hadoop.version}</yarn.version>
<hbase.version>0.94.6</hbase.version>
+ <flume.version>1.4.0</flume.version>
--- End diff --
Maybe so; I wanted to focus this on harmonizing versions only rather than upgrading
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16626471
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Through spark-streaming then? take a look at `mvn dependency:tree -pl external/flume-sink/`; it is included. It is not a big deal now that the dependency here is not on a fixed version, but inherits from the parent. But it seems that most modules do not declare this themselves.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53149251
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19105/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch **passes** unit tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53135372
Yes, the test code depends on test code from streaming - not on non-test code (directly - the test code may depend on it - isn't it maven's job to figure out transitive deps?). If test dependencies do not bring in transitive dependencies that is a bug in maven no? How is a user supposed to list all the dependencies of his test dependencies in his own pom?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16639309
--- Diff: pom.xml ---
@@ -125,6 +125,7 @@
<protobuf.version>2.4.1</protobuf.version>
<yarn.version>${hadoop.version}</yarn.version>
<hbase.version>0.94.6</hbase.version>
+ <flume.version>1.4.0</flume.version>
--- End diff --
Yes, lets not bump this up so late in the release process. We dont know what crazy dependency behavior can change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53137854
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16625779
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
The spark sink does not depend on core, so core won't pull this in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53237810
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53242451
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19136/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch **passes** unit tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16625739
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
It's already brought in by core. I don't think this is harmful, but just a bit nonstandard. Except for streaming, the other modules don't seem to do this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53218619
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19130/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53140011
@pwendell @harishreedharan @srowen There is a better fix for fixing the maven build alone. There is no reason to depend on spark-streaming tests. So I eliminated that dependency completely and submitted a PR
https://github.com/apache/spark/pull/2101
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1726
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1726#discussion_r16625663
--- Diff: external/flume-sink/pom.xml ---
@@ -65,12 +66,15 @@
</exclusions>
</dependency>
<dependency>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
--- End diff --
Why is this being removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53135223
How do you explain the compile failure then? The test code does plainly
depend on the non-test streaming module. It depends on `TestSuiteBase` from
streaming which depends on non-test code. I think the kicker here is that
Maven test dependencies do not bring in transitive dependencies, while
non-test dependencies do. Hence I think this is a legit failure that needs
to be fixed.
On Sat, Aug 23, 2014 at 12:43 AM, Hari Shreedharan <notifications@github.com
> wrote:
> Actually, the Spark Sink test code only depends on the test jar. The test
> jar seems to depend on the main jar. I am not sure why the sink's pom must
> also pull in the main jar even though it does not depend on it directly.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1726#issuecomment-53134963>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53148099
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19105/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Jenkins build failing due t...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-50958139
Cool, https://github.com/apache/spark/commit/25cad6adf6479fb00265df06d5f77599f8defd26 fixes the core flume-sink test issue. Still worth a thought about these other touch-ups here, especially the avro version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by witgo <gi...@git.apache.org>.
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-52922473
I tested, but compile failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by harishreedharan <gi...@git.apache.org>.
Github user harishreedharan commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53136441
Looks like there is no real way to have the test jar depend on the non-test jar. The only way to do this is to do the "preferred way" in this page: http://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html that requires a ton of work
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53220264
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19130/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-52951761
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19059/consoleFull) for PR 1726 at commit [`138523b`](https://github.com/apache/spark/commit/138523be794b0e0c75e0aca344fc79e9a97c5681).
* This patch **passes** unit tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53218469
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: SPARK-2798 [BUILD] Correct several small error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1726#issuecomment-53238173
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19136/consoleFull) for PR 1726 at commit [`a46e2c6`](https://github.com/apache/spark/commit/a46e2c6abbb56553abab6d3e4da690c7d9e47772).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org