You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by JoshRosen <gi...@git.apache.org> on 2015/12/01 04:06:22 UTC
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/10054
[SPARK-12065] Upgrade Tachyon from 0.8.1 to 0.8.2
This commit upgrades the Tachyon dependency from 0.8.1 to 0.8.2.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark upgrade-to-tachyon-0.8.2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10054.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10054
----
commit e5ecbe04c8e1096215f21c54f71e657f560dd374
Author: Josh Rosen <jo...@databricks.com>
Date: 2015-12-01T03:01:37Z
Upgrade Tachyon from 0.8.1 to 0.8.2.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160863931
**[Test build #46944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46944/consoleFull)** for PR 10054 at commit [`e5ecbe0`](https://github.com/apache/spark/commit/e5ecbe04c8e1096215f21c54f71e657f560dd374).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10054#discussion_r46326032
--- Diff: core/pom.xml ---
@@ -270,7 +270,7 @@
<dependency>
<groupId>org.tachyonproject</groupId>
<artifactId>tachyon-client</artifactId>
- <version>0.8.1</version>
+ <version>0.8.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
--- End diff --
Hmm, I guess we need this because of how Maven handles transitive compile-scope dependencies. According to https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope:
> Each of the scopes (except for import) affects transitive dependencies in different ways, as is demonstrated in the table below. If a dependency is set to the scope in the left column, transitive dependencies of that dependency with the scope across the top row will result in a dependency in the main project with the scope listed at the intersection. If no scope is listed, it means the dependency will be omitted.
<table border="1" class="table table-striped">
<tbody><tr class="a">
<td align="left"></td>
<td align="left">compile</td>
<td align="left">provided</td>
<td align="left">runtime</td>
<td align="left">test</td></tr>
<tr class="b">
<td align="left">compile</td>
<td align="left">compile(*)</td>
<td align="left">-</td>
<td align="left">runtime</td>
<td align="left">-</td></tr>
<tr class="a">
<td align="left">provided</td>
<td align="left">provided</td>
<td align="left">-</td>
<td align="left">provided</td>
<td align="left">-</td></tr>
<tr class="b">
<td align="left">runtime</td>
<td align="left">runtime</td>
<td align="left">-</td>
<td align="left">runtime</td>
<td align="left">-</td></tr>
<tr class="a">
<td align="left">test</td>
<td align="left">test</td>
<td align="left">-</td>
<td align="left">test</td>
<td align="left">-</td></tr></tbody></table>
My reading of this is that a compile-scope dependency on `tachyon-client` will result in Tachyon client's shaded dependencies being pulled in. Given this, I'm a bit confused about why the `tachyon-client` JAR is published with a non-dependency-reduced POM.
This is corroborated by #8577, which needed to add additional exclusions because of this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160861174
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160834864
/cc @calvinjia
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160886593
**[Test build #46944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46944/consoleFull)** for PR 10054 at commit [`e5ecbe0`](https://github.com/apache/spark/commit/e5ecbe04c8e1096215f21c54f71e657f560dd374).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10054
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160851686
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by calvinjia <gi...@git.apache.org>.
Github user calvinjia commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160836977
@JoshRosen
LGTM, thanks for updating this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10054#discussion_r46327951
--- Diff: core/pom.xml ---
@@ -270,7 +270,7 @@
<dependency>
<groupId>org.tachyonproject</groupId>
<artifactId>tachyon-client</artifactId>
- <version>0.8.1</version>
+ <version>0.8.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
--- End diff --
I've filed https://tachyon.atlassian.net/browse/TACHYON-1380 to ask Tachyon to change the handling of dependnecies in this POM for future releases. For now, though, I've verified that we don't need to change any of these exclusions because the client's dependencies haven't changed, so I think this PR should be good to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160851689
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46933/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160851627
**[Test build #46933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46933/consoleFull)** for PR 10054 at commit [`e5ecbe0`](https://github.com/apache/spark/commit/e5ecbe04c8e1096215f21c54f71e657f560dd374).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10054#discussion_r46457541
--- Diff: core/pom.xml ---
@@ -270,7 +270,7 @@
<dependency>
<groupId>org.tachyonproject</groupId>
<artifactId>tachyon-client</artifactId>
- <version>0.8.1</version>
+ <version>0.8.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
--- End diff --
Ah, there was some slight confusion here: I somehow parsed "Distribution of Tachyon client module with shaded packages in jar" as "shades all dependencies", which isn't true, so there's not an issue here. My bad.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160836096
**[Test build #46933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46933/consoleFull)** for PR 10054 at commit [`e5ecbe0`](https://github.com/apache/spark/commit/e5ecbe04c8e1096215f21c54f71e657f560dd374).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/10054#discussion_r46324376
--- Diff: core/pom.xml ---
@@ -270,7 +270,7 @@
<dependency>
<groupId>org.tachyonproject</groupId>
<artifactId>tachyon-client</artifactId>
- <version>0.8.1</version>
+ <version>0.8.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
--- End diff --
Hey, quick question: why do we need these exclusions? AFAIK `tachyon-client` is an uber-JAR which shades all of its dependencies: http://mvnrepository.com/artifact/org.tachyonproject/tachyon-client/0.8.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160886669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46944/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-12065] Upgrade Tachyon from 0.8.1 to 0....
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10054#issuecomment-160886665
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org