You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by atokhy <gi...@git.apache.org> on 2016/04/30 04:59:25 UTC
[GitHub] spark pull request: [SPARK-15024] NoClassDefFoundError in spark-ex...
GitHub user atokhy opened a pull request:
https://github.com/apache/spark/pull/12800
[SPARK-15024] NoClassDefFoundError in spark-examples due to Guava rel…
## What changes were proposed in this pull request?
This change will only shade and relocate Guava in the spark-examples JAR used by the Apache Bigtop distribution (as of writing the Bigtop distribution uses the unshaded JAR). Since only Guava and Jetty are relocated, and since there are 0 references to Jetty in the spark-examples codebase, only Guava needs to be shaded. The shaded spark-assembly JAR provides the rest of the dependencies needed to get the spark-examples to run without any NoClassDefFoundErrors
## How was this patch tested?
mvn package -DrecompileMode=all -Pbigtop-dist -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver -Dprotobuf.version=2.5.0
$ unzip -l target/scala-2.10/spark-examples-1.6.1-hadoop2.6.0.jar | grep org.spark-project.guava | wc -l
1602
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/atokhy/spark spark-examples-guava
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12800.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12800
----
commit 4130e0614b9237e51e774b22a971fd7aa06b088d
Author: Aaron Tokhy <to...@amazon.com>
Date: 2016-04-30T02:50:26Z
[SPARK-15024] NoClassDefFoundError in spark-examples due to Guava relocation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15024] NoClassDefFoundError in spark-ex...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/12800#issuecomment-215948210
This is opened vs 1.6 when it should be master.
@vanzin were we zapping the examples JAR anyway?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #12800: [SPARK-15024] NoClassDefFoundError in spark-examp...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12800
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12800: [SPARK-15024] NoClassDefFoundError in spark-examples due...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/12800
@atokhy that's a little weird. The examples module should be inheriting the guava shading from the parent pom. I have a 1.6 build locally (with patches, so I'll check the official package tomorrow) and HiveFromSpark does not reference unshaded guava classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15024] NoClassDefFoundError in spark-ex...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/12800#issuecomment-215932736
(It looks the last part of the title is truncated)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #12800: [SPARK-15024] NoClassDefFoundError in spark-examples due...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/12800
Hi @atokhy, it seems inactive without an answer to the comment above. Should this be better closed if you are not able to proceed this further?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15024] NoClassDefFoundError in spark-ex...
Posted by atokhy <gi...@git.apache.org>.
Github user atokhy commented on the pull request:
https://github.com/apache/spark/pull/12800#issuecomment-216426119
This is a problem only found in the 1.6 branch (potentially in Spark 1.6.2), and it does not apply to Apache master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15024] NoClassDefFoundError in spark-ex...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12800#issuecomment-215927866
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org