You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gvramana <gi...@git.apache.org> on 2015/04/07 17:36:15 UTC
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
GitHub user gvramana opened a pull request:
https://github.com/apache/spark/pull/5393
[SPARK-5818][SQL] unable to use "add jar" in hql
TranslatingClassLoader is used for Spark-shell, while current hive's add jar can work only with URLClassLoader.
So jar has to be directly added to spark driver's class loader or its parent loader, in case of spark-shell
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gvramana/spark addjar_fromshell
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5393.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5393
----
commit 1f73a6c67c5a6de89da2b3625ea559a83d6a6a76
Author: Venkata Ramana Gollamudi <ra...@huawei.com>
Date: 2015-04-07T14:37:22Z
fixed adding jar to shell class loader
commit 275ca086038acafb9ef8fc5582c863105ad19358
Author: Venkata Ramana Gollamudi <ra...@huawei.com>
Date: 2015-04-07T15:28:36Z
Added tests
These tests will run only when compiled with -Phive
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5393#issuecomment-90612516
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the pull request:
https://github.com/apache/spark/pull/5393#issuecomment-92603181
Hi, Can some one verify this patch. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/spark/pull/5393#discussion_r28392460
--- Diff: repl/pom.xml ---
@@ -150,6 +150,16 @@
</build>
<profiles>
<profile>
+ <id>hive</id>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.spark</groupId>
+ <artifactId>spark-hive_${scala.binary.version}</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ </profile>
+ <profile>
--- End diff --
No, dependency is added in hive profile only. So if assembly is built with -Phive option then hive dependency is added to repl, so that hive is available in class path for repl.
If assembly is built without -Phive option then dependency is not added and the testcase is ignored. Testcase also checks runtime if hiveContext class is available, if not testcase is ignored.
I have manually tested both the cases of building with -Phive and without hive. It will not impact assembly creation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5393
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/5393#discussion_r28363938
--- Diff: repl/pom.xml ---
@@ -150,6 +150,16 @@
</build>
<profiles>
<profile>
+ <id>hive</id>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.spark</groupId>
+ <artifactId>spark-hive_${scala.binary.version}</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ </profile>
+ <profile>
--- End diff --
We can't make this change as I think it means that we will always build hive with the spark assembly, correct?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-5818][SQL] unable to use "add jar" in h...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/5393#issuecomment-137569208
Thanks for working on this. Can we close this issue until you have time to bring it up to date?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org