You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by Sephiroth-Lin <gi...@git.apache.org> on 2015/04/12 10:05:34 UTC
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
GitHub user Sephiroth-Lin opened a pull request:
https://github.com/apache/spark/pull/5478
[SPARK-6869][PySpark] Pass PYTHONPATH to executor, so that executor can read pyspark file from local file system on executor node
From SPARK-1920 and SPARK-1520 we know PySpark on Yarn can not work when the assembly jar are package by JDK 1.7+, so pass the PYTHONPATH(set in spark-env.sh) to executor so that executor python process can read pyspark file from local file system rather than from assembly jar.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Sephiroth-Lin/spark SPARK-6869
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5478.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5478
----
commit 413fa25dde845146153a58793ca6b3ec3a820ea8
Author: linweizhong <li...@huawei.com>
Date: 2015-04-12T08:02:43Z
Pass PYTHONPATH to executor
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93719432
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30415/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by WangTaoTheTonic <gi...@git.apache.org>.
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93726246
Yeah, `spark.executorEnv.PYTHONPATH` is PySpark's `spark.executor.extraClassPath` in some way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93750739
[Test build #30417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30417/consoleFull) for PR 5478 at commit [`3a0ec77`](https://github.com/apache/spark/commit/3a0ec77bc092dfe3dd9c467f1979d946c7f8cca3).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-92023463
[Test build #30111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30111/consoleFull) for PR 5478 at commit [`413fa25`](https://github.com/apache/spark/commit/413fa25dde845146153a58793ca6b3ec3a820ea8).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93969225
[Test build #30478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30478/consoleFull) for PR 5478 at commit [`c63f31f`](https://github.com/apache/spark/commit/c63f31fc008801649450d6dffc7f2ba7cc77645d).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by lianhuiwang <gi...@git.apache.org>.
Github user lianhuiwang commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-94111781
@Sephiroth-Lin i think later i will submit my PR based on this PR and then please help me review it. thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93902319
[Test build #30462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30462/consoleFull) for PR 5478 at commit [`547fd95`](https://github.com/apache/spark/commit/547fd957ba224c86cf828890562b2eafde2b8ecb).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by sryza <gi...@git.apache.org>.
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93061403
IIUC, the motivation for this change is that the assembly jar distribution mechanism doesn't work for some Java versions.
I agree with Andrew that, if at all possible, we should avoid deployment models that expect PySpark or anything to be on every node. Even if we advise against it, it increases the number of places one needs to check when debugging why something does or does not appear on the executor PYTHONPATH.
Are there workarounds for the Java versions issue that don't require python to be installed on the NodeManagers?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93706094
[Test build #30415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30415/consoleFull) for PR 5478 at commit [`052e288`](https://github.com/apache/spark/commit/052e288877b195341d3898f2b2aa6b23c2d5b680).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by WangTaoTheTonic <gi...@git.apache.org>.
Github user WangTaoTheTonic commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93718724
Hey guys, after discussion with @Sephiroth-Lin offline, we have some questions:
* If we try to ship the zip file to executer backend with --py-files:
* User must use --py-file to point to the xxx.zip file we provide;
* We must change format of directories(use xxx.zip to replace python dir) in our release.
* Even the zip file is not very big, shipping it everytime still cost time.
* Inspired by the former solution, I thought we might add some config like `spark.exectuor.extraClassPath` in PySpark side, which points to the files needed by user's application so that user could use them without shipping.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95099367
[Test build #30747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30747/consoleFull) for PR 5478 at commit [`d012cde`](https://github.com/apache/spark/commit/d012cdebf154bb5458c060d50d751f97c5bf3384).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-92023483
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30111/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93750759
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30417/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93492829
That sounds good. We can ship it through `--py-files`. @Sephiroth-Lin Would you mind updating the PR and the JIRA to reflect this change in intention?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by lianhuiwang <gi...@git.apache.org>.
Github user lianhuiwang commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93873704
@sryza we can export PYSPARK_ARCHIVES_PATH=local://xx/pyspark.zip;local://xx/py4j.zip in spark-env.sh and we also can export PYSPARK_ARCHIVES_PATH=hdfs://xx/pyspaark.zip. then in SparkSubmit we can automatically add PYSPARK_ARCHIVES_PATH to yarn's dist files. and then Spark-on-Yarn can put dist files to YARN distributed cache. these work is same as Spark assembly jar.
@andrewor14 in my test, the pyspark.zip is 378KB, i think it is very small than the assembly jar.so it can put it to dist files automatically.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93729802
[Test build #30417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30417/consoleFull) for PR 5478 at commit [`3a0ec77`](https://github.com/apache/spark/commit/3a0ec77bc092dfe3dd9c467f1979d946c7f8cca3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93057067
@Sephiroth-Lin the point of running PySpark on YARN is that the user does not have to install Spark on the slave machines. Instead, we package the python files in the assembly jar, which is automatically shipped by YARN to all containers.
This change assumes that the python files will already be present on the slave machines, since `PYTHONPATH` reads from the local file system. I don't believe this is a deployment requirement that we want to enforce, especially since the user must now ensure all Spark python files are consistent across all the machines (as they must do so in standalone mode).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by sryza <gi...@git.apache.org>.
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93490885
That alternative solution makes sense to me. If it's not going to be added to the classpath, it might make more sense to use a zip than a jar.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-96867560
@tgravescs yes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93779804
> if we install Spark on each node, then we can set spark.executorEnv.PYTHONPATH
That's a workaround, but not really in the spirit of how Spark-on-YARN is expected to work. For example, you'd have to have all nodes have `$SPARK_HOME` be the same for that to work.
If this must be distributed as a separate file, then Sandy's solution is the way to go. The pyspark zip needs to be treated the same way the spark assembly is: if no configuration, find it locally and upload it to nodes using the distributed cache; add a config option so that users can store that file on HDFS or even use `local:` URIs in case they want to manually distribute the file.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93670839
[Test build #30407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30407/consoleFull) for PR 5478 at commit [`51ebb26`](https://github.com/apache/spark/commit/51ebb262b0a0ff30603b3ce650c474870a152582).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93790785
Also, how big is the actual zip? I would imagine that it's at least one or two orders of magnitude smaller than the assembly jar, so it shouldn't be expensive especially if we cache it. As others have pointed out, the whole point of Spark on YARN is that the user doesn't need to install it on every node, and doing this through `spark.executorEnv.PYTHONPATH` defeats this purpose.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by lianhuiwang <gi...@git.apache.org>.
Github user lianhuiwang commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93762922
yes, i think in SparkSubmit we can automatically add PYSPARK_ARCHIVES_PATH to dist files. and then in Client and ExecutorRunnable can set PYTHONPATH according to PYSPARK_ARCHIVES_PATH if it exist. and if user set PYTHONPATH, PYSPARK_ARCHIVES_PATH is unused. i have run successfully on yarn client and cluster.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5478
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95150787
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30748/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93721949
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30414/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93930774
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30462/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-94030618
@Sephiroth-Lin I looked at the latest changes and it doesn't seem to reflect what I had in mind. My proposal was to automatically zip up the python files in `SparkSubmit` and add it to the `--py-files`. If you do that then I believe all the `PYTHONPATH` will just be automatically set up as well. You can take a look at Java's `ZipEntry` or [this post](http://stackoverflow.com/questions/4773778/creating-zip-archive-in-java) as a reference on how to do this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93724717
@andrewor14 @sryza @WangTaoTheTonic As I have test again, if we install Spark on each node, then we can set spark.executorEnv.PYTHONPATH=${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.8.2.1-src.zip to pass PYTHONPATH to executor. So this PR is another solution to run PySpark on yan if we don't install Spark on each node.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95102969
@andrewor14 Sorry, these days I am busy, now I have update the code. ^-^
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95150780
[Test build #30748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30748/consoleFull) for PR 5478 at commit [`5d9bcb6`](https://github.com/apache/spark/commit/5d9bcb675d5bc0d619c90039e8a2852f80ae3758).
* This patch **passes all tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
* This patch **adds the following new dependencies:**
* `commons-math3-3.1.1.jar`
* `snappy-java-1.1.1.6.jar`
* This patch **removes the following dependencies:**
* `commons-math3-3.4.1.jar`
* `snappy-java-1.1.1.7.jar`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95148308
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30747/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93705732
[Test build #30414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30414/consoleFull) for PR 5478 at commit [`309679a`](https://github.com/apache/spark/commit/309679acec332c7315357a3430795f6b2880019e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93690925
[Test build #30407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30407/consoleFull) for PR 5478 at commit [`51ebb26`](https://github.com/apache/spark/commit/51ebb262b0a0ff30603b3ce650c474870a152582).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-94594191
@Sephiroth-Lin would you mind closing this PR then?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-94331295
@lianhuiwang OK.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-92006593
[Test build #30111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30111/consoleFull) for PR 5478 at commit [`413fa25`](https://github.com/apache/spark/commit/413fa25dde845146153a58793ca6b3ec3a820ea8).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93930705
[Test build #30462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30462/consoleFull) for PR 5478 at commit [`547fd95`](https://github.com/apache/spark/commit/547fd957ba224c86cf828890562b2eafde2b8ecb).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by sryza <gi...@git.apache.org>.
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93778753
Regarding the performance issue, this can be solved with the YARN distributed cache in the same way it works for the Spark assembly jar. If the file is placed on HDFS in a public location, it will be cached on the nodes as a public YARN local resource so it doesn't need to be downloaded each time an app is submitted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93954581
[Test build #30478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30478/consoleFull) for PR 5478 at commit [`c63f31f`](https://github.com/apache/spark/commit/c63f31fc008801649450d6dffc7f2ba7cc77645d).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93705830
@andrewor14 @sryza Done, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95148298
[Test build #30747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30747/consoleFull) for PR 5478 at commit [`d012cde`](https://github.com/apache/spark/commit/d012cdebf154bb5458c060d50d751f97c5bf3384).
* This patch **passes all tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
* This patch **adds the following new dependencies:**
* `commons-math3-3.1.1.jar`
* `snappy-java-1.1.1.6.jar`
* This patch **removes the following dependencies:**
* `commons-math3-3.4.1.jar`
* `snappy-java-1.1.1.7.jar`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-95103607
[Test build #30748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30748/consoleFull) for PR 5478 at commit [`5d9bcb6`](https://github.com/apache/spark/commit/5d9bcb675d5bc0d619c90039e8a2852f80ae3758).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Pass PYTHONPATH to execu...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93270239
@andrewor14 @sryza Yes, to assume that the python files will already be present on the slave machines is not very reasonable. But if user want to use PySpark, then they must compile the Spark in JDK1.6, but I think now most user are use JDK1.7+. Maybe a good solution is package the PySpark in another jar and automatically shipped by YARN to all containers. And add this jar to PYTHONPATH with asseambly jar.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-96145643
@andrewor14 @sryza how about your opinions? thanks. @lianhuiwang please help me review this, thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93719423
[Test build #30415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30415/consoleFull) for PR 5478 at commit [`052e288`](https://github.com/apache/spark/commit/052e288877b195341d3898f2b2aa6b23c2d5b680).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93969235
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30478/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-96678984
so is this competing directly with https://github.com/apache/spark/pull/5580?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93690948
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30407/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5478#issuecomment-93721930
[Test build #30414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30414/consoleFull) for PR 5478 at commit [`309679a`](https://github.com/apache/spark/commit/309679acec332c7315357a3430795f6b2880019e).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org