You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/29 01:17:27 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
HyukjinKwon opened a new pull request #34134:
URL: https://github.com/apache/spark/pull/34134
### What changes were proposed in this pull request?
Self-contained.
### Why are the changes needed?
For user's more information on available Python dependency management in PySpark.
### Does this PR introduce _any_ user-facing change?
Yes, documentation change.
### How was this patch tested?
Manaully built the docs and checked the results:
<img width="918" alt="Screen Shot 2021-09-29 at 10 11 56 AM" src="https://user-images.githubusercontent.com/6477701/135186536-2f271378-d06b-4c6b-a4be-691ce395db9f.png">
<img width="976" alt="Screen Shot 2021-09-29 at 10 12 22 AM" src="https://user-images.githubusercontent.com/6477701/135186541-0f4c5615-bc49-48e2-affd-dc2f5c0334bf.png">
<img width="920" alt="Screen Shot 2021-09-29 at 10 12 42 AM" src="https://user-images.githubusercontent.com/6477701/135186551-0b613096-7c86-4562-b345-ddd60208367b.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929831577
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48213/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929744931
**[Test build #143691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143691/testReport)** for PR 34134 at commit [`791fa7e`](https://github.com/apache/spark/commit/791fa7e1f376f9142821a870a3b55aeef892f531).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929786375
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48205/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mkgray commented on a change in pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
mkgray commented on a change in pull request #34134:
URL: https://github.com/apache/spark/pull/34134#discussion_r718095616
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
Minor typo, should be third-party
##########
File path: docs/submitting-applications.md
##########
@@ -35,7 +35,8 @@ script as shown here while passing your jar.
For Python, you can use the `--py-files` argument of `spark-submit` to add `.py`, `.zip` or `.egg`
files to be distributed with your application. If you depend on multiple Python files we recommend
-packaging them into a `.zip` or `.egg`.
+packaging them into a `.zip` or `.egg`. For thrid-party Python dependencies,
Review comment:
Minor typo, should be third-party
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34134:
URL: https://github.com/apache/spark/pull/34134#discussion_r718109786
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
oops, thanks man
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
```suggestion
to the runtime path by passing a comma-separated list to `--py-files`. For third-party Python dependencies,
```
##########
File path: docs/submitting-applications.md
##########
@@ -35,7 +35,8 @@ script as shown here while passing your jar.
For Python, you can use the `--py-files` argument of `spark-submit` to add `.py`, `.zip` or `.egg`
files to be distributed with your application. If you depend on multiple Python files we recommend
-packaging them into a `.zip` or `.egg`.
+packaging them into a `.zip` or `.egg`. For thrid-party Python dependencies,
Review comment:
```suggestion
packaging them into a `.zip` or `.egg`. For third-party Python dependencies,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929779122
**[Test build #143697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143697/testReport)** for PR 34134 at commit [`6e31a31`](https://github.com/apache/spark/commit/6e31a3150a021b42a935bfc401682304a37a8f47).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929854421
Merged to master and branch-3.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929744931
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929760754
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48205/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929793414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929779122
**[Test build #143697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143697/testReport)** for PR 34134 at commit [`6e31a31`](https://github.com/apache/spark/commit/6e31a3150a021b42a935bfc401682304a37a8f47).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929855305
+1, late LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929749402
**[Test build #143691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143691/testReport)** for PR 34134 at commit [`791fa7e`](https://github.com/apache/spark/commit/791fa7e1f376f9142821a870a3b55aeef892f531).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929831617
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48213/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929784576
**[Test build #143697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143697/testReport)** for PR 34134 at commit [`6e31a31`](https://github.com/apache/spark/commit/6e31a3150a021b42a935bfc401682304a37a8f47).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929831617
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48213/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929749548
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143691/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929749548
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143691/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929749548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929854333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #34134:
URL: https://github.com/apache/spark/pull/34134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34134:
URL: https://github.com/apache/spark/pull/34134#discussion_r718110005
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
```suggestion
to the runtime path by passing a comma-separated list to `--py-files`. For third-party Python dependencies,
```
##########
File path: docs/submitting-applications.md
##########
@@ -35,7 +35,8 @@ script as shown here while passing your jar.
For Python, you can use the `--py-files` argument of `spark-submit` to add `.py`, `.zip` or `.egg`
files to be distributed with your application. If you depend on multiple Python files we recommend
-packaging them into a `.zip` or `.egg`.
+packaging them into a `.zip` or `.egg`. For thrid-party Python dependencies,
Review comment:
```suggestion
packaging them into a `.zip` or `.egg`. For third-party Python dependencies,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mkgray commented on a change in pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
mkgray commented on a change in pull request #34134:
URL: https://github.com/apache/spark/pull/34134#discussion_r718095616
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
Minor typo, should be third-party
##########
File path: docs/submitting-applications.md
##########
@@ -35,7 +35,8 @@ script as shown here while passing your jar.
For Python, you can use the `--py-files` argument of `spark-submit` to add `.py`, `.zip` or `.egg`
files to be distributed with your application. If you depend on multiple Python files we recommend
-packaging them into a `.zip` or `.egg`.
+packaging them into a `.zip` or `.egg`. For thrid-party Python dependencies,
Review comment:
Minor typo, should be third-party
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929793414
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929854333
Thanks guys.
Merged to master. Let me merge this in branch-3.2 too - very trivial doc change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #34134:
URL: https://github.com/apache/spark/pull/34134#discussion_r718109786
##########
File path: docs/rdd-programming-guide.md
##########
@@ -241,12 +241,12 @@ For a complete list of options, run `spark-shell --help`. Behind the scenes,
In the PySpark shell, a special interpreter-aware SparkContext is already created for you, in the
variable called `sc`. Making your own SparkContext will not work. You can set which master the
context connects to using the `--master` argument, and you can add Python .zip, .egg or .py files
-to the runtime path by passing a comma-separated list to `--py-files`. You can also add dependencies
+to the runtime path by passing a comma-separated list to `--py-files`. For thrid-party Python dependencies,
Review comment:
oops, thanks man
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929813025
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48213/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #34134:
URL: https://github.com/apache/spark/pull/34134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929744931
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929749548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929744931
**[Test build #143691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143691/testReport)** for PR 34134 at commit [`791fa7e`](https://github.com/apache/spark/commit/791fa7e1f376f9142821a870a3b55aeef892f531).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34134: [MINOR][DOCS] Mention other Python dependency tools in documentation
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34134:
URL: https://github.com/apache/spark/pull/34134#issuecomment-929855305
+1, late LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org