You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by ScrapCodes <gi...@git.apache.org> on 2018/09/27 06:59:41 UTC
[GitHub] spark pull request #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid ru...
GitHub user ScrapCodes opened a pull request:
https://github.com/apache/spark/pull/22564
[SPARK-25282][K8s][DOC] Improved docs to avoid running into InvalidClassException.
## What changes were proposed in this pull request?
Documentation changes, on client mode. A user needs to make a distribution before he can use client mode.
## How was this patch tested?
Doc improvement comments are based on running locally. The patch itself is a document patch, so may not need testing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ScrapCodes/spark SPARK-25282/docs-k8s-client-mode
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22564.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22564
----
commit 0a4cd605b5b872148fb4649f9b3e167125f4f653
Author: Prashant Sharma <pr...@...>
Date: 2018-09-27T06:52:25Z
[SPARK-25282][K8s][DOC] Improved docs to avoid running into InvalidClassException.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/spark/pull/22564
Looks like this is working without making a release. It is not clear what change could have fixed the problem. Closing the PR for now.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96672/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
**[Test build #96672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96672/testReport)** for PR 22564 at commit [`4ba798a`](https://github.com/apache/spark/commit/4ba798a09b691b59067c8c63c36501b1c131ba17).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3523/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
**[Test build #96674 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96674/testReport)** for PR 22564 at commit [`a765a4d`](https://github.com/apache/spark/commit/a765a4d8e12451da98ae2fa660377bc8c17cf5d3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid ru...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes closed the pull request at:
https://github.com/apache/spark/pull/22564
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid ru...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22564#discussion_r223188473
--- Diff: docs/running-on-kubernetes.md ---
@@ -123,6 +123,20 @@ Starting with Spark 2.4.0, it is possible to run Spark applications on Kubernete
runs in client mode, the driver can run inside a pod or on a physical host. When running an application in client mode,
it is recommended to account for the following factors:
+### Make a distribution
+
+In the case you are working with latest code directly from the git code base of Apache Spark, by building from the source.
--- End diff --
This and the sentence below aren't quite grammatical. This also kind of duplicates the docs on how to build a release. Why not just say something like:
When testing against a custom build of Spark, from the latest source code or custom modifications, it's necessary to create a release and deploy images to a Docker repository. See the [Docker images] section, and [docs on releasing]. Example:
... example command
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96674/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3523/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3523/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/spark/pull/22564
I am sorry for the trouble, @liyinan926 and @srowen.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid ru...
Posted by liyinan926 <gi...@git.apache.org>.
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22564#discussion_r223074005
--- Diff: docs/running-on-kubernetes.md ---
@@ -123,6 +123,20 @@ Starting with Spark 2.4.0, it is possible to run Spark applications on Kubernete
runs in client mode, the driver can run inside a pod or on a physical host. When running an application in client mode,
it is recommended to account for the following factors:
+### Make a distribution
+
+In the case you are working with latest code directly from the git code base of Apache Spark, by building from the source.
--- End diff --
Nit: `the latest code` and `a docker repository` below. Also please use `Spark` with a capital `S` consistently.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
**[Test build #96672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96672/testReport)** for PR 22564 at commit [`4ba798a`](https://github.com/apache/spark/commit/4ba798a09b691b59067c8c63c36501b1c131ba17).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3521/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
**[Test build #96674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96674/testReport)** for PR 22564 at commit [`a765a4d`](https://github.com/apache/spark/commit/a765a4d8e12451da98ae2fa660377bc8c17cf5d3).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22564
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3521/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/spark/pull/22564
@liyinan926 Do you have some comments, if this will be helpful.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by liyinan926 <gi...@git.apache.org>.
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/22564
BTW: just realized that the image build script at `bin/docker-image-tool.sh` works both in a source directory and in an unzipped distribution. So you shouldn't need to make a distribution to build an image.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22564
What does this have to do with InvalidClassException by the way?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22564: [SPARK-25282][K8s][DOC] Improved docs to avoid running i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22564
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3521/
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org