You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by xueyumusic <gi...@git.apache.org> on 2018/06/23 13:55:06 UTC
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
GitHub user xueyumusic opened a pull request:
https://github.com/apache/spark/pull/21624
[SPARK-24639][DOC] Add three config in the doc
## What changes were proposed in this pull request?
add three config which are mentioned in the pr #21567 , they are `spark.python.task.killTimeout`, `spark.worker.driverTerminateTimeout` and `spark.ui.consoleProgress.update.interval`
## How was this patch tested?
doc build
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xueyumusic/spark addconfig1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21624.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21624
----
commit 16c256d8206123df9487c57c6779ad2b0e0211d0
Author: xueyu <27...@...>
Date: 2018-06-23T13:49:37Z
add some configs
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21624
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/21624
cc @zsxwing
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21624
**[Test build #93064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93064/testReport)** for PR 21624 at commit [`6886111`](https://github.com/apache/spark/commit/6886111fa39fecbbc0461304c99abdfcc5caf33a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by xueyumusic <gi...@git.apache.org>.
Github user xueyumusic commented on a diff in the pull request:
https://github.com/apache/spark/pull/21624#discussion_r199145469
--- Diff: docs/configuration.md ---
@@ -456,6 +456,13 @@ Apart from these, the following properties are also available, and may be useful
from JVM to Python worker for every task.
</td>
</tr>
+<tr>
+ <td><code>spark.python.task.killTimeout</code></td>
+ <td>2s</td>
+ <td>
+ How long to wait before killing the python worker if a task cannot be interrupted.
--- End diff --
updated and fix confilct, please have a review, thanks, @zsxwing @jiangxb1987
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21624
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21624
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21624
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93064/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21624#discussion_r206698219
--- Diff: docs/configuration.md ---
@@ -456,6 +456,40 @@ Apart from these, the following properties are also available, and may be useful
from JVM to Python worker for every task.
</td>
</tr>
+<tr>
+ <td><code>spark.python.task.killTimeout</code></td>
+ <td>2s</td>
+ <td>
+ Timeout to wait before killing the python worker when a task cannot be interrupted
+ </td>
+</tr>
+<tr>
+ <td><code>spark.sql.repl.eagerEval.enabled</code></td>
+ <td>false</td>
+ <td>
+ Enable eager evaluation or not. If true and the REPL you are using supports eager evaluation,
+ Dataset will be ran automatically. The HTML table which generated by <code>_repl_html_</code>
+ called by notebooks like Jupyter will feedback the queries user have defined. For plain Python
+ REPL, the output will be shown like <code>dataframe.show()</code>
+ (see <a href="https://issues.apache.org/jira/browse/SPARK-24215">SPARK-24215</a> for more details).
+ </td>
+</tr>
+<tr>
+ <td><code>spark.sql.repl.eagerEval.maxNumRows</code></td>
+ <td>20</td>
+ <td>
+ Default number of rows in eager evaluation output HTML table generated by <code>_repr_html_</code> or plain text,
+ this only take effect when <code>spark.sql.repl.eagerEval.enabled</code> is set to true.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.sql.repl.eagerEval.truncate</code></td>
--- End diff --
spark.sql.* configs should be in sql-programming-guide.md
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21624
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21624
**[Test build #93064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93064/testReport)** for PR 21624 at commit [`6886111`](https://github.com/apache/spark/commit/6886111fa39fecbbc0461304c99abdfcc5caf33a).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21624
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21624#discussion_r206698447
--- Diff: docs/configuration.md ---
@@ -541,6 +575,13 @@ Apart from these, the following properties are also available, and may be useful
Python binary executable to use for PySpark in both driver and executors.
</td>
</tr>
+<tr>
+ <td><code>spark.worker.driverTerminateTimeout</code></td>
--- End diff --
Likewise spark.worker.* configs are in spark-standalone.md, it appears
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21624
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21624: [SPARK-24639][DOC] Add three config in the doc
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21624#discussion_r198189239
--- Diff: docs/configuration.md ---
@@ -456,6 +456,13 @@ Apart from these, the following properties are also available, and may be useful
from JVM to Python worker for every task.
</td>
</tr>
+<tr>
+ <td><code>spark.python.task.killTimeout</code></td>
+ <td>2s</td>
+ <td>
+ How long to wait before killing the python worker if a task cannot be interrupted.
--- End diff --
"Timeout to wait before killing the python worker when a task cannot be interrupted."
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org