You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by shahidki31 <gi...@git.apache.org> on 2018/10/01 15:36:26 UTC
[GitHub] spark pull request #22601: [SPARK-25583][DOCS]Add history-server related con...
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/22601
[SPARK-25583][DOCS]Add history-server related configuration in the documentation.
## What changes were proposed in this pull request?
Add history-server related configuration in the documentation.
Some of the history server related configurations were missing in the documentation.Like, 'spark.history.store.maxDiskUsage', 'spark.ui.liveUpdate.period' etc.
## How was this patch tested?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shahidki31/spark historyConf
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22601.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22601
----
commit 4da04d362da645f1e85229386c98abeebfcb5572
Author: Shahid <sh...@...>
Date: 2018-10-01T15:16:57Z
Add history-server related configuration in the documentation.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22601
`spark.history.fs.endEventReparseChunkSize` and `spark.history.fs.inProgressOptimization.enabled` exist since `branch-2.4`. So, this PR lands at master/2.4. If you want to fix some in `branch-2.3`, please make another PR. :)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22601
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22601: [SPARK-25583][DOC]Add history-server related conf...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22601
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22601
Thank you, @shahidki31 .
Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22601
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22601
So, there is no other undocumented configurations?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...
Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22601
Hi @dongjoon-hyun , I have addressed the comments. Thank you.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22601
Also please update `[DOCS]` with `[DOC]` in the PR title.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22601: [SPARK-25583][DOCS]Add history-server related con...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22601#discussion_r221830041
--- Diff: docs/configuration.md ---
@@ -807,6 +814,14 @@ Apart from these, the following properties are also available, and may be useful
Allows jobs and stages to be killed from the web UI.
</td>
</tr>
+<tr>
+ <td><code>spark.ui.liveUpdate.period</code></td>
+ <td>100ms</td>
+ <td>
+ How often to update live entities. For live applications, this avoids a few
--- End diff --
Could you add `-1 means "never update" when replaying applications, meaning only the last write will happen.` before `For live applications`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOC]Add history-server related configurati...
Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22601
Thanks a lot @dongjoon-hyun
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22601
Hi @dongjoon-hyun. Thanks for the review.
These confugurations are recently added in the history server (2.3 +) , which are not there in the documentation, but it needs to
add in the documentation.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22601
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22601: [SPARK-25583][DOCS]Add history-server related configurat...
Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22601
@dongjoon-hyun . There is one more, "ASYNC_TRACKING_ENABLED", but it is not configurable for history server. For live UI, it is configurable. I am not sure whether this need to add in the documentation.
Rest everything has covered.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22601: [SPARK-25583][DOCS]Add history-server related con...
Posted by shahidki31 <gi...@git.apache.org>.
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22601#discussion_r221847126
--- Diff: docs/configuration.md ---
@@ -807,6 +814,14 @@ Apart from these, the following properties are also available, and may be useful
Allows jobs and stages to be killed from the web UI.
</td>
</tr>
+<tr>
+ <td><code>spark.ui.liveUpdate.period</code></td>
+ <td>100ms</td>
+ <td>
+ How often to update live entities. For live applications, this avoids a few
--- End diff --
Thanks updated.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org