You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/26 12:33:27 UTC
[GitHub] [spark] beliefer opened a new pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
beliefer opened a new pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035
### What changes were proposed in this pull request?
The `configuration.md` exists some config not organized by `ConfigEntry`.
This PR supplements version for configuration only appear in configuration doc.
### Why are the changes needed?
Supplemental configuration version information.
### Does this PR introduce any user-facing change?
'No'.
### How was this patch tested?
Jenkins test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398538302
##########
File path: docs/configuration.md
##########
@@ -312,6 +318,7 @@ of the most common options to set are:
the Kubernetes device plugin naming convention. (e.g. For GPUs on Kubernetes
this config would be set to nvidia.com or amd.com)
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27362, commit ID: 1277f8fa92da85d9e39d9146e3099fcb75c71a8f#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398543425
##########
File path: docs/configuration.md
##########
@@ -867,6 +889,7 @@ Apart from these, the following properties are also available, and may be useful
block transfer. For environments where off-heap memory is tightly limited, users may wish to
turn this off to force all allocations from Netty to be on-heap.
</td>
+ <td>1.2.0</td>
Review comment:
SPARK-4188, commit ID: c1ea5c542f3267c0b23a7775887e3a6ece793fe3#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-606378064
Merged to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604406572
**[Test build #120421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120421/testReport)** for PR 28035 at commit [`9df81a8`](https://github.com/apache/spark/commit/9df81a8b01e561de3bafd70e30d7f9510efd106b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962231
##########
File path: docs/configuration.md
##########
@@ -2526,27 +2573,30 @@ like shuffle, just replace "rpc" with "shuffle" in the property names except
<code>spark.{driver|executor}.rpc.netty.dispatcher.numThreads</code>, which is only for RPC module.
<table class="table">
-<tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr>
+<tr><th>Property Name</th><th>Default</th><th>Meaning</th><th>Since Version</th></tr>
<tr>
<td><code>spark.{driver|executor}.rpc.io.serverThreads</code></td>
<td>
Fall back on <code>spark.rpc.io.serverThreads</code>
</td>
<td>Number of threads used in the server thread pool</td>
+ <td>1.6.0</td>
Review comment:
SPARK-10745, commit ID: 7c5b641808740ba5eed05ba8204cdbaf3fc579f5#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961624
##########
File path: docs/configuration.md
##########
@@ -2080,6 +2123,7 @@ Apart from these, the following properties are also available, and may be useful
that write events to eventLogs. Consider increasing value if the listener events corresponding to eventLog queue
are dropped. Increasing this value may result in the driver using more memory.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-28574, commit ID: c212c9d9ed7375cd1ea16c118733edd84037ec0d#diff-eb519ad78cc3cf0b95839cc37413b509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398538892
##########
File path: docs/configuration.md
##########
@@ -337,13 +344,15 @@ of the most common options to set are:
<em>Note:</em> This will be overridden by SPARK_LOCAL_DIRS (Standalone), MESOS_SANDBOX (Mesos) or
LOCAL_DIRS (YARN) environment variables set by the cluster manager.
</td>
+ <td>0.5.0</td>
</tr>
<tr>
<td><code>spark.logConf</code></td>
<td>false</td>
<td>
Logs the effective SparkConf as INFO when a SparkContext is started.
</td>
+ <td>0.9.0</td>
Review comment:
No JIRA ID, commit ID: d8bcc8e9a095c1b20dd7a17b6535800d39bff80e#diff-364713d7776956cb8b0a771e9b62f82d
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398960306
##########
File path: docs/configuration.md
##########
@@ -1692,23 +1725,29 @@ Apart from these, the following properties are also available, and may be useful
<td>2.1.0</td>
</tr>
<tr>
- <td><code>spark.hadoop.cloneConf</code></td>
- <td>false</td>
- <td>If set to true, clones a new Hadoop <code>Configuration</code> object for each task. This
+ <td><code>spark.hadoop.cloneConf</code></td>
+ <td>false</td>
+ <td>
+ If set to true, clones a new Hadoop <code>Configuration</code> object for each task. This
option should be enabled to work around <code>Configuration</code> thread-safety issues (see
<a href="https://issues.apache.org/jira/browse/SPARK-2546">SPARK-2546</a> for more details).
This is disabled by default in order to avoid unexpected performance regressions for jobs that
- are not affected by these issues.</td>
+ are not affected by these issues.
+ </td>
+ <td>1.0.3</td>
Review comment:
SPARK-2546, commit ID: 6d8f1dd15afdc7432b5721c89f9b2b402460322b#diff-83eb37f7b0ebed3c14ccb7bff0d577c2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398542220
##########
File path: docs/configuration.md
##########
@@ -756,6 +774,7 @@ Apart from these, the following properties are also available, and may be useful
artifact server like Artifactory. Details on the settings file format can be
found at <a href="http://ant.apache.org/ivy/history/latest-milestone/settings.html">Settings Files</a>
</td>
+ <td>2.2.0</td>
Review comment:
SPARK-17568, commit ID: 3bc2eff8880a3ba8d4318118715ea1a47048e3de#diff-4d2ab44195558d5a9d5f15b8803ef39d
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398538033
##########
File path: docs/configuration.md
##########
@@ -302,6 +307,7 @@ of the most common options to set are:
write to STDOUT a JSON string in the format of the ResourceInformation class. This has a
name and an array of addresses.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27024, commit ID: db2e3c43412e4a7fb4a46c58d73d9ab304a1e949#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398541947
##########
File path: docs/configuration.md
##########
@@ -744,6 +761,7 @@ Apart from these, the following properties are also available, and may be useful
<code>spark.jars.packages</code>. This will override the Ivy property <code>ivy.default.ivy.user.dir</code>
which defaults to ~/.ivy2.
</td>
+ <td>1.3.0</td>
Review comment:
SPARK-5341, commit ID: 3b7acd22ab4a134c74746e3b9a803dbd34d43855#diff-63a5d817d2d45ae24de577f6a1bd80f9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398540020
##########
File path: docs/configuration.md
##########
@@ -636,6 +648,7 @@ Apart from these, the following properties are also available, and may be useful
Add the environment variable specified by <code>EnvironmentVariableName</code> to the Executor
process. The user can specify multiple of these to set multiple environment variables.
</td>
+ <td>0.9.0</td>
Review comment:
No JIRA ID, commit ID: 642029e7f43322f84abe4f7f36bb0b1b95d8101d#diff-529fc5c06b9731c1fbda6f3db60b16aa
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398960902
##########
File path: docs/configuration.md
##########
@@ -1865,7 +1905,7 @@ Apart from these, the following properties are also available, and may be useful
block transfer. For environments where off-heap memory is tightly limited, users may wish to
turn this off to force all allocations to be on-heap.
</td>
- <td></td>
+ <td>3.0.0</td>
Review comment:
SPARK-24920, commit ID: e103c4a5e72bab8862ff49d6d4c1e62e642fc412#diff-0ac65da2bc6b083fb861fe410c7688c2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961494
##########
File path: docs/configuration.md
##########
@@ -2062,6 +2103,7 @@ Apart from these, the following properties are also available, and may be useful
Consider increasing value, if the listener events corresponding to appStatus queue are dropped.
Increasing this value may result in the driver using more memory.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-28574, commit ID: c212c9d9ed7375cd1ea16c118733edd84037ec0d#diff-eb519ad78cc3cf0b95839cc37413b509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961069
##########
File path: docs/configuration.md
##########
@@ -1877,7 +1917,7 @@ Apart from these, the following properties are also available, and may be useful
essentially allows it to try a range of ports from the start port specified
to port + maxRetries.
</td>
- <td></td>
+ <td>1.1.1</td>
Review comment:
SPARK-3565, commit ID: 32f2222e915f31422089139944a077e2cbd442f9#diff-d239aee594001f8391676e1047a0381e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604407116
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398537781
##########
File path: docs/configuration.md
##########
@@ -293,6 +297,7 @@ of the most common options to set are:
<code>spark.executor.resource.{resourceName}.discoveryScript</code>
for the executor to find the resource on startup.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27760, commit ID: d30284b5a51dd784f663eb4eea37087b35a54d00#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398545750
##########
File path: docs/configuration.md
##########
@@ -887,6 +911,7 @@ Apart from these, the following properties are also available, and may be useful
application (see <code>spark.shuffle.service.enabled</code> option below). If set below 1,
will fallback to OS default defined by Netty's <code>io.netty.util.NetUtil#SOMAXCONN</code>.
</td>
+ <td>1.1.1</td>
Review comment:
SPARK-2468, commit ID: 66b4c81db7e826c00f7fb449b8a8af810cf7dd9a#diff-bdee8e601924d41e93baa7287189e878
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604406572
**[Test build #120421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120421/testReport)** for PR 28035 at commit [`9df81a8`](https://github.com/apache/spark/commit/9df81a8b01e561de3bafd70e30d7f9510efd106b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961894
##########
File path: docs/configuration.md
##########
@@ -2271,6 +2316,7 @@ Apart from these, the following properties are also available, and may be useful
in order to assign resource slots (e.g. a 0.2222 configuration, or 1/0.2222 slots will become
4 tasks/resource, not 5).
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27760, commit ID: d30284b5a51dd784f663eb4eea37087b35a54d00#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398960438
##########
File path: docs/configuration.md
##########
@@ -1692,23 +1725,29 @@ Apart from these, the following properties are also available, and may be useful
<td>2.1.0</td>
</tr>
<tr>
- <td><code>spark.hadoop.cloneConf</code></td>
- <td>false</td>
- <td>If set to true, clones a new Hadoop <code>Configuration</code> object for each task. This
+ <td><code>spark.hadoop.cloneConf</code></td>
+ <td>false</td>
+ <td>
+ If set to true, clones a new Hadoop <code>Configuration</code> object for each task. This
option should be enabled to work around <code>Configuration</code> thread-safety issues (see
<a href="https://issues.apache.org/jira/browse/SPARK-2546">SPARK-2546</a> for more details).
This is disabled by default in order to avoid unexpected performance regressions for jobs that
- are not affected by these issues.</td>
+ are not affected by these issues.
+ </td>
+ <td>1.0.3</td>
</tr>
<tr>
- <td><code>spark.hadoop.validateOutputSpecs</code></td>
- <td>true</td>
- <td>If set to true, validates the output specification (e.g. checking if the output directory already exists)
+ <td><code>spark.hadoop.validateOutputSpecs</code></td>
+ <td>true</td>
+ <td>
+ If set to true, validates the output specification (e.g. checking if the output directory already exists)
used in saveAsHadoopFile and other variants. This can be disabled to silence exceptions due to pre-existing
- output directories. We recommend that users do not disable this except if trying to achieve compatibility with
- previous versions of Spark. Simply use Hadoop's FileSystem API to delete output directories by hand.
- This setting is ignored for jobs generated through Spark Streaming's StreamingContext, since
- data may need to be rewritten to pre-existing output directories during checkpoint recovery.</td>
+ output directories. We recommend that users do not disable this except if trying to achieve compatibility
+ with previous versions of Spark. Simply use Hadoop's FileSystem API to delete output directories by hand.
+ This setting is ignored for jobs generated through Spark Streaming's StreamingContext, since data may
+ need to be rewritten to pre-existing output directories during checkpoint recovery.
+ </td>
+ <td>1.0.1</td>
Review comment:
SPARK-1677, commit ID: 8100cbdb7546e8438019443cfc00683017c81278#diff-f70e97c099b5eac05c75288cb215e080
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398542873
##########
File path: docs/configuration.md
##########
@@ -849,6 +869,7 @@ Apart from these, the following properties are also available, and may be useful
set to a non-zero value. This retry logic helps stabilize large shuffles in the face of long GC
pauses or transient network connectivity issues.
</td>
+ <td>1.2.0</td>
Review comment:
SPARK-4188, commit ID: c1ea5c542f3267c0b23a7775887e3a6ece793fe3#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398538611
##########
File path: docs/configuration.md
##########
@@ -337,13 +344,15 @@ of the most common options to set are:
<em>Note:</em> This will be overridden by SPARK_LOCAL_DIRS (Standalone), MESOS_SANDBOX (Mesos) or
LOCAL_DIRS (YARN) environment variables set by the cluster manager.
</td>
+ <td>0.5.0</td>
Review comment:
No JIRA ID, commit ID: 0e93891d3d7df849cff6442038c111ffd42a5243#diff-17fd275d280b667722664ed833c6402a
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398537453
##########
File path: docs/configuration.md
##########
@@ -226,6 +229,7 @@ of the most common options to set are:
the Kubernetes device plugin naming convention. (e.g. For GPUs on Kubernetes
this config would be set to nvidia.com or amd.com)
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27362, commit ID: 1277f8fa92da85d9e39d9146e3099fcb75c71a8f#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398546396
##########
File path: docs/configuration.md
##########
@@ -926,6 +952,7 @@ Apart from these, the following properties are also available, and may be useful
<code>spark.shuffle.io.retryWait</code>), if those limits are reached the task will fail with
fetch failure.
</td>
+ <td>2.3.0</td>
Review comment:
SPARK-21175, commit ID: 799e13161e89f1ea96cb1bc7b507a05af2e89cd0#diff-0ac65da2bc6b083fb861fe410c7688c2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398546706
##########
File path: docs/configuration.md
##########
@@ -1233,13 +1260,15 @@ Apart from these, the following properties are also available, and may be useful
<td>
How many finished executions the Spark UI and status APIs remember before garbage collecting.
</td>
+ <td>1.5.0</td>
Review comment:
SPARK-8861 and SPARK-8862, commit ID: ebc3aad272b91cf58e2e1b4aa92b49b8a947a045#diff-81764e4d52817f83bdd5336ef1226bd9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398536941
##########
File path: docs/configuration.md
##########
@@ -206,6 +207,7 @@ of the most common options to set are:
<code>spark.driver.resource.{resourceName}.discoveryScript</code>
for the driver to find the resource on startup.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27760, commit ID: d30284b5a51dd784f663eb4eea37087b35a54d00#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398547044
##########
File path: docs/configuration.md
##########
@@ -1233,13 +1260,15 @@ Apart from these, the following properties are also available, and may be useful
<td>
How many finished executions the Spark UI and status APIs remember before garbage collecting.
</td>
+ <td>1.5.0</td>
</tr>
<tr>
<td><code>spark.streaming.ui.retainedBatches</code></td>
<td>1000</td>
<td>
How many finished batches the Spark UI and status APIs remember before garbage collecting.
</td>
+ <td>1.0.0</td>
Review comment:
SPARK-1386, commit ID: f36dc3fed0a0671b0712d664db859da28c0a98e2#diff-56b8d67d07284cfab165d5363bd3500e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604407123
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25129/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604412589
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398541502
##########
File path: docs/configuration.md
##########
@@ -727,6 +742,7 @@ Apart from these, the following properties are also available, and may be useful
repositories given by the command-line option <code>--repositories</code>. For more details, see
<a href="submitting-applications.html#advanced-dependency-management">Advanced Dependency Management</a>.
</td>
+ <td>1.5.0</td>
Review comment:
SPARK-9263, commit ID: 34335719a372c1951fdb4dd25b75b086faf1076f#diff-63a5d817d2d45ae24de577f6a1bd80f9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962703
##########
File path: docs/configuration.md
##########
@@ -2737,7 +2787,7 @@ Spark subsystems.
Executable for executing sparkR shell in client modes for driver. Ignored in cluster modes. It is the same as environment variable <code>SPARKR_DRIVER_R</code>, but take precedence over it.
<code>spark.r.shell.command</code> is used for sparkR shell while <code>spark.r.driver.command</code> is used for running R script.
</td>
- <td></td>
+ <td>2.1.0</td>
Review comment:
SPARK-17178, commit ID: fa6347938fc1c72ddc03a5f3cd2e929b5694f0a6#diff-a78ecfc6a89edfaf0b60a5eaa0381970
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962552
##########
File path: docs/configuration.md
##########
@@ -2728,7 +2778,7 @@ Spark subsystems.
<td>
Executable for executing R scripts in client modes for driver. Ignored in cluster modes.
</td>
- <td></td>
+ <td>1.5.3</td>
Review comment:
SPARK-10971, commit ID: 9695f452e86a88bef3bcbd1f3c0b00ad9e9ac6e1#diff-025470e1b7094d7cf4a78ea353fb3981
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398546060
##########
File path: docs/configuration.md
##########
@@ -915,6 +940,7 @@ Apart from these, the following properties are also available, and may be useful
<td>
Cache entries limited to the specified memory footprint, in bytes unless otherwise specified.
</td>
+ <td>2.3.0</td>
Review comment:
SPARK-21501, commit ID: 1662e93119d68498942386906de309d35f4a135f#diff-97d5edc927a83a678e013ae00343df94
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398536245
##########
File path: docs/configuration.md
##########
@@ -143,6 +143,7 @@ of the most common options to set are:
<td>
The name of your application. This will appear in the UI and in log data.
</td>
+ <td>0.9.0</td>
Review comment:
No JIRA ID, commit ID: 994f080f8ae3372366e6004600ba791c8a372ff0#diff-529fc5c06b9731c1fbda6f3db60b16aa
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604412429
**[Test build #120421 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120421/testReport)** for PR 28035 at commit [`9df81a8`](https://github.com/apache/spark/commit/9df81a8b01e561de3bafd70e30d7f9510efd106b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961218
##########
File path: docs/configuration.md
##########
@@ -1920,7 +1960,7 @@ Apart from these, the following properties are also available, and may be useful
out and giving up. To avoid unwilling timeout caused by long pause like GC,
you can set larger value.
</td>
- <td></td>
+ <td>1.1.1</td>
Review comment:
SPARK-2677, commit ID: bd3ce2ffb8964abb4d59918ebb2c230fe4614aa2#diff-f748e95f2aa97ed715afa53ddeeac9de
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-610370866
Merged to branch-3.0 too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604412599
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120421/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604412589
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398541667
##########
File path: docs/configuration.md
##########
@@ -735,6 +751,7 @@ Apart from these, the following properties are also available, and may be useful
Comma-separated list of groupId:artifactId, to exclude while resolving the dependencies
provided in <code>spark.jars.packages</code> to avoid dependency conflicts.
</td>
+ <td>1.5.0</td>
Review comment:
SPARK-9263, commit ID: 34335719a372c1951fdb4dd25b75b086faf1076f#diff-63a5d817d2d45ae24de577f6a1bd80f9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398547319
##########
File path: docs/configuration.md
##########
@@ -1633,6 +1662,7 @@ Apart from these, the following properties are also available, and may be useful
Default number of partitions in RDDs returned by transformations like <code>join</code>,
<code>reduceByKey</code>, and <code>parallelize</code> when not set by user.
</td>
+ <td>0.5.0</td>
Review comment:
No JIrA ID, commit ID: e5c4cd8a5e188592f8786a265c0cd073c69ac886#diff-0544ebf7533fa70ff5103e0fe1f0b036
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398539389
##########
File path: docs/configuration.md
##########
@@ -467,6 +477,7 @@ Apart from these, the following properties are also available, and may be useful
Instead, please set this through the <code>--driver-java-options</code> command line option or in
your default properties file.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-23472, commit ID: f83000597f250868de9722d8285fed013abc5ecf#diff-a78ecfc6a89edfaf0b60a5eaa0381970
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398539120
##########
File path: docs/configuration.md
##########
@@ -352,6 +361,7 @@ of the most common options to set are:
The cluster manager to connect to. See the list of
<a href="submitting-applications.html#master-urls"> allowed master URL's</a>.
</td>
+ <td>0.9.0</td>
Review comment:
SPARK-544, commit ID: 2573add94cf920a88f74d80d8ea94218d812704d#diff-529fc5c06b9731c1fbda6f3db60b16aa
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604407116
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961667
##########
File path: docs/configuration.md
##########
@@ -2089,6 +2133,7 @@ Apart from these, the following properties are also available, and may be useful
Consider increasing value if the listener events corresponding to streams queue are dropped. Increasing
this value may result in the driver using more memory.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-28574, commit ID: c212c9d9ed7375cd1ea16c118733edd84037ec0d#diff-eb519ad78cc3cf0b95839cc37413b509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398960612
##########
File path: docs/configuration.md
##########
@@ -1728,6 +1767,7 @@ Apart from these, the following properties are also available, and may be useful
Version 2 may have better performance, but version 1 may handle failures better in certain situations,
as per <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4815">MAPREDUCE-4815</a>.
</td>
+ <td>2.2.0</td>
Review comment:
SPARK-20107, commit ID: edc87d76efea7b4d19d9d0c4ddba274a3ccb8752#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962291
##########
File path: docs/configuration.md
##########
@@ -2526,27 +2573,30 @@ like shuffle, just replace "rpc" with "shuffle" in the property names except
<code>spark.{driver|executor}.rpc.netty.dispatcher.numThreads</code>, which is only for RPC module.
<table class="table">
-<tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr>
+<tr><th>Property Name</th><th>Default</th><th>Meaning</th><th>Since Version</th></tr>
<tr>
<td><code>spark.{driver|executor}.rpc.io.serverThreads</code></td>
<td>
Fall back on <code>spark.rpc.io.serverThreads</code>
</td>
<td>Number of threads used in the server thread pool</td>
+ <td>1.6.0</td>
</tr>
<tr>
<td><code>spark.{driver|executor}.rpc.io.clientThreads</code></td>
<td>
Fall back on <code>spark.rpc.io.clientThreads</code>
</td>
<td>Number of threads used in the client thread pool</td>
+ <td>1.6.0</td>
Review comment:
SPARK-10745, commit ID: 7c5b641808740ba5eed05ba8204cdbaf3fc579f5#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398547660
##########
File path: docs/configuration.md
##########
@@ -1652,6 +1682,7 @@ Apart from these, the following properties are also available, and may be useful
Communication timeout to use when fetching files added through SparkContext.addFile() from
the driver.
</td>
+ <td>1.0.0</td>
Review comment:
No JIRA ID, commit ID: f6f9d02e85d17da2f742ed0062f1648a9293e73c#diff-d239aee594001f8391676e1047a0381e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28035: [SPARK-31269][DOC]
Supplement version for configuration only appear in configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604412599
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120421/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398537201
##########
File path: docs/configuration.md
##########
@@ -216,6 +218,7 @@ of the most common options to set are:
name and an array of addresses. For a client-submitted driver, discovery script must assign
different resource addresses to this driver comparing to other drivers on the same host.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-27488, commit ID: 74e5e41eebf9ed596b48e6db52a2a9c642e5cbc3#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962017
##########
File path: docs/configuration.md
##########
@@ -2335,6 +2381,7 @@ Apart from these, the following properties are also available, and may be useful
<td>
Number of consecutive stage attempts allowed before a stage is aborted.
</td>
+ <td>2.2.0</td>
Review comment:
SPARK-13369, commit ID: 7b5d873aef672aa0aee41e338bab7428101e1ad3#diff-6a9ff7fb74fd490a50462d45db2d5e11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398540396
##########
File path: docs/configuration.md
##########
@@ -670,6 +683,7 @@ Apart from these, the following properties are also available, and may be useful
by <code>pstats.Stats()</code>. If this is specified, the profile result will not be displayed
automatically.
</td>
+ <td>1.2.0</td>
Review comment:
SPARK-3478, commit ID: 1aa549ba9839565274a12c52fa1075b424f138a6#diff-d6fe2792e44f6babc94aabfefc8b9bce
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398547958
##########
File path: docs/configuration.md
##########
@@ -1664,6 +1695,7 @@ Apart from these, the following properties are also available, and may be useful
disabled in order to use Spark local directories that reside on NFS filesystems (see
<a href="https://issues.apache.org/jira/browse/SPARK-6313">SPARK-6313</a> for more details).
</td>
+ <td>1.2.2</td>
Review comment:
SPARK-6313, commit ID: a2a94a154bdd00753b8d5e344d712664c7151050#diff-d239aee594001f8391676e1047a0381e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961558
##########
File path: docs/configuration.md
##########
@@ -2071,6 +2113,7 @@ Apart from these, the following properties are also available, and may be useful
executor management listeners. Consider increasing value if the listener events corresponding to
executorManagement queue are dropped. Increasing this value may result in the driver using more memory.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-28574, commit ID: c212c9d9ed7375cd1ea16c118733edd84037ec0d#diff-eb519ad78cc3cf0b95839cc37413b509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398543145
##########
File path: docs/configuration.md
##########
@@ -858,6 +879,7 @@ Apart from these, the following properties are also available, and may be useful
large clusters. For clusters with many hard disks and few hosts, this may result in insufficient
concurrency to saturate all disks, and so users may consider increasing this value.
</td>
+ <td>1.2.1</td>
Review comment:
SPARK-4740, commit ID: 441ec3451730c7ae3dbef8952e313071d6147ab6#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962443
##########
File path: docs/configuration.md
##########
@@ -2526,27 +2573,30 @@ like shuffle, just replace "rpc" with "shuffle" in the property names except
<code>spark.{driver|executor}.rpc.netty.dispatcher.numThreads</code>, which is only for RPC module.
<table class="table">
-<tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr>
+<tr><th>Property Name</th><th>Default</th><th>Meaning</th><th>Since Version</th></tr>
<tr>
<td><code>spark.{driver|executor}.rpc.io.serverThreads</code></td>
<td>
Fall back on <code>spark.rpc.io.serverThreads</code>
</td>
<td>Number of threads used in the server thread pool</td>
+ <td>1.6.0</td>
</tr>
<tr>
<td><code>spark.{driver|executor}.rpc.io.clientThreads</code></td>
<td>
Fall back on <code>spark.rpc.io.clientThreads</code>
</td>
<td>Number of threads used in the client thread pool</td>
+ <td>1.6.0</td>
</tr>
<tr>
<td><code>spark.{driver|executor}.rpc.netty.dispatcher.numThreads</code></td>
<td>
Fall back on <code>spark.rpc.netty.dispatcher.numThreads</code>
</td>
<td>Number of threads used in RPC message dispatcher thread pool</td>
+ <td>3.0.0</td>
Review comment:
SPARK-29398, commit ID: 2f0a38cb50e3e8b4b72219c7b2b8b15d51f6b931#diff-a68a21481fea5053848ca666dd3201d8
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398962896
##########
File path: docs/configuration.md
##########
@@ -2769,6 +2819,7 @@ Spark subsystems.
Checkpoint interval for graph and message in Pregel. It used to avoid stackOverflowError due to long lineage chains
after lots of iterations. The checkpoint is disabled by default.
</td>
+ <td>2.2.0</td>
Review comment:
SPARK-5484, commit ID: f971ce5dd0788fe7f5d2ca820b9ea3db72033ddc#diff-e399679417ffa6eeedf26a7630baca16
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398543988
##########
File path: docs/configuration.md
##########
@@ -875,6 +898,7 @@ Apart from these, the following properties are also available, and may be useful
(Netty only) How long to wait between retries of fetches. The maximum delay caused by retrying
is 15 seconds by default, calculated as <code>maxRetries * retryWait</code>.
</td>
+ <td>1.2.1</td>
Review comment:
No JIRA ID, commit ID: 5e5d8f469a1bea9bbe606f772ccdcab7c184c651#diff-d2ce9b38bdc38ca9d7119f9c2cf79907
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398541096
##########
File path: docs/configuration.md
##########
@@ -680,6 +694,7 @@ Apart from these, the following properties are also available, and may be useful
(e.g. <code>512m</code>, <code>2g</code>).
If the memory used during aggregation goes above this amount, it will spill the data into disks.
</td>
+ <td>1.1.0</td>
Review comment:
SPARK-2538, commit ID: 14174abd421318e71c16edd24224fd5094bdfed4#diff-d6fe2792e44f6babc94aabfefc8b9bce
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398540311
##########
File path: docs/configuration.md
##########
@@ -659,7 +672,7 @@ Apart from these, the following properties are also available, and may be useful
By default the <code>pyspark.profiler.BasicProfiler</code> will be used, but this can be overridden by
passing a profiler class in as a parameter to the <code>SparkContext</code> constructor.
</td>
- <td></td>
+ <td>1.2.0</td>
Review comment:
SPARK-3478, commit ID: 1aa549ba9839565274a12c52fa1075b424f138a6#diff-d6fe2792e44f6babc94aabfefc8b9bce
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398539738
##########
File path: docs/configuration.md
##########
@@ -540,6 +551,7 @@ Apart from these, the following properties are also available, and may be useful
verbose gc logging to a file named for the executor ID of the app in /tmp, pass a 'value' of:
<code>-verbose:gc -Xloggc:/tmp/{{APP_ID}}-{{EXECUTOR_ID}}.gc</code>
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-23472, commit ID: f83000597f250868de9722d8285fed013abc5ecf#diff-a78ecfc6a89edfaf0b60a5eaa0381970
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398542543
##########
File path: docs/configuration.md
##########
@@ -764,6 +783,7 @@ Apart from these, the following properties are also available, and may be useful
Comma-separated list of additional remote repositories to search for the maven coordinates
given with <code>--packages</code> or <code>spark.jars.packages</code>.
</td>
+ <td>2.3.0</td>
Review comment:
SPARK-21403, commit ID: d8257b99ddae23f702f312640a5335ddb4554403#diff-4d2ab44195558d5a9d5f15b8803ef39d
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398548403
##########
File path: docs/configuration.md
##########
@@ -1672,6 +1704,7 @@ Apart from these, the following properties are also available, and may be useful
Whether to overwrite files added through SparkContext.addFile() when the target file exists and
its contents do not match those of the source.
</td>
+ <td>1.0.0</td>
Review comment:
No JIRA ID, commit ID: 84670f2715392859624df290c1b52eb4ed4a9cb1#diff-d239aee594001f8391676e1047a0381e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#issuecomment-604407123
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25129/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398961429
##########
File path: docs/configuration.md
##########
@@ -2053,6 +2093,7 @@ Apart from these, the following properties are also available, and may be useful
that register to the listener bus. Consider increasing value, if the listener events corresponding
to shared queue are dropped. Increasing this value may result in the driver using more memory.
</td>
+ <td>3.0.0</td>
Review comment:
SPARK-28574, commit ID: c212c9d9ed7375cd1ea16c118733edd84037ec0d#diff-eb519ad78cc3cf0b95839cc37413b509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #28035:
[SPARK-31269][DOC] Supplement version for configuration only appear in
configuration doc
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #28035: [SPARK-31269][DOC] Supplement version for configuration only appear in configuration doc
URL: https://github.com/apache/spark/pull/28035#discussion_r398960746
##########
File path: docs/configuration.md
##########
@@ -1842,7 +1882,7 @@ Apart from these, the following properties are also available, and may be useful
need to be increased, so that incoming connections are not dropped when a large number of
connections arrives in a short period of time.
</td>
- <td></td>
+ <td>3.0.0</td>
Review comment:
SPARK-27868, commit ID: 09ed64d795d3199a94e175273fff6fcea6b52131#diff-76e731333fb756df3bff5ddb3b731c46
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org