You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/12/02 09:44:00 UTC
[jira] [Resolved] (SPARK-33557)
spark.storage.blockManagerSlaveTimeoutMs default value does not follow
spark.network.timeout value when the latter was changed
[ https://issues.apache.org/jira/browse/SPARK-33557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-33557.
----------------------------------
Fix Version/s: 3.1.0
Resolution: Fixed
Issue resolved by pull request 30547
[https://github.com/apache/spark/pull/30547]
> spark.storage.blockManagerSlaveTimeoutMs default value does not follow spark.network.timeout value when the latter was changed
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-33557
> URL: https://issues.apache.org/jira/browse/SPARK-33557
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.0.0, 3.0.1
> Reporter: Ohad
> Assignee: Yang Jie
> Priority: Minor
> Fix For: 3.1.0
>
>
> According to the documentation "spark.network.timeout" is the default timeout for "spark.storage.blockManagerSlaveTimeoutMs" which implies that when the user sets "spark.network.timeout" the effective value of "spark.storage.blockManagerSlaveTimeoutMs" should also be changed if it was not specifically changed.
> However this is not the case since the default value of "spark.storage.blockManagerSlaveTimeoutMs" is always the default value of "spark.network.timeout" (120s)
>
> "spark.storage.blockManagerSlaveTimeoutMs" is defined in the package object of "org.apache.spark.internal.config" as follows:
> {code:java}
> private[spark] val STORAGE_BLOCKMANAGER_SLAVE_TIMEOUT =
> ConfigBuilder("spark.storage.blockManagerSlaveTimeoutMs")
> .version("0.7.0")
> .timeConf(TimeUnit.MILLISECONDS)
> .createWithDefaultString(Network.NETWORK_TIMEOUT.defaultValueString)
> {code}
> So it seems like the its default value is indeed "fixed" to "spark.network.timeout" default value.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org