[jira] [Created] (SPARK-33557) spark.storage.blockManagerSlaveTimeoutMs default value does not follow spark.network.timeout value when the latter was changed

["Ohad (Jira)" <ji...@apache.org>]

Ohad created SPARK-33557:
----------------------------

             Summary: spark.storage.blockManagerSlaveTimeoutMs default value does not follow spark.network.timeout value when the latter was changed
                 Key: SPARK-33557
                 URL: https://issues.apache.org/jira/browse/SPARK-33557
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.0.1, 3.0.0
            Reporter: Ohad


According to the documentation "spark.network.timeout" is the default timeout for "spark.storage.blockManagerSlaveTimeoutMs" which implies that when the user sets "spark.network.timeout"  the effective value of "spark.storage.blockManagerSlaveTimeoutMs" should also be changed if it was not specifically changed.

However this is not the case since the default value of "spark.storage.blockManagerSlaveTimeoutMs" is always the default value of "spark.network.timeout" (120s)

 

"spark.storage.blockManagerSlaveTimeoutMs" is defined in the package object of "org.apache.spark.internal.config" as follows:
{code:java}
private[spark] val STORAGE_BLOCKMANAGER_SLAVE_TIMEOUT =
  ConfigBuilder("spark.storage.blockManagerSlaveTimeoutMs")
    .version("0.7.0")
    .timeConf(TimeUnit.MILLISECONDS)
    .createWithDefaultString(Network.NETWORK_TIMEOUT.defaultValueString)
{code}
So it seems like the its default value is indeed "fixed" to "spark.network.timeout" default value.

 

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org