You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "caoxuewen (JIRA)" <ji...@apache.org> on 2017/07/09 14:57:00 UTC

[jira] [Created] (SPARK-21353) add checkValue in spark.internal.config about how to correctly set configurations

caoxuewen created SPARK-21353:
---------------------------------

             Summary: add checkValue in spark.internal.config about how to correctly set configurations
                 Key: SPARK-21353
                 URL: https://issues.apache.org/jira/browse/SPARK-21353
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.3.0
            Reporter: caoxuewen
            Priority: Minor


add checkValue for configurations:

spark.driver.memory
spark.executor.memory
spark.blockManager.port
spark.task.cpus
spark.dynamicAllocation.minExecutors
spark.task.maxFailures
spark.blacklist.task.maxTaskAttemptsPerExecutor
spark.blacklist.task.maxTaskAttemptsPerNode
spark.blacklist.application.maxFailedTasksPerExecutor
spark.blacklist.stage.maxFailedTasksPerExecutor
spark.blacklist.application.maxFailedExecutorsPerNode
spark.blacklist.stage.maxFailedExecutorsPerNode
spark.scheduler.listenerbus.metrics.maxListenerClassesTimed
spark.ui.retainedTasks
spark.blockManager.port
spark.driver.blockManager.port
spark.files.maxPartitionBytes
spark.files.openCostInBytes
spark.shuffle.accurateBlockThreshold
spark.shuffle.registration.timeout
spark.shuffle.registration.maxAttempts
spark.reducer.maxReqSizeShuffleToMem
and we copy the document from http://spark.apache.org/docs/latest/configuration.html

spark.driver.userClassPathFirst
spark.driver.memory
spark.executor.userClassPathFirst
spark.executor.memory
spark.yarn.isPython
spark.task.cpus
spark.dynamicAllocation.minExecutors
spark.dynamicAllocation.initialExecutors
spark.dynamicAllocation.maxExecutors
spark.shuffle.service.enabled
spark.submit.pyFiles
spark.task.maxFailures
spark.blacklist.enabled
spark.blacklist.task.maxTaskAttemptsPerExecutor
spark.blacklist.task.maxTaskAttemptsPerNode
spark.blacklist.stage.maxFailedTasksPerExecutor
spark.blacklist.stage.maxFailedExecutorsPerNode
spark.pyspark.driver.python
spark.pyspark.python
spark.ui.retainedTasks
spark.io.encryption.enabled
spark.io.encryption.keygen.algorithm
spark.io.encryption.keySizeBits
spark.authenticate.enableSaslEncryption
spark.authenticate



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org