You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Karthik Palaniappan (JIRA)" <ji...@apache.org> on 2017/08/20 07:07:00 UTC

[jira] [Created] (SPARK-21792) Document Spark Streaming Dynamic Allocation Configuration

Karthik Palaniappan created SPARK-21792:
-------------------------------------------

             Summary: Document Spark Streaming Dynamic Allocation Configuration
                 Key: SPARK-21792
                 URL: https://issues.apache.org/jira/browse/SPARK-21792
             Project: Spark
          Issue Type: Documentation
          Components: Documentation, DStreams
    Affects Versions: 2.2.0
            Reporter: Karthik Palaniappan


Support for dynamic allocation was added in https://issues.apache.org/jira/browse/SPARK-12133, and it included new config parameters in https://github.com/apache/spark/blob/branch-2.0/streaming/src/main/scala/org/apache/spark/streaming/scheduler/ExecutorAllocationManager.scala#L189.

These new parameters should be added to http://spark.apache.org/docs/latest/configuration.html.

Out of curiosity: why can't Core's dynamic allocation and Streaming's dynamic allocation both be enabled? It seems like Streaming's dynamic allocation properties should always be used for streaming applications, and Core's should always be ignored. (Either way this should be highlighted in the documentation).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org