You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2018/08/26 21:17:00 UTC

[jira] [Created] (SPARK-25245) Explain regarding limiting modification on "spark.sql.shuffle.partitions" for structured streaming

Jungtaek Lim created SPARK-25245:
------------------------------------

             Summary: Explain regarding limiting modification on "spark.sql.shuffle.partitions" for structured streaming
                 Key: SPARK-25245
                 URL: https://issues.apache.org/jira/browse/SPARK-25245
             Project: Spark
          Issue Type: Documentation
          Components: Structured Streaming
    Affects Versions: 2.4.0
            Reporter: Jungtaek Lim


Couple of users wondered why "spark.sql.shuffle.partitions" keeps unchanged when they changed the config value after running the query. Some of them even submitted the patch as this behavior as a bug. But it is based on how state is partitioned and the behavior is intentional.

Looks like it's worth to explain it to guide doc so that no more users will be wondered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org