You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Juliusz Sompolski (JIRA)" <ji...@apache.org> on 2018/01/02 14:26:01 UTC

[jira] [Created] (SPARK-22938) Assert that SQLConf.get is accessed only on the driver.

Juliusz Sompolski created SPARK-22938:
-----------------------------------------

             Summary: Assert that SQLConf.get is accessed only on the driver.
                 Key: SPARK-22938
                 URL: https://issues.apache.org/jira/browse/SPARK-22938
             Project: Spark
          Issue Type: Test
          Components: SQL
    Affects Versions: 2.2.1
            Reporter: Juliusz Sompolski


Assert if code tries to access SQLConf.get on executor.
This can lead to hard to detect bugs, where the executor will read fallbackConf, falling back to default config values, ignoring potentially changed non-default configs.
If a config is to be passed to executor code, it needs to be read on the driver, and passed explicitly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org