You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2015/05/08 10:22:01 UTC
[jira] [Updated] (SPARK-7478) Add a SQLContext.getOrCreate to
maintain a singleton instance of SQLContext
[ https://issues.apache.org/jira/browse/SPARK-7478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-7478:
---------------------------------
Description:
Having a SQLContext singleton would make it easier for applications to use a lazily instantiated single shared instance of SQLContext when needed. It would avoid problems like
1. In REPL/notebook environment, rerunning the line "val sqlContext = new SQLContext" multiple times created different contexts while overriding the reference to previous context, leading to issues like registered temp tables going missing.
2. In Streaming, creating SQLContext directly leads to serialization/deserialization issues when attempting to recover from DStream checkpoints. See [SPARK-6770].
was:
Having a SQLContext singleton would make it easier for applications to use a lazily instantiated single shared instance of SQLContext when needed. It would avoid problems like
1. In REPL/notebook environment, rerunning the line "val sqlContext = new SQLContext" multiple times created different contexts while overriding the reference to previous context, leading to issues like registered temp tables going missing.
2. In Streaming, creating SQLContext directly leads to serialization/deserialization issues when attempting to recover from DStream checkpoints. See [SPARK-6770](https://issues.apache.org/jira/browse/SPARK-6770)
> Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
> ---------------------------------------------------------------------------
>
> Key: SPARK-7478
> URL: https://issues.apache.org/jira/browse/SPARK-7478
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
>
> Having a SQLContext singleton would make it easier for applications to use a lazily instantiated single shared instance of SQLContext when needed. It would avoid problems like
> 1. In REPL/notebook environment, rerunning the line "val sqlContext = new SQLContext" multiple times created different contexts while overriding the reference to previous context, leading to issues like registered temp tables going missing.
> 2. In Streaming, creating SQLContext directly leads to serialization/deserialization issues when attempting to recover from DStream checkpoints. See [SPARK-6770].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org