You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by da...@apache.org on 2016/09/02 17:08:18 UTC
spark git commit: [SPARK-17261] [PYSPARK] Using HiveContext after
re-creating SparkContext in Spark 2.0 throws
"Java.lang.illegalStateException: Cannot call methods on a stopped
sparkContext"
Repository: spark
Updated Branches:
refs/heads/master 6bcbf9b74 -> ea6622865
[SPARK-17261] [PYSPARK] Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"
## What changes were proposed in this pull request?
Set SparkSession._instantiatedContext as None so that we can recreate SparkSession again.
## How was this patch tested?
Tested manually using the following command in pyspark shell
```
spark.stop()
spark = SparkSession.builder.enableHiveSupport().getOrCreate()
spark.sql("show databases").show()
```
Author: Jeff Zhang <zj...@apache.org>
Closes #14857 from zjffdu/SPARK-17261.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ea662286
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ea662286
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ea662286
Branch: refs/heads/master
Commit: ea662286561aa9fe321cb0a0e10cdeaf60440b90
Parents: 6bcbf9b
Author: Jeff Zhang <zj...@apache.org>
Authored: Fri Sep 2 10:08:14 2016 -0700
Committer: Davies Liu <da...@gmail.com>
Committed: Fri Sep 2 10:08:14 2016 -0700
----------------------------------------------------------------------
python/pyspark/sql/session.py | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/ea662286/python/pyspark/sql/session.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/session.py b/python/pyspark/sql/session.py
index 61fa107..8418abf 100644
--- a/python/pyspark/sql/session.py
+++ b/python/pyspark/sql/session.py
@@ -595,6 +595,7 @@ class SparkSession(object):
"""Stop the underlying :class:`SparkContext`.
"""
self._sc.stop()
+ SparkSession._instantiatedContext = None
@since(2.0)
def __enter__(self):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org