You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ue...@apache.org on 2017/09/08 05:26:15 UTC
spark git commit: [SPARK-21950][SQL][PYTHON][TEST]
pyspark.sql.tests.SQLTests2 should stop SparkContext.
Repository: spark
Updated Branches:
refs/heads/master c26976fe1 -> 57bc1e9eb
[SPARK-21950][SQL][PYTHON][TEST] pyspark.sql.tests.SQLTests2 should stop SparkContext.
## What changes were proposed in this pull request?
`pyspark.sql.tests.SQLTests2` doesn't stop newly created spark context in the test and it might affect the following tests.
This pr makes `pyspark.sql.tests.SQLTests2` stop `SparkContext`.
## How was this patch tested?
Existing tests.
Author: Takuya UESHIN <ue...@databricks.com>
Closes #19158 from ueshin/issues/SPARK-21950.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/57bc1e9e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/57bc1e9e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/57bc1e9e
Branch: refs/heads/master
Commit: 57bc1e9eb452284cbed090dbd5008eb2062f1b36
Parents: c26976f
Author: Takuya UESHIN <ue...@databricks.com>
Authored: Fri Sep 8 14:26:07 2017 +0900
Committer: Takuya UESHIN <ue...@databricks.com>
Committed: Fri Sep 8 14:26:07 2017 +0900
----------------------------------------------------------------------
python/pyspark/sql/tests.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/57bc1e9e/python/pyspark/sql/tests.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/tests.py b/python/pyspark/sql/tests.py
index 3d87ccf..1bc889c 100644
--- a/python/pyspark/sql/tests.py
+++ b/python/pyspark/sql/tests.py
@@ -2619,8 +2619,12 @@ class SQLTests2(ReusedPySparkTestCase):
self.sc.stop()
sc = SparkContext('local[4]', self.sc.appName)
spark = SparkSession.builder.getOrCreate()
- df = spark.createDataFrame([(1, 2)], ["c", "c"])
- df.collect()
+ try:
+ df = spark.createDataFrame([(1, 2)], ["c", "c"])
+ df.collect()
+ finally:
+ spark.stop()
+ sc.stop()
class UDFInitializationTests(unittest.TestCase):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org