You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2018/09/23 02:16:40 UTC
spark git commit: [MINOR][PYSPARK] Always Close the tempFile in
_serialize_to_jvm
Repository: spark
Updated Branches:
refs/heads/master 6ca87eb2e -> 0fbba76fa
[MINOR][PYSPARK] Always Close the tempFile in _serialize_to_jvm
## What changes were proposed in this pull request?
Always close the tempFile after `serializer.dump_stream(data, tempFile)` in _serialize_to_jvm
## How was this patch tested?
N/A
Closes #22523 from gatorsmile/fixMinor.
Authored-by: gatorsmile <ga...@gmail.com>
Signed-off-by: hyukjinkwon <gu...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0fbba76f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0fbba76f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0fbba76f
Branch: refs/heads/master
Commit: 0fbba76faa00a18eef5d8c2ef2e673744d0d490b
Parents: 6ca87eb
Author: gatorsmile <ga...@gmail.com>
Authored: Sun Sep 23 10:16:33 2018 +0800
Committer: hyukjinkwon <gu...@apache.org>
Committed: Sun Sep 23 10:16:33 2018 +0800
----------------------------------------------------------------------
python/pyspark/context.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/0fbba76f/python/pyspark/context.py
----------------------------------------------------------------------
diff --git a/python/pyspark/context.py b/python/pyspark/context.py
index 87255c4..0924d3d 100644
--- a/python/pyspark/context.py
+++ b/python/pyspark/context.py
@@ -537,8 +537,10 @@ class SparkContext(object):
# parallelize from there.
tempFile = NamedTemporaryFile(delete=False, dir=self._temp_dir)
try:
- serializer.dump_stream(data, tempFile)
- tempFile.close()
+ try:
+ serializer.dump_stream(data, tempFile)
+ finally:
+ tempFile.close()
return reader_func(tempFile.name)
finally:
# we eagerily reads the file so we can delete right after.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org