You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2017/01/12 09:05:05 UTC
spark git commit: [SPARK-19164][PYTHON][SQL] Remove unused
UserDefinedFunction._broadcast
Repository: spark
Updated Branches:
refs/heads/master 2c586f506 -> 5db35b312
[SPARK-19164][PYTHON][SQL] Remove unused UserDefinedFunction._broadcast
## What changes were proposed in this pull request?
Removes `UserDefinedFunction._broadcast` and `UserDefinedFunction.__del__` method.
## How was this patch tested?
Existing unit tests.
Author: zero323 <ze...@users.noreply.github.com>
Closes #16538 from zero323/SPARK-19164.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5db35b31
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5db35b31
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5db35b31
Branch: refs/heads/master
Commit: 5db35b312e96dea07f03100c64b58723c2430cd7
Parents: 2c586f5
Author: zero323 <ze...@users.noreply.github.com>
Authored: Thu Jan 12 01:05:02 2017 -0800
Committer: Reynold Xin <rx...@databricks.com>
Committed: Thu Jan 12 01:05:02 2017 -0800
----------------------------------------------------------------------
python/pyspark/sql/functions.py | 6 ------
1 file changed, 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/5db35b31/python/pyspark/sql/functions.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/functions.py b/python/pyspark/sql/functions.py
index 7fe901a..66d993a 100644
--- a/python/pyspark/sql/functions.py
+++ b/python/pyspark/sql/functions.py
@@ -1826,7 +1826,6 @@ class UserDefinedFunction(object):
def __init__(self, func, returnType, name=None):
self.func = func
self.returnType = returnType
- self._broadcast = None
self._judf = self._create_judf(name)
def _create_judf(self, name):
@@ -1842,11 +1841,6 @@ class UserDefinedFunction(object):
name, wrapped_func, jdt)
return judf
- def __del__(self):
- if self._broadcast is not None:
- self._broadcast.unpersist()
- self._broadcast = None
-
def __call__(self, *cols):
sc = SparkContext._active_spark_context
jc = self._judf.apply(_to_seq(sc, cols, _to_java_column))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org