You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by da...@apache.org on 2015/06/19 09:12:32 UTC
spark git commit: [SPARK-8339] [PYSPARK] integer division for python 3
Repository: spark
Updated Branches:
refs/heads/master a2016b4bc -> fdf63f124
[SPARK-8339] [PYSPARK] integer division for python 3
Itertools islice requires an integer for the stop argument. Switching to integer division here prevents a ValueError when vs is evaluated above.
davies
This is my original work, and I license it to the project.
Author: Kevin Conor <ke...@discoverybayconsulting.com>
Closes #6794 from kconor/kconor-patch-1 and squashes the following commits:
da5e700 [Kevin Conor] Integer division for batch size
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fdf63f12
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fdf63f12
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fdf63f12
Branch: refs/heads/master
Commit: fdf63f12490c674cc1877ddf7b70343c4fd6f4f1
Parents: a2016b4
Author: Kevin Conor <ke...@discoverybayconsulting.com>
Authored: Fri Jun 19 00:12:20 2015 -0700
Committer: Davies Liu <da...@databricks.com>
Committed: Fri Jun 19 00:12:20 2015 -0700
----------------------------------------------------------------------
python/pyspark/serializers.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/fdf63f12/python/pyspark/serializers.py
----------------------------------------------------------------------
diff --git a/python/pyspark/serializers.py b/python/pyspark/serializers.py
index d8cdcda..7f9d0a3 100644
--- a/python/pyspark/serializers.py
+++ b/python/pyspark/serializers.py
@@ -272,7 +272,7 @@ class AutoBatchedSerializer(BatchedSerializer):
if size < best:
batch *= 2
elif size > best * 10 and batch > 1:
- batch /= 2
+ batch //= 2
def __repr__(self):
return "AutoBatchedSerializer(%s)" % self.serializer
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org