You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by jo...@apache.org on 2014/01/29 06:31:09 UTC
[2/2] git commit: Merge pull request #523 from JoshRosen/SPARK-1043
Merge pull request #523 from JoshRosen/SPARK-1043
Switch from MUTF8 to UTF8 in PySpark serializers.
This fixes SPARK-1043, a bug introduced in 0.9.0 where PySpark couldn't serialize strings > 64kB.
This fix was written by @tyro89 and @bouk in #512. This commit squashes and rebases their pull request in order to fix some merge conflicts.
Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/f8c742ce
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/f8c742ce
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/f8c742ce
Branch: refs/heads/master
Commit: f8c742ce274fbae2a9e616d4c97469b6a22069bb
Parents: 84670f2 1381fc7
Author: Josh Rosen <jo...@apache.org>
Authored: Tue Jan 28 21:30:20 2014 -0800
Committer: Josh Rosen <jo...@apache.org>
Committed: Tue Jan 28 21:30:20 2014 -0800
----------------------------------------------------------------------
.../org/apache/spark/api/python/PythonRDD.scala | 18 +++++++---
.../spark/api/python/PythonRDDSuite.scala | 35 ++++++++++++++++++++
python/pyspark/context.py | 4 +--
python/pyspark/serializers.py | 6 ++--
python/pyspark/worker.py | 8 ++---
5 files changed, 57 insertions(+), 14 deletions(-)
----------------------------------------------------------------------