[GitHub] HyukjinKwon commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1
URL: https://github.com/apache/spark/pull/20691#discussion_r249083190
 
 

 ##########
 File path: python/pyspark/serializers.py
 ##########
 @@ -65,7 +65,7 @@
     from itertools import izip as zip, imap as map
 else:
     import pickle
-    protocol = 3
+    protocol = min(pickle.HIGHEST_PROTOCOL, 4)
 
 Review comment:
   I actually think we can always use `HIGHEST_PROTOCOL` because PySpark's going to throw an exception for mismatched Python versions anyway. Cloudpickle also does that. Let's match it.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org