[GitHub] [spark] HyukjinKwon opened a new pull request #24519: [SPARK-27612][PYTHON] Use Python's default protocol instead of highest protocol

[GitBox <gi...@apache.org>]

HyukjinKwon opened a new pull request #24519: [SPARK-27612][PYTHON] Use Python's default protocol instead of highest protocol
URL: https://github.com/apache/spark/pull/24519
 
 
   ## What changes were proposed in this pull request?
   
   This PR partially reverts https://github.com/apache/spark/pull/20691
   
   After we changed the Python protocol to highest ones, seems like it introduced a correctness bug. This potentially affects all Python related code paths.
   
   I suspect a bug related to Pryolite (maybe opcodes `MEMOIZE`, `FRAME` and/or our `RowPickler`). I would like to stick to default protocol for now and investigate the issue separately.
   
   I will separately investigate later to bring highest protocol back.
   
   ## How was this patch tested?
   
   Unittest was added.
   
   ```bash
   ./run-tests --python-executables=python3.7 --testname "pyspark.sql.tests.test_serde SerdeTests.test_int_array_serialization"
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org