[GitHub] xuanyuanking edited a comment on issue #23470: [SPARK-26549][PySpark] Fix for python worker reuse take no effect for Python3

[GitBox <gi...@apache.org>]

xuanyuanking edited a comment on issue #23470: [SPARK-26549][PySpark] Fix for python worker reuse take no effect for Python3
URL: https://github.com/apache/spark/pull/23470#issuecomment-451857517
 
 
   Thanks Wenchen Liangchi and Hyukjin for your comment, the JIRA description has more detailed and the code I added before: https://issues.apache.org/jira/browse/SPARK-26549.
   ~~I think the bug here triggered by Python2 has handled the -1 value while Python3 is not.
   The root cause is different behavior and call stack between Python2 and Python3, I'm still keeping tracking this, will give more detailed log and trace stack soon, any help and advise is appreciated.~~
   
   Sorry for the mess, the bug only for `sc.parallelize(xrange(x))`, it's nothing to do with specific python version, I didn't realize that the code path difference between xrange and range in 'parallelize'... I'll change the JIRA and PR description.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org