[GitHub] [spark] jiangxb1987 commented on issue #20393: [SPARK-23207][SQL] Shuffle+Repartition on a DataFrame could lead to incorrect answers

[GitBox <gi...@apache.org>]

jiangxb1987 commented on issue #20393: [SPARK-23207][SQL] Shuffle+Repartition on a DataFrame could lead to incorrect answers
URL: https://github.com/apache/spark/pull/20393#issuecomment-518454320
 
 
   > @jiangxb1987 could you please provide a little guidance on how to run the example repro for this issue? Spark seems to fail the job entirely when the kill switch brings down the executor, and consequently triggers the need to re-execute upstream stages, which Spark seems to punt on i.e., not do.
   
   IIRC When I run this example on databricks notebook, I started a cluster with 20 workers, and set `spark.stage.maxConsecutiveAttempts` to a very big number so the failed stage keeps retry.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org