[GitHub] [spark] jerryshao opened a new pull request #25552: [SPARK-28849][CORE] Add number of transferTo calls to avoid infinite loop in some occasional cases

[GitBox <gi...@apache.org>]

jerryshao opened a new pull request #25552: [SPARK-28849][CORE] Add number of transferTo calls to avoid infinite loop in some occasional cases
URL: https://github.com/apache/spark/pull/25552
 
 
   This patch propose a way to detect the infinite loop issue in `transferTo` and make it fail fast. The detailed issue is shown in https://issues.apache.org/jira/browse/SPARK-28849. 
   
   Here propose a new undocumented configuration and a way to track the number of `transferTo` calls, if the number is larger than the specified value, it will jump out of the loop and throw an exception. 
   
   Typically this will not be happened, and user don't need to set this configuration explicitly, in our environment it happens occasionally and cause the task hung infinitely.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org