[GitHub] [spark] Ngone51 commented on pull request #39918: [SPARK-42366][SHUFFLE] Log shuffle data corruption diagnose cause

["Ngone51 (via GitHub)" <gi...@apache.org>]

Ngone51 commented on PR #39918:
URL: https://github.com/apache/spark/pull/39918#issuecomment-1420450697

   I doubt we use the DEBUG level in this case. The corruption cause here can only be either the disk issue or the network issue right now. And both of them could be temporary (problematic disk could be persistent but spark doesn't guarantee writing files on the same disk partition each time) or difficult to reproduce. So I'm afraid using the DEBUG level could miss the cause easily in the first place.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org