[GitHub] [spark] Ngone51 commented on pull request #34018: [SPARK-36772] FinalizeShuffleMerge fails with an exception due to attempt id not matching

[GitBox <gi...@apache.org>]

Ngone51 commented on pull request #34018:
URL: https://github.com/apache/spark/pull/34018#issuecomment-921955984


   I'm thinking of the potential compatibility issue in the future by using the `int` type of `appAttemptId` inside `BlockStoreClinet`. Using `int` might impede the future extension of CMs (e.g., K8s, Standalone, or something new)  if they want the arbitrary attempt id to include more info. So I think it's good to keep it as a string. However, since push-based shuffle would require a comparable number value for its functionality,  we'd still require the attempt id to at least includes one comparable number value, e.g., (xxx_attempt_id_12). Then, in the case of push-based shuffle, we can just extract the number value into `int` number, which would be compatible with the current push-based shuffle protocols. 
   @mridulm @zhouyejoe WDYT?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org