[GitHub] [spark] cloud-fan commented on pull request #31643: [SPARK-34534] Fix blockIds order when use FetchShuffleBlocks to fetch blocks

[GitBox <gi...@apache.org>]

cloud-fan commented on pull request #31643:
URL: https://github.com/apache/spark/pull/31643#issuecomment-787701759


   Seems like there are two proposals:
   1. regenerate the block ids when building the `FetchShuffleBlocks` request.
   2. leverage the fact the block ids are ordered already.
   
   I think the goal is the same: keep block ids and the fetch request in sync. And these 2 proposals have no actual difference if the block ids are indeed ordered (you will regenerate the same ordered block ids).
   
   I'm in favor of 2 because it ensures the things specified in the doc of `createFetchShuffleBlocksMsg`. I think it's better to not only keep block ids and the fetch request in sync, but also ensure the block ids are ordered, to eliminate randomness.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org