You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/02 21:56:30 UTC

[GitHub] [spark] otterc commented on pull request #36411: [SPARK-39072][SHUFFLE]Fast fail the remaining push blocks if shuffle …

otterc commented on PR #36411:
URL: https://github.com/apache/spark/pull/36411#issuecomment-1115410092

   @wankunde Can you please provide more details/logs of the problem that you are trying to solve. In specific, can you provide some logs that exhibit the below
   
   > After the shuffle stage is finalized, the reduce fetch blocks RPC will be blocked if there are still many map output blocks in flight.
   
   `ShuffleBlockPusher` already uses config `maxBlocksInFlightPerAddress`. Why do we need a new configuration for this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org