You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Joe <jo...@net2020.org> on 2018/11/07 16:25:16 UTC

How does shuffle operation work in Spark?

Hello,
I'm looking for a detailed description of the shuffle operation in 
Spark, something that would explain what are the criteria for assigning 
blocks to nodes, how many go where, what happens when there are memory 
constraints, etc.
If anyone knows of such a document I'd appreciate a link (or a detailed 
answer).
Thanks a lot,

Joe


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org