You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Jacek Laskowski <ja...@japila.pl> on 2018/04/23 19:57:16 UTC

ShuffledHashJoin's selection criteria

Hi,

I've been reviewing the code of JoinSelection for ShuffledHashJoin and
can't understand how !RowOrdering.isOrderable(leftKeys) can be ever met for
the second case (copying the entire code for a quick look):

         if !conf.preferSortMergeJoin && canBuildRight(joinType) &&
canBuildLocalHashMap(right)
           && muchSmaller(right, left) ||
           !RowOrdering.isOrderable(leftKeys) =>

         if !conf.preferSortMergeJoin && canBuildLeft(joinType) &&
canBuildLocalHashMap(left)
           && muchSmaller(left, right) ||
           !RowOrdering.isOrderable(leftKeys) =>

How is !RowOrdering.isOrderable(leftKeys) possible in the second case? I
must be missing something...again :( Please help.

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski