You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by maropu <gi...@git.apache.org> on 2018/05/02 08:57:14 UTC

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

Github user maropu commented on the issue:

    https://github.com/apache/spark/pull/21109
  
    Since I feel this is a limited case, I'm not certainly sure this optimization needs to be handled in smj. For spatial or temporal use cases, is it not enough to add dummy join keys to split tasks into pieces for workaround?
    ```
    'col1a === 'col2a and 'col1dummyKey === 'col2dummyKey and ('col1b < 'col2b + 3) and ('col1b > 'col2b - 3)
    ```
    Btw, can you fix this issue by more simpler code change?  (I'm not sure this big change pays the performance gain...)



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org