You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2015/07/20 02:47:04 UTC

[jira] [Commented] (SPARK-9083) If order by clause has non-deterministic expressions, we should add a project to materialize results of these expressions

    [ https://issues.apache.org/jira/browse/SPARK-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633006#comment-14633006 ] 

Reynold Xin commented on SPARK-9083:
------------------------------------

This should also be fixed for cluster by, sort by, etc. 

> If order by clause has non-deterministic expressions, we should add a project to materialize results of these expressions 
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-9083
>                 URL: https://issues.apache.org/jira/browse/SPARK-9083
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Yin Huai
>            Assignee: Wenchen Fan
>
> When a order by clause has a non-deterministic expression, we actually evaluate it twice, once in the exchange operator when we try to figure out the range partitioner's boundaries and once in the sort operator. We should use a project to materialize the result first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org