[jira] [Resolved] (SPARK-3109) Sql query with OR condition should be handled above PhysicalOperation layer

["Michael Armbrust (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Armbrust resolved SPARK-3109.
-------------------------------------
    Resolution: Won't Fix

Hey Alex, I'm going to close this as I'm not sure its actually possible to do this optimization in the general case.  If I'm wrong or if you have a more specific instance where you think this optimization could work please feel free to reopen.

> Sql query with OR condition should be handled above PhysicalOperation layer
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-3109
>                 URL: https://issues.apache.org/jira/browse/SPARK-3109
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 1.0.2
>            Reporter: Alex Liu
>
> For query like 
> {code}
> select d, e  from test where a = 1 and b = 1 and c = 1 and d > 20 or  d < 0
> {code}
> Spark SQL pushes the whole query to PhysicalOperation. I haven't check how Spark SQL internal query plan works, but I think "OR" condition in the above query should be handled above physical operation. Physical operation should have the following query
> {code} select d, e from test where a = 1 and b = 1 and c  = 1 and d > 20 {code}
> OR
> {code}select d, e from test where d < 0 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org