[jira] [Assigned] (SPARK-9983) Local physical operators for query execution

["Reynold Xin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin reassigned SPARK-9983:
----------------------------------

    Assignee: Reynold Xin  (was: Shixiong Zhu)

> Local physical operators for query execution
> --------------------------------------------
>
>                 Key: SPARK-9983
>                 URL: https://issues.apache.org/jira/browse/SPARK-9983
>             Project: Spark
>          Issue Type: Story
>          Components: SQL
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>
> In distributed query execution, there are two kinds of operators:
> (1) operators that exchange data between different executors or threads: examples include broadcast, shuffle.
> (2) operators that process data in a single thread: examples include project, filter, group by, etc.
> This ticket proposes clearly differentiating them and creating local operators in Spark. This leads to a lot of benefits: easier to test, easier to optimize data exchange, better design (single responsibility), and potentially even having a hyper-optimized single-node version of DataFrame.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org