[jira] [Assigned] (SPARK-16898) Adds argument type information for typed logical plan like MapElements, TypedFilter, and AppendColumn

["Apache Spark (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-16898:
------------------------------------

    Assignee: Apache Spark

> Adds argument type information for typed logical plan like MapElements, TypedFilter, and AppendColumn
> -----------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-16898
>                 URL: https://issues.apache.org/jira/browse/SPARK-16898
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Sean Zhong
>            Assignee: Apache Spark
>            Priority: Minor
>
> Typed logical plan like MapElements, TypedFilter, and AppendColumn contains a closure field: {{func: (T) => Boolean}}. For example class TypedFilter's signature is:
> {code}
> case class TypedFilter(
>     func: AnyRef,
>     deserializer: Expression,
>     child: LogicalPlan) extends UnaryNode
> {code} 
> From the above class signature, we cannot easily find:
> 1. What is the input argument's type of the closure {{func}}? How do we know which apply method to pick if there are multiple overloaded apply methods?
> 2. What is the input argument's schema? 
> With this info, it is easier for us to define some custom optimizer rule to translate these typed logical plan to more efficient implementation, like the closure optimization idea in SPARK-14083.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org