[jira] [Created] (SPARK-16898) Adds argument type information for typed logical plan likMapElements, TypedFilter, and AppendColumn

["Sean Zhong (JIRA)" <ji...@apache.org>]

Sean Zhong created SPARK-16898:
----------------------------------

             Summary: Adds argument type information for typed logical plan likMapElements, TypedFilter, and AppendColumn
                 Key: SPARK-16898
                 URL: https://issues.apache.org/jira/browse/SPARK-16898
             Project: Spark
          Issue Type: Bug
            Reporter: Sean Zhong
            Priority: Minor


Typed logical plan like MapElements, TypedFilter, and AppendColumn contains a closure field: {{func: (T) => Boolean}}. For example class TypedFilter's signature is:

{code}
case class TypedFilter(
    func: AnyRef,
    deserializer: Expression,
    child: LogicalPlan) extends UnaryNode
{code} 

From the above class signature, we cannot easily find:
1. What is the input argument's type of the closure {{func}}? How do we know which apply method to pick if there are multiple overloaded apply methods?
2. What is the input argument's schema? 

With this info, it is easier for us to define some custom optimizer rule to translate these typed logical plan to more efficient implementation, like the closure optimization idea in SPARK-14083.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org