[jira] [Resolved] (SPARK-34906) Refactor TreeNode's children handling methods into specialized traits

["Herman van Hövell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-34906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Herman van Hövell resolved SPARK-34906.
---------------------------------------
    Fix Version/s: 3.2.0
         Assignee: Ali Afroozeh
       Resolution: Fixed

> Refactor TreeNode's children handling methods into specialized traits
> ---------------------------------------------------------------------
>
>                 Key: SPARK-34906
>                 URL: https://issues.apache.org/jira/browse/SPARK-34906
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.1
>            Reporter: Ali Afroozeh
>            Assignee: Ali Afroozeh
>            Priority: Major
>             Fix For: 3.2.0
>
>
> Spark query plan node hierarchy has specialized traits (or abstract classes) for handling nodes with fixed number of children, for example UnaryExpression, UnaryNode and UnaryExec for representing an expression, a logical plan and a physical plan with only one child, respectively. This PR refactors the TreeNode hierarchy by extracting the children handling functionality into the following traits. UnaryExpression` and other similar classes now extend the corresponding new trait:
> {{trait LeafLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
>  {{  override final def children: Seq[T] = Nil}}
>  {{}}}
> {{trait UnaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
>  {{  def child: T}}
>  {{  @transient override final lazy val children: Seq[T] = child :: Nil}}
>  {{}}}
> {{trait BinaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
>  {{  def left: T}}
>  {{  def right: T}}
>  {{  @transient override final lazy val children: Seq[T] = left :: right :: Nil}}
>  {{}}}
> {{trait TernaryLike[T <: TreeNode[T]] { self: TreeNode[T] =>}}
>  {{  def first: T}}
>  {{  def second: T}}
>  {{  def third: T}}
>  {{  @transient override final lazy val children: Seq[T] = first :: second :: third :: Nil}}
>  {{}}}
>  
> This refactoring, which is part of a bigger effort to make tree transformations in Spark more efficient, has two benefits:
>  * It moves the children handling to a single place, instead of being spread in specific subclasses, which will help the future optimizations for tree traversals.
>  * It allows to mix in these traits with some concrete node types that could not extend the previous classes. For example, expressions with one child that extend AggregateFunction cannot extend UnaryExpression as AggregateFunction defines the foldable method final while UnaryExpression defines it as non final. With the new traits, we can directly extend the concrete class from UnaryLike in these cases. Classes with more specific child handling will make tree traversal methods faster.
> In this PR we have also updated many concrete node types to extend these traits to benefit from more specific child handling.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org