[jira] [Assigned] (SPARK-59580) TreeNode construction allocates unused cache state

["Gengliang Wang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-59580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gengliang Wang reassigned SPARK-59580:
--------------------------------------

    Assignee: Ben Hollis

> TreeNode construction allocates unused cache state
> --------------------------------------------------
>
>                 Key: SPARK-59580
>                 URL: https://issues.apache.org/jira/browse/SPARK-59580
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 4.2.0
>            Reporter: Ben Hollis
>            Assignee: Ben Hollis
>            Priority: Minor
>              Labels: pull-request-available
>
> Every Catalyst {{TreeNode}} eagerly allocates cache objects and initializer closures for five properties. Most nodes never read all five properties, so expression and plan construction pays for cache state that is never used.
> This overhead compounds across large Catalyst trees. On the measured JVM, constructing an otherwise unused {{Literal}} allocates 280 bytes, of which 199 bytes are avoidable cache overhead. The same overhead affects expressions, logical plans, physical plans, and other {{TreeNode}} subclasses.
> Acceptance criteria:
>  * Constructing tree nodes does not allocate cache objects for properties that are never read.
>  * Existing {{TreeNode}} behavior and concurrency guarantees remain unchanged.
>  * A regression test detects renewed unused cache allocation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org