[jira] [Updated] (SPARK-28217) Allow a pluggable statistics plan visitor for a logical plan.

["Terry Kim (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-28217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Terry Kim updated SPARK-28217:
------------------------------
    Summary: Allow a pluggable statistics plan visitor for a logical plan.  (was: Allow a custom statistics logical plan visitor to be plugged in.)

> Allow a pluggable statistics plan visitor for a logical plan.
> -------------------------------------------------------------
>
>                 Key: SPARK-28217
>                 URL: https://issues.apache.org/jira/browse/SPARK-28217
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Terry Kim
>            Priority: Major
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Spark currently has two built-in statistics plan visitor: SizeInBytesOnlyStatsPlanVisitor and BasicStatsPlanVisitor. However, this is a bit limited since there is no way to plug in a custom plan visitor - from which a custom query optimizer can benefit from.
> We can provide a Spark conf that the user can specify to override the built-in plan visitor:
> {code:scala}
> // First create your custom stat plan visitor.
> class MyStatsPlanVisitor extends LogicalPlanVisitor[Statistics] {
>   // Implement LogicalPlanVisitor[Statistics] trait
> }
> // Set the visitor via Spark conf.
> spark.conf.set("spark.sql.catalyst.statsPlanVisitorClass", "MyStatsPlanVisitor")
> // Now, stat() on a LogicalPlan object will use MyStatsPlanVisitor as a stat plan visitor.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org