[GitHub] [spark] pengbo removed a comment on issue #24389: [SPARK-27482][SQL][Web UI]: Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page

[GitBox <gi...@apache.org>]

pengbo removed a comment on issue #24389: [SPARK-27482][SQL][Web UI]: Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
URL: https://github.com/apache/spark/pull/24389#issuecomment-484111600
 
 
   > I agree with the use case, but the implementation is too hacky. We need a general approach to propagate the statistics from logical plan to physical plan.
   
   How about:
   1. Add `stats` field and getter/setter in `SparkPlanStats` (trait of `SparkPlan`)
   2. Create a `StatisticsPropagation` `SparkPlan` strategy as the first strategy after `extraPlanningStrategies` that creates a `PlanLater(loginPlan)` with `stats` copied from `LogicalPlan`. To avoid infinity loop, one variable `isStatsPropagated` in `LogicalPlan` may be needed to indicate whether the `LogicalPlan` has been applied the strategy.
   3. `SparkPlan.makeCopy` copy the `stats` as well
   
   Your feedback will be appreciated. @cloud-fan 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org