You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/09/13 05:36:00 UTC

[jira] [Commented] (SPARK-36735) Adjust overhead of cached relation for DPP

    [ https://issues.apache.org/jira/browse/SPARK-36735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413905#comment-17413905 ] 

Apache Spark commented on SPARK-36735:
--------------------------------------

User 'viirya' has created a pull request for this issue:
https://github.com/apache/spark/pull/33975

> Adjust overhead of cached relation for DPP
> ------------------------------------------
>
>                 Key: SPARK-36735
>                 URL: https://issues.apache.org/jira/browse/SPARK-36735
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.3.0
>            Reporter: L. C. Hsieh
>            Priority: Major
>
> Currently we calculate if there is benefit of pruning with DPP by simply summing up the size of all scan relations as the overhead. However, for cached relations, the overhead should be different than a non-cached relation. This proposes to use adjusted overhead for cached relation with DPP.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org