You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/08/09 01:54:00 UTC
[jira] [Commented] (SPARK-36454) Not push down partition filter to
ORCScan for DSv2
[ https://issues.apache.org/jira/browse/SPARK-36454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17395683#comment-17395683 ]
Apache Spark commented on SPARK-36454:
--------------------------------------
User 'huaxingao' has created a pull request for this issue:
https://github.com/apache/spark/pull/33680
> Not push down partition filter to ORCScan for DSv2
> --------------------------------------------------
>
> Key: SPARK-36454
> URL: https://issues.apache.org/jira/browse/SPARK-36454
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Huaxin Gao
> Priority: Minor
>
> Seems to me that partition filter is only used for partition pruning and shouldn't be pushed down to ORCScan. We don't push down partition filter to ORCScan in DSv1, and we don't push down partition filter for parquet in both DSv1 and DSv2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org