You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2014/12/02 13:31:13 UTC
[jira] [Commented] (SPARK-4693) PruningPredicates may be wrong if
predicates contains an empty AttributeSet() references
[ https://issues.apache.org/jira/browse/SPARK-4693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231410#comment-14231410 ]
Apache Spark commented on SPARK-4693:
-------------------------------------
User 'YanTangZhai' has created a pull request for this issue:
https://github.com/apache/spark/pull/3556
> PruningPredicates may be wrong if predicates contains an empty AttributeSet() references
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-4693
> URL: https://issues.apache.org/jira/browse/SPARK-4693
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: YanTang Zhai
> Priority: Minor
>
> The sql "select * from spark_test::for_test where abs(20141202) is not null" has predicates=List(IS NOT NULL HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)) and
> partitionKeyIds=AttributeSet(). PruningPredicates is List(IS NOT NULL HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)). Then the exception "java.lang.IllegalArgumentException: requirement failed: Partition pruning predicates only supported for partitioned tables." is thrown.
> The sql "select * from spark_test::for_test_partitioned_table where abs(20141202) is not null and type_id=11 and platform = 3" with partitioned key insert_date has predicates=List(IS NOT NULL HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202), (type_id#12 = 11), (platform#8 = 3)) and partitionKeyIds=AttributeSet(insert_date#24). PruningPredicates is List(IS NOT NULL HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFAbs(20141202)).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org