You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Rajesh Tadi (Jira)" <ji...@apache.org> on 2020/05/06 09:53:00 UTC
[jira] [Comment Edited] (SPARK-31648) Filtering is supported only
on partition keys of type string Issue
[ https://issues.apache.org/jira/browse/SPARK-31648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100629#comment-17100629 ]
Rajesh Tadi edited comment on SPARK-31648 at 5/6/20, 9:52 AM:
--------------------------------------------------------------
[~angerszhuuu] Below is the SQL I have used.
select * from testdb.partbuck_test where country_cd='India';
My table structure will look similar as below.
Schema:
||col_name||data_type||comment||
|ID|bigint|null|
|NAME|string|null|
|COUNTRY_CD|string|null|
|# Partition Information| | |
|# col_name|data_type|comment|
|COUNTRY_CD|string|null|
was (Author: rajesh tadi):
[~angerszhuuu] Below is the SQL I have used.
select * from testdb.partbuck_test where country_cd='India';
My table structure will look similar as below.
Schema:
+----------------------------+------------------------------------------------------------------------------+----------+
|col_name |data_type |comment|
+----------------------------+------------------------------------------------------------------------------+----------+
|ID |bigint |null |
|NAME |string |null |
|............................. |........... |null |
|............................. |........... |null |
|............................. |........... |null |
|COUNTRY_CD |string |null |
|# Partition Information | | |
|# col_name |data_type |comment|
|COUNTRY_CD |string |null |
+----------------------------+------------------------------------------------------------------------------+----------+
> Filtering is supported only on partition keys of type string Issue
> ------------------------------------------------------------------
>
> Key: SPARK-31648
> URL: https://issues.apache.org/jira/browse/SPARK-31648
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.4
> Reporter: Rajesh Tadi
> Priority: Major
> Attachments: Spark Bug.txt
>
>
> When I submit a SQL with partition filter I see the below error. I tried setting Spark Configuration spark.sql.hive.manageFilesourcePartitions to false but I still see the same issue.
> java.lang.RuntimeException: Caught Hive MetaException attempting to get partition metadata by filter from Hive.
> java.lang.reflect.InvocationTargetException: org.apache.hadoop.hive.metastore.api.MetaException: Filtering is supported only on partition keys of type string
> org.apache.hadoop.hive.metastore.api.MetaException: Filtering is supported only on partition keys of type string
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org