[jira] [Commented] (HIVE-23886) Filter Query on External table produce no result if hive.metastore.expression.proxy set to MsckPartitionExpressionProxy

["Syed Shameerur Rahman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162485#comment-17162485 ] 

Syed Shameerur Rahman commented on HIVE-23886:
----------------------------------------------

[~Rajkumar Singh] Any specific reason for setting hive.metastore.expression.proxy to MsckPartitionExpressionProxy ?

> Filter Query on External table produce no result if hive.metastore.expression.proxy set to MsckPartitionExpressionProxy
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-23886
>                 URL: https://issues.apache.org/jira/browse/HIVE-23886
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 3.1.0
>            Reporter: Rajkumar Singh
>            Priority: Major
>
> query such as "select count(1) from tpcds_10_parquet.store_returns where sr_returned_date_sk=2452802" return row count as 0 even though partition has enough rows in it.
> upon investigation, I found that partition list  passed during the StatsUtils.getNumRows is of zero size.
> https://github.com/apache/hive/blob/ccaf783a198e142b408cb57415c4262d27b45831/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java#L438-L439
> it seems partitionlist is retrieved during PartitionPruner
> https://github.com/apache/hive/blob/36bf7f00731e3b95af3e5eeaa4ce39b375974a74/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java#L439
> Hive serialized this filter expression using Kryo before passing to HMS
> https://github.com/apache/hive/blob/36bf7f00731e3b95af3e5eeaa4ce39b375974a74/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3931
> on the server-side if the hive.metastore.expression.proxy set to MsckPartitionExpressionProxy it tries to convert this expression into the string 
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MsckPartitionExpressionProxy.java#L50
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MsckPartitionExpressionProxy.java#L56
> because of this bad filter expression hive did not retrieve any partitions, I think to make it work hive should try to deserialize it similar to PartitionExpressionForMetastore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)