You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2022/11/10 07:36:00 UTC
[jira] [Updated] (HIVE-26667) Incompatible expression deserialization against latest HMS
[ https://issues.apache.org/jira/browse/HIVE-26667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihua Deng updated HIVE-26667:
-------------------------------
Description:
When an old Metastore client requests listPartitionsByExpr against the lastest HMS, an exception would be thrown:
{noformat}
MetaException(message:Unable to find class:
)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_by_expr(ThriftHiveMetastore.java:3273)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_by_expr(ThriftHiveMetastore.java:3260)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByExpr(HiveMetaStoreClient.java:1488){noformat}
This was caused by a gap between old client and server on (de)serializing the expression. Old client doesn't stream the expression’s class type into bytes, while HMS reads the class type from serialized bytes first, which makes the trouble. Other APIs that need to (de)serialize expression may be suffered as well.
was:
When an old Hive Metastore client issues listPartitionsByExpr against the lastest HMS, an exception would be thrown:
{noformat}
MetaException(message:Unable to find class:
)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result.read(ThriftHiveMetastore.java)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_by_expr(ThriftHiveMetastore.java:3273)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_by_expr(ThriftHiveMetastore.java:3260)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByExpr(HiveMetaStoreClient.java:1488){noformat}
This was caused by a gap between old client and server on (de)serializing the expression. In old client, we don’t stream the expression’s class type into bytes, while the server should read the class type from serialized bytes firstly, which makes the trouble. Other APIs that need to (de)serialize expression may be suffered as well.
> Incompatible expression deserialization against latest HMS
> ----------------------------------------------------------
>
> Key: HIVE-26667
> URL: https://issues.apache.org/jira/browse/HIVE-26667
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Affects Versions: 4.0.0-alpha-1
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When an old Metastore client requests listPartitionsByExpr against the lastest HMS, an exception would be thrown:
>
> {noformat}
> MetaException(message:Unable to find class:
> )
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result$get_partitions_by_expr_resultStandardScheme.read(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partitions_by_expr_result.read(ThriftHiveMetastore.java)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_by_expr(ThriftHiveMetastore.java:3273)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_by_expr(ThriftHiveMetastore.java:3260)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByExpr(HiveMetaStoreClient.java:1488){noformat}
> This was caused by a gap between old client and server on (de)serializing the expression. Old client doesn't stream the expression’s class type into bytes, while HMS reads the class type from serialized bytes first, which makes the trouble. Other APIs that need to (de)serialize expression may be suffered as well.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)