You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/05/19 01:37:00 UTC
[jira] [Commented] (IMPALA-10613) Expose table and partition
metadata over HMS API
[ https://issues.apache.org/jira/browse/IMPALA-10613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347241#comment-17347241 ]
ASF subversion and git services commented on IMPALA-10613:
----------------------------------------------------------
Commit 5c85bf5c54fb1e0dffbe01b3f70c289de3700a66 in impala's branch refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5c85bf5 ]
Revert "Revert "IMPALA-10613: Standup HMS thrift server in Catalog""
This reverts commit 829d1a6ab4643b07877fb410971b67f1b1d1b045.
Additionally, this patch has couple of addendums which are related
to the original change:
1. Bug fix the original reverted commit which uses
isSetGetFileMetadata instead of isGetFileMetadata
(see https://gerrit.cloudera.org/#/c/17330/)
2. Fix for intermittent failures on CatalogHmsFileMetadataTest
due to the limitation of the catalogd's HMS client requirement
of need to set "hive.metastore.execute.setugi" to false.
Change-Id: Icbe93f3ae4efd585d4b0092a9ac7081b0b2c1c44
Reviewed-on: http://gerrit.cloudera.org:8080/17429
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Aman Sinha <am...@cloudera.com>
> Expose table and partition metadata over HMS API
> ------------------------------------------------
>
> Key: IMPALA-10613
> URL: https://issues.apache.org/jira/browse/IMPALA-10613
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
> Fix For: Impala 4.0
>
>
> Catalogd caches the table and partition metadata. If an external FE needs to be supported to query using the Impala, it would need to get this metadata from catalogd to compile the query and generate the plan. While a subset of the metadata which is cached in catalogd, is sourced from Hive metastore, it also caches file metadata which is needed by the Impala backend to create the Impala plan. It would be good to expose the table and partition metadata cached in catalogd over HMS API so that any Hive metastore client (e.g spark, hive) can potentially use this metadata to create a plan. This JIRA tracks the work needed to expose this information over catalogd.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org