You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/07/22 17:34:00 UTC
[jira] [Commented] (IMPALA-10815) Ignore events on non-default hive
catalogs
[ https://issues.apache.org/jira/browse/IMPALA-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385686#comment-17385686 ]
ASF subversion and git services commented on IMPALA-10815:
----------------------------------------------------------
Commit a7a1cb7106699c1d7f852c4d5782e16a7fdba169 in impala's branch refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a7a1cb7 ]
IMPALA-10815: Ignore events on non-default hive catalogs
Hive-3 supports a new type in metastore called catalogs. Even
though impala does not support custom catalogs, it is still possible
that some external HMS client creates objects within a non-default
catalog. This can become problematic when the objects within
the custom catalog match with the name of other objects in the default
catalog. For example, dropping a custom catalog generates a
DROP_DATABASE event on default database of that catalog. When such
event is processed, the events processor can remove the default
database.
This patch adds logic to ignore all the events which are
generated on such non-default catalog objects. The default value of
catalog is defined in the hive-site.xml of the metastore client which
is used by catalogd. If the value is not present it default to "hive".
Additionally, it also adds the code to validate that the default
catalog name defined in the hive-site.xml of the catalogd is same
as in metastore server side. If the values do not match, the
events processor does not come up.
This patch also removes some config validations which are specific
to hive-2 since we don't support hive-2 anymore in master.
Testing:
1. Added a new test which creates a custom hive catalog and events
on it. The test makes sure that such events do not affect objects in
catalogd.
Change-Id: Ided463b2a98331d3d305bbe92fbf3a5d2e197acf
Reviewed-on: http://gerrit.cloudera.org:8080/17707
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Ignore events on non-default hive catalogs
> ------------------------------------------
>
> Key: IMPALA-10815
> URL: https://issues.apache.org/jira/browse/IMPALA-10815
> Project: IMPALA
> Issue Type: Bug
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Minor
>
> Hive-3 introduces a new object called catalog which is like a namespace for database and tables. Currently, Impala does not support hive catalog. However, if there are events on such non-default catalogs the events processing applies these events on the catalogd if the database and table name matches. Until we support custom catalogs in hive we should ignore the events coming from such non-default catalog objects.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org