You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by zy...@apache.org on 2023/06/07 12:05:20 UTC
[doris] branch master updated: [doc](multi-catalog)huawei mrs event sync doc (#20466)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 809bb46518 [doc](multi-catalog)huawei mrs event sync doc (#20466)
809bb46518 is described below
commit 809bb4651895c3d951f4b98fc7366aa426f169c4
Author: zhangdong <49...@qq.com>
AuthorDate: Wed Jun 7 20:05:12 2023 +0800
[doc](multi-catalog)huawei mrs event sync doc (#20466)
---
.../docs/lakehouse/multi-catalog/multi-catalog.md | 29 +++++++++++++++++++++-
.../docs/lakehouse/multi-catalog/multi-catalog.md | 29 +++++++++++++++++++++-
2 files changed, 56 insertions(+), 2 deletions(-)
diff --git a/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md b/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md
index d878de39ee..9f266f9fed 100644
--- a/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md
+++ b/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md
@@ -347,7 +347,7 @@ The automatic update feature involves the following parameters in fe.conf:
2. `hms_events_polling_interval_ms`: This specifies the interval between two readings, which is set to 10000 by default. (Unit: millisecond)
3. `hms_events_batch_size_per_rpc`: This specifies the maximum number of events that are read at a time, which is set to 500 by default.
-To enable automatic update, you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2:
+To enable automatic update(Excluding Huawei MRS), you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2:
```
<property>
@@ -365,6 +365,33 @@ To enable automatic update, you need to modify the hive-site.xml of HMS and then
```
+Huawei's MRS needs to change hivemetastore-site.xml and restart HMS and HiveServer2:
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>org.apache.hive.hcatalog.listener.DbNotificationListener</value>
+</property>
+```
+
+Note: Value is appended with commas separated from the original value, not overwritten.For example, the default configuration for MRS 3.1.0 is
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener</value>
+</property>
+```
+
+We need to change to
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener</value>
+</property>
+```
+
> Note: To enable automatic update, whether for existing Catalogs or newly created Catalogs, all you need is to set `enable_hms_events_incremental_sync` to `true`, and then restart the FE node. You don't need to manually update the metadata before or after the restart.
<version since="dev">
diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md b/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md
index dd8dfd70b9..6a528d2302 100644
--- a/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md
+++ b/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md
@@ -347,7 +347,7 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参
2. `hms_events_polling_interval_ms`: 读取 event 的间隔时间，默认值为 10000，单位：毫秒。
3. `hms_events_batch_size_per_rpc`: 每次读取 event 的最大数量，默认值为 500。
-如果想使用该特性，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2：
+如果想使用该特性(华为MRS除外)，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2：
```
<property>
@@ -365,6 +365,33 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参
```
+华为的MRS需要更改hivemetastore-site.xml 并重启HMS和HiveServer2：
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>org.apache.hive.hcatalog.listener.DbNotificationListener</value>
+</property>
+```
+
+注意：value是在原有值的基础上以逗号分隔追加，而不是覆盖。例如MRS 3.1.0默认配置为
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener</value>
+</property>
+```
+
+我们需要改为
+
+```
+<property>
+ <name>metastore.transactional.event.listeners</name>
+ <value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener</value>
+</property>
+```
+
> 使用建议： 无论是之前已经创建好的catalog现在想改为自动刷新，还是新创建的 catalog，都只需要把 `enable_hms_events_incremental_sync` 设置为true，重启fe节点，无需重启之前或之后再手动刷新元数据。
<version since="dev">
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org