You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by sh...@apache.org on 2019/03/22 02:53:49 UTC
[kylin] branch document updated: KYLIN-3890 Add doc about usage of
./bin/metadata.sh
This is an automated email from the ASF dual-hosted git repository.
shaofengshi pushed a commit to branch document
in repository https://gitbox.apache.org/repos/asf/kylin.git
The following commit(s) were added to refs/heads/document by this push:
new e209263 KYLIN-3890 Add doc about usage of ./bin/metadata.sh
e209263 is described below
commit e2092637d76a869f81bcd9cda7aea0debbb24392
Author: yuzhang <sh...@163.com>
AuthorDate: Thu Mar 21 11:23:39 2019 +0800
KYLIN-3890 Add doc about usage of ./bin/metadata.sh
---
website/_docs/howto/howto_backup_metadata.cn.md | 45 +++++++++++++++++++++++++
website/_docs/howto/howto_backup_metadata.md | 45 +++++++++++++++++++++++++
2 files changed, 90 insertions(+)
diff --git a/website/_docs/howto/howto_backup_metadata.cn.md b/website/_docs/howto/howto_backup_metadata.cn.md
index 56db4c8..387efb6 100644
--- a/website/_docs/howto/howto_backup_metadata.cn.md
+++ b/website/_docs/howto/howto_backup_metadata.cn.md
@@ -14,6 +14,43 @@ kylin.metadata.url=kylin_metadata@hbase
这表明元数据会被保存在一个叫作 “kylin_metadata”的htable 里。你可以在 hbase shell 里 scan 该 htbale 来获取它。
+## 元数据路径
+
+Kylin使用`resource root path + resource name + resource suffix`作为key值(HBase中的rowkey)来存储元数据。你可以参考如下表格使用`./bin/metastore.sh`命令。
+
+| Resource root path | resource name | resource suffix
+| --------------------| :---------------------| :--------------|
+| /cube | /cube name | .json |
+| /cube_desc | /cube name | .json |
+| /cube_statistics | /cube name/uuid | .seq |
+| /model_desc | /model name | .json |
+| /dict | /DATABASE.TABLE/COLUMN/uuid | .dict |
+| /project | /project name | .json |
+| /table_snapshot | /DATABASE.TABLE/uuid | .snapshot |
+| /table | /DATABASE.TABLE--project name | .json |
+| /table_exd | /DATABASE.TABLE--project name | .json |
+| /execute | /job id | |
+| /execute_out | /job id-step index | |
+| /kafaka | /DATABASE.TABLE | .json |
+| /streaming | /DATABASE.TABLE | .json |
+| /user | /user name | |
+
+## 查看元数据
+
+Kylin以二进制字节的格式将元数据存储在HBase中，如果你想要查看一些元数据，可以运行：
+
+{% highlight Groff markup %}
+./bin/metastore.sh list /path/to/store/metadata
+{% endhighlight %}
+
+列出存储在指定路径下的所有实体元数据。然后运行：
+
+{% highlight Groff markup %}
+./bin/metastore.sh cat /path/to/store/entity/metadata.
+{% endhighlight %}
+
+查看某个实体的元数据。
+
## 使用二进制包来备份 metadata
有时你需要将 Kylin 的 metadata store 从 hbase 备份到磁盘文件系统。在这种情况下，假设你在部署 Kylin 的 hadoop 命令行（或沙盒）里，你可以到KYLIN_HOME并运行：
@@ -24,6 +61,14 @@ kylin.metadata.url=kylin_metadata@hbase
来将你的元数据导出到本地目录，这个目录在KYLIN_HOME/metadata_backps下，它的命名规则使用了当前时间作为参数：KYLIN_HOME/meta_backups/meta_year_month_day_hour_minute_second 。
+此外, 你可以运行:
+
+{% highlight Groff markup %}
+./bin/metastore.sh fetch /path/to/store/metadata
+{% endhighlight %}
+
+有选择地导出元数据. 举个栗子, 运行 `./bin/metastore.sh fetch /cube_desc/` 获取所有的cube desc元数据, 或者运行 `./bin/metastore.sh fetch /cube_desc/kylin_sales_cube.json` 导出单个cube desc的元数据。
+
## 使用二进制包来恢复 metadata
万一你发现你的元数据被搞得一团糟，想要恢复先前的备份：
diff --git a/website/_docs/howto/howto_backup_metadata.md b/website/_docs/howto/howto_backup_metadata.md
index deaaa70..0df0aab 100644
--- a/website/_docs/howto/howto_backup_metadata.md
+++ b/website/_docs/howto/howto_backup_metadata.md
@@ -14,6 +14,43 @@ kylin.metadata.url=kylin_metadata@hbase
This indicates that the metadata will be saved as a htable called `kylin_metadata`. You can scan the htable in hbase shell to check it out.
+## Metadata directory
+
+Kylin metastore use `resource root path + resource name + resource suffix` as key (rowkey in hbase) to store metadata. You can refer to the following table to use `./bin/metastore.sh`.
+
+| Resource root path | resource name | resource suffix
+| --------------------| :---------------------| :--------------|
+| /cube | /cube name | .json |
+| /cube_desc | /cube name | .json |
+| /cube_statistics | /cube name/uuid | .seq |
+| /model_desc | /model name | .json |
+| /dict | /DATABASE.TABLE/COLUMN/uuid | .dict |
+| /project | /project name | .json |
+| /table_snapshot | /DATABASE.TABLE/uuid | .snapshot |
+| /table | /DATABASE.TABLE--project name | .json |
+| /table_exd | /DATABASE.TABLE--project name | .json |
+| /execute | /job id | |
+| /execute_out | /job id-step index | |
+| /kafaka | /DATABASE.TABLE | .json |
+| /streaming | /DATABASE.TABLE | .json |
+| /user | /user name | |
+
+## View metadata
+
+Kylin store metadata in Byte format in HBase. If you want to view some metadata, you can run:
+
+{% highlight Groff markup %}
+./bin/metastore.sh list /path/to/store/metadata
+{% endhighlight %}
+
+to list the entity stored in specified directory, and then run:
+
+{% highlight Groff markup %}
+./bin/metastore.sh cat /path/to/store/entity/metadata.
+{% endhighlight %}
+
+to view one entity metadata.
+
## Backup metadata with binary package
Sometimes you need to backup the Kylin's metadata store from hbase to your disk file system.
@@ -25,6 +62,14 @@ In such cases, assuming you're on the hadoop CLI(or sandbox) where you deployed
to dump your metadata to your local folder a folder under KYLIN_HOME/metadata_backps, the folder is named after current time with the syntax: KYLIN_HOME/meta_backups/meta_year_month_day_hour_minute_second
+In addition, you can run:
+
+{% highlight Groff markup %}
+./bin/metastore.sh fetch /path/to/store/metadata
+{% endhighlight %}
+
+to dump metadata selectively. For example, run `./bin/metastore.sh fetch /cube_desc/` to get all cube desc metadata, or run `./bin/metastore.sh fetch /cube_desc/kylin_sales_cube.json` to get single cube desc metadata.
+
## Restore metadata with binary package
In case you find your metadata store messed up, and you want to restore to a previous backup: