You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by ni...@apache.org on 2019/12/25 11:49:16 UTC
[kylin] branch document updated: Add KYLIN-4143,4098,4175,4178 doc
This is an automated email from the ASF dual-hosted git repository.
nic pushed a commit to branch document
in repository https://gitbox.apache.org/repos/asf/kylin.git
The following commit(s) were added to refs/heads/document by this push:
new abd442e Add KYLIN-4143,4098,4175,4178 doc
abd442e is described below
commit abd442e535dbda23dcb1035aeba1c562d6278175
Author: yaqian.zhang <59...@qq.com>
AuthorDate: Wed Dec 25 15:37:56 2019 +0800
Add KYLIN-4143,4098,4175,4178 doc
---
website/_docs30/howto/howto_use_restapi.cn.md | 5 +++++
website/_docs30/howto/howto_use_restapi.md | 5 +++++
website/_docs30/install/configuration.cn.md | 30 +++++++++++++++++++++++++++
website/_docs30/install/configuration.md | 29 ++++++++++++++++++++++++++
4 files changed, 69 insertions(+)
diff --git a/website/_docs30/howto/howto_use_restapi.cn.md b/website/_docs30/howto/howto_use_restapi.cn.md
index 1361b2f..fb50e15 100644
--- a/website/_docs30/howto/howto_use_restapi.cn.md
+++ b/website/_docs30/howto/howto_use_restapi.cn.md
@@ -23,6 +23,7 @@ This page lists the major RESTful APIs provided by Kylin.
* [Disable cube](#disable-cube)
* [Purge cube](#purge-cube)
* [Delete segment](#delete-segment)
+ * [Auto-Merge segment](#auto-merge-segment)
* MODEL
* [Create model](#create-model)
* [Get modelDescData](#get-modeldescdata)
@@ -842,6 +843,10 @@ curl -X PUT -H "Authorization: Basic XXXXXXXXX" -H 'Content-Type: application/js
## Delete Segment
`DELETE /kylin/api/cubes/{cubeName}/segs/{segmentName}`
+
+## Auto-merge Segment
+`PUT /kylin/api/cubes/{cubeName}/automerge`
+
***
## Create Model
diff --git a/website/_docs30/howto/howto_use_restapi.md b/website/_docs30/howto/howto_use_restapi.md
index 8bb46b3..31c3fec 100644
--- a/website/_docs30/howto/howto_use_restapi.md
+++ b/website/_docs30/howto/howto_use_restapi.md
@@ -23,6 +23,7 @@ This page lists the major RESTful APIs provided by Kylin.
* [Disable cube](#disable-cube)
* [Purge cube](#purge-cube)
* [Delete segment](#delete-segment)
+ * [Auto-Merge segment](#auto-merge-segment)
* MODEL
* [Create model](#create-model)
* [Get modelDescData](#get-modeldescdata)
@@ -842,6 +843,10 @@ curl -X PUT -H "Authorization: Basic XXXXXXXXX" -H 'Content-Type: application/js
## Delete Segment
`DELETE /kylin/api/cubes/{cubeName}/segs/{segmentName}`
+
+## Auto-merge Segment
+`PUT /kylin/api/cubes/{cubeName}/automerge`
+
***
## Create Model
diff --git a/website/_docs30/install/configuration.cn.md b/website/_docs30/install/configuration.cn.md
index 8dc8b0d..d81609e 100644
--- a/website/_docs30/install/configuration.cn.md
+++ b/website/_docs30/install/configuration.cn.md
@@ -44,6 +44,9 @@ permalink: /cn/docs30/install/configuration.html
- [启用邮件通知](#email-notification)
- [启用 Cube Planner](#cube-planner)
- [HBase 存储](#hbase-config)
+ - [备用 Hbase 存储](#secondary-hbase)
+ - [任务调度安全模式](#safe-mode)
+ - [任务输出](#job-output)
- [启用压缩](#compress-config)
- [实时 OLAP](#realtime-olap)
- [清理存储配置](#storage-clean-up-configuration)
@@ -516,6 +519,33 @@ Cube 构建默认在 **Extract Fact Table Distinct Column** 这一步为每一
+### 备用 Hbase 存储 {#secondary-hbase}
+
+Kylin支持用户配置备用Hbase，这样在集群迁移时，Kylin仍然可以从旧集群中查询到构建好的Cube数据。
+
+- `kylin.secondary.storage.url`: 指定备用Hbase的集群地址以及metadata url. 例如 *kylin.secondary.storage.url=hostname:kylin_metadata@hbase*.
+- `hbase.zookeeper.quorum`: 指定备用Hbase所在集群的Zookeeper信息. 例如 *hbase.zookeeper.quorum=hostname:11000,zookeeper.znode.parent=/hbase/*, 如果还有其他的配置项，可以以<key> = <value>的形式添加在后面.
+
+
+
+### 任务调度安全模式 {#safe-mode}
+
+为了保证安全的对Hbase集群作出更改，用户可以为Kylin开启安全模式。在安全模式下，未创建Htable的任务将不会被调度，其他任务则会继续执行至结束。所有可以执行的任务完成后，用户可以更改配置到一个新的Hbase集群，然后再调度执行其他任务。
+
+- `kylin.job.scheduler.safemode`: 是否开启安全模式. 默认值为FALSE.
+- `kylin.job.scheduler.safemode.runnable-projects`: 不受安全模式影响的Project. 例如 *kylin.job.scheduler.safemode.runnable-projects=learn_kylin*.
+
+
+
+### 任务输出 {#job-output}
+
+为了避免job output的内容太多，用户可以设置output的最大长度。
+
+- `kylin.job.execute-output.max-size`: Job output的最大长度. 默认值为10484760.
+- `kylin.engine.spark.output.max-size`: Spark job output的最大长度. 默认值为10484760.
+
+
+
### 启用压缩 {#compress-config}
Kylin 在默认状态下不会启用压缩，不支持的压缩算法会阻碍 Kylin 的构建任务，但是一个合适的压缩算法可以减少存储开销和网络开销，提高整体系统运行效率。
diff --git a/website/_docs30/install/configuration.md b/website/_docs30/install/configuration.md
index d153540..5a7f398 100644
--- a/website/_docs30/install/configuration.md
+++ b/website/_docs30/install/configuration.md
@@ -43,6 +43,9 @@ permalink: /docs30/install/configuration.html
- [Enable Email Notification](#email-notification)
- [Enable Cube Planner](#cube-planner)
- [HBase Storage](#hbase-config)
+ - [Secondary Hbase Storage](#secondary-hbase)
+ - [Job Scheduler Safe Mode](#safe-mode)
+ - [Job Output](#job-output)
- [Enable Compression](#compress-config)
- [Real-time OLAP](#realtime-olap)
- [Storage Clean up Configuration](#storage-clean-up-configuration)
@@ -513,6 +516,32 @@ Both Kylin and HBase use compression when writing to disk, so Kylin will multipl
- `kylin.storage.hbase.scan-cache-rows`: specifies the number of scan cache lines. The default value is 1024.
+### Secondary Hbase Storage {#secondary-hbase}
+
+Kylin support secondary hbase storage and made the kylin cluster can query cube data from the old hbase cluster during the cluster migration.
+
+- `kylin.secondary.storage.url`: specifies the secondary hbase cluster and metadata path. Such as *kylin.secondary.storage.url=hostname:kylin_metadata@hbase*.
+- `hbase.zookeeper.quorum`: specifies the zookeeper information where secondary hbase cluster located. Such as *hbase.zookeeper.quorum=hostname:11000,zookeeper.znode.parent=/hbase/*, if there are other parameters, they can be added in the form of <key> = <value>.
+
+
+
+### Job Scheduler Safe Mode {#safe-mode}
+
+In order to change HBase cluster safely, user can turn on the safe mode for kylin. In safe mode, the job have not create htable cannot be scheduled, and the other jobs can run continuous. After all running job finished, user can change the cluster config to the new one, and rest of job can be scheduled again.
+
+- `kylin.job.scheduler.safemode`: Whether to turn on safe mode. The default value is FALSE.
+- `kylin.job.scheduler.safemode.runnable-projects`: Projects not affected by safe mode. Such as *kylin.job.scheduler.safemode.runnable-projects=learn_kylin*.
+
+
+
+### Job Output {#job-output}
+
+In order to avoid job output content is too large, user could set the max length of output.
+
+- `kylin.job.execute-output.max-size`: The max length of job output. The default value is 10484760.
+- `kylin.engine.spark.output.max-size`: The max length of spark job output. The default value is 10484760.
+
+
### Enable Compression {#compress-config}