You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by ma...@apache.org on 2016/03/17 11:30:39 UTC
[1/2] kylin git commit: KYLIN-1249 client library
Repository: kylin
Updated Branches:
refs/heads/document 62ace3d86 -> 31daab759
KYLIN-1249 client library
Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/31daab75
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/31daab75
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/31daab75
Branch: refs/heads/document
Commit: 31daab7594a1d3a7c9c0822d216b12233d0d6482
Parents: 44a830d
Author: Hongbin Ma <ma...@apache.org>
Authored: Thu Mar 17 18:30:17 2016 +0800
Committer: Hongbin Ma <ma...@apache.org>
Committed: Thu Mar 17 18:30:28 2016 +0800
----------------------------------------------------------------------
website/_data/docs-cn.yml | 1 +
website/_data/docs.yml | 1 +
website/_docs/tutorial/kylin_client_tool.cn.md | 2 --
website/_docs/tutorial/kylin_client_tool.md | 2 --
4 files changed, 2 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kylin/blob/31daab75/website/_data/docs-cn.yml
----------------------------------------------------------------------
diff --git a/website/_data/docs-cn.yml b/website/_data/docs-cn.yml
index f69fbe5..eeaca00 100644
--- a/website/_data/docs-cn.yml
+++ b/website/_data/docs-cn.yml
@@ -17,4 +17,5 @@
- tutorial/tableau_91
- tutorial/powerbi
- tutorial/odbc
+ - tutorial/kylin_client_tool
http://git-wip-us.apache.org/repos/asf/kylin/blob/31daab75/website/_data/docs.yml
----------------------------------------------------------------------
diff --git a/website/_data/docs.yml b/website/_data/docs.yml
index c557ae7..5e5de00 100644
--- a/website/_data/docs.yml
+++ b/website/_data/docs.yml
@@ -44,6 +44,7 @@
- tutorial/tableau_91
- tutorial/powerbi
- tutorial/odbc
+ - tutorial/kylin_client_tool
- title: How To
docs:
http://git-wip-us.apache.org/repos/asf/kylin/blob/31daab75/website/_docs/tutorial/kylin_client_tool.cn.md
----------------------------------------------------------------------
diff --git a/website/_docs/tutorial/kylin_client_tool.cn.md b/website/_docs/tutorial/kylin_client_tool.cn.md
index b8c6d8f..1f0bbcb 100644
--- a/website/_docs/tutorial/kylin_client_tool.cn.md
+++ b/website/_docs/tutorial/kylin_client_tool.cn.md
@@ -3,8 +3,6 @@ layout: docs-cn
title: Kylin Client Tool 使用教程
categories: 教程
permalink: /cn/docs/tutorial/kylin_client_tool.html
-version: v1.0
-since: v1.2.0
---
> Kylin-client-tool是一个用python编写的，完全基于kylin的rest api的工具。可以实现kylin的cube创建，按时build cube，job的提交、调度、查看、取消与恢复。
http://git-wip-us.apache.org/repos/asf/kylin/blob/31daab75/website/_docs/tutorial/kylin_client_tool.md
----------------------------------------------------------------------
diff --git a/website/_docs/tutorial/kylin_client_tool.md b/website/_docs/tutorial/kylin_client_tool.md
index f81a4cf..5b3ee02 100644
--- a/website/_docs/tutorial/kylin_client_tool.md
+++ b/website/_docs/tutorial/kylin_client_tool.md
@@ -3,8 +3,6 @@ layout: docs
title: Kylin Client Tool Tutorial
categories: tutorial
permalink: /docs/tutorial/kylin_client_tool.html
-version: v1.0
-since: v1.2.0
---
> Kylin-client-tool is a tool coded with python, completely based on Kylin's restful apis.
[2/2] kylin git commit: Add documents for kylin_client_tool.
Posted by ma...@apache.org.
Add documents for kylin_client_tool.
Signed-off-by: Hongbin Ma <ma...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/44a830da
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/44a830da
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/44a830da
Branch: refs/heads/document
Commit: 44a830da9d7a64c669673f4b8f283525a55ded12
Parents: 62ace3d
Author: nichunen <ni...@mininglamp.com>
Authored: Wed Feb 24 22:54:42 2016 +0800
Committer: Hongbin Ma <ma...@apache.org>
Committed: Thu Mar 17 18:30:28 2016 +0800
----------------------------------------------------------------------
website/_docs/tutorial/kylin_client_tool.cn.md | 99 +++++++++++++++++++
website/_docs/tutorial/kylin_client_tool.md | 100 ++++++++++++++++++++
2 files changed, 199 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kylin/blob/44a830da/website/_docs/tutorial/kylin_client_tool.cn.md
----------------------------------------------------------------------
diff --git a/website/_docs/tutorial/kylin_client_tool.cn.md b/website/_docs/tutorial/kylin_client_tool.cn.md
new file mode 100644
index 0000000..b8c6d8f
--- /dev/null
+++ b/website/_docs/tutorial/kylin_client_tool.cn.md
@@ -0,0 +1,99 @@
+---
+layout: docs-cn
+title: Kylin Client Tool 使用教程
+categories: 教程
+permalink: /cn/docs/tutorial/kylin_client_tool.html
+version: v1.0
+since: v1.2.0
+---
+
+> Kylin-client-tool是一个用python编写的，完全基于kylin的rest api的工具。可以实现kylin的cube创建，按时build cube，job的提交、调度、查看、取消与恢复。
+
+## 安装
+1.确认运行环境安装了python2.6/2.7
+
+2.本工具需安装第三方python包apscheduler和requests，运行setup.sh进行安装，mac用户运行setup-mac.sh进行安装。也可用setuptools进行安装
+
+## 配置
+修改工具目录下的settings/settings.py文件，进行配置
+
+`KYLIN_USER` Kylin用户名
+
+`KYLIN_PASSWORD` Kylin的密码
+
+`KYLIN_REST_HOST` Kylin的地址
+
+`KYLIN_REST_PORT` Kylin的端口
+
+`KYLIN_JOB_MAX_COCURRENT` 允许同时build的job数量
+
+`KYLIN_JOB_MAX_RETRY` cube build出现error后，允许的重启job次数
+
+## 命令行的使用
+本工具使用optparse通过命令行来执行操作，具体用法可通过`python kylin_client_tool.py －h`来查看
+
+## cube的创建
+本工具定义了一种读手写的文本，来快速cube创建的方法，格式如下
+
+`cube名|fact table名|维度1,维度1类型;维度2,维度2类型...|指标1,指标1表达式,指标1类型...|设置项|filter|`
+
+设置项内有以下选项，
+
+`no_dictionary` 设置Rowkeys中不生成dictionary的维度及其长度
+
+`mandatory_dimension` 设置Rowkeys中mandatory的维度
+
+`aggregation_group` 设置aggregation group
+
+`partition_date_column` 设置partition date column
+
+`partition_date_start` 设置partition start date
+
+具体例子可以查看cube_def.csv文件，目前不支持含lookup table的cube创建
+
+使用`-c`命令进行创建，用`-F`指定cube定义文件，例如
+
+`python kylin_client_tool.py -c -F cube_def.csv`
+
+## build cube
+###使用cube定义文件build
+使用`-b`命令，需要用`-F`指定cube定义文件，如果指定了partition date column，通过`-T`指定end date(year-month-day格式)，若不指定，以当前时间为end date，例如
+
+`python kylin_client_tool.py -b -F cube_def.csv -T 2016-03-01`
+
+###使用cube名文件build
+用`-f`指定cube名文件，文件每行一个cube名
+
+`python kylin_client_tool.py -b -f cube_names.csv -T 2016-03-01`
+
+###直接命令行写cube名build
+用`-C`指定cube名，通过逗号进行分隔
+
+`python kylin_client_tool.py -b -C client_tool_test1,client_tool_test2 -T 2016-03-01`
+
+## job管理
+###查看job状态
+使用`-s`命令查看，用`-f`指定cube名文件，用`-C`指定cube名，若不指定，将查看所有cube状态。用`-S`指定job状态，R表示`Running`，E表示`Error`，F表示`Finished`，D表示`Discarded`，例如：
+
+`python kylin_client_tool.py -s -C kylin_sales_cube -f cube_names.csv -S F`
+
+###恢复job
+用`-r`命令恢复job，用`-f`指定cube名文件，用`-C`指定cube名，若不指定，将恢复所有Error状态的job，例如：
+
+`python kylin_client_tool.py -r -C kylin_sales_cube -f cube_names.csv`
+
+###取消job
+用`-k`命令取消job，用`-f`指定cube名文件，用`-C`指定cube名，若不指定，将取消所有Running或Error状态的job，例如：
+
+`python kylin_client_tool.py -k -C kylin_sales_cube -f cube_names.csv`
+
+## 定时build cube
+### 每隔一段时间build cube
+在cube build命令的基础上，使用`-B i`指定每隔一段时间build的方式，使用`-O`指定间隔的小时数，例如：
+
+`python kylin_client_tool.py -b -F cube_def.csv -B i -O 1`
+
+### 设定时间build cube
+使用`-B t`指定按时build cube的方式，使用`-O`指定build时间，用逗号进行分隔
+
+`python kylin_client_tool.py -b -F cube_def.csv -T 2016-03-04 -B t -O 2016,3,1,0,0,0`
http://git-wip-us.apache.org/repos/asf/kylin/blob/44a830da/website/_docs/tutorial/kylin_client_tool.md
----------------------------------------------------------------------
diff --git a/website/_docs/tutorial/kylin_client_tool.md b/website/_docs/tutorial/kylin_client_tool.md
new file mode 100644
index 0000000..f81a4cf
--- /dev/null
+++ b/website/_docs/tutorial/kylin_client_tool.md
@@ -0,0 +1,100 @@
+---
+layout: docs
+title: Kylin Client Tool Tutorial
+categories: tutorial
+permalink: /docs/tutorial/kylin_client_tool.html
+version: v1.0
+since: v1.2.0
+---
+
+> Kylin-client-tool is a tool coded with python, completely based on Kylin's restful apis.
+> With it, users can easily create Kylin Cubes, timely build cubes, summit/schedule/check/cancel/resume jobs.
+
+## Install
+1.Make sure python2.6/2.7 installed
+
+2.Two python packages `apscheduler` and `requests` are needed, run `setup.sh` to install. For mac users, please run`setup-mac.sh`. Or you can install them with setuptools.
+
+## Settings
+Modify settings/settings.py to set,
+
+`KYLIN_USER` Kylin's username
+
+`KYLIN_PASSWORD` Kylin's password
+
+`KYLIN_REST_HOST` Kylin's address
+
+`KYLIN_REST_PORT` Kylin's port
+
+`KYLIN_JOB_MAX_COCURRENT` Max concurrent jobs for cube building
+
+`KYLIN_JOB_MAX_RETRY` Max failover time after job failed
+
+## About command lines
+This tool is used by command lines, please run `python kylin_client_tool.py －h` to check for details.
+
+## Cube's creation
+This tool has its own cube definition method, the format is as below,
+
+`cube name|fact table name|dimension1,type of dimension1;dimension2,type of dimension2...|measure1,expression of measure1,type of measure1...|settings|filter|`
+
+`settings` has options below,
+
+`no_dictionary` Set dimensions not generate dictionaries in "Rowkeys" and their lengths
+
+`mandatory_dimension` Set mandatory dimensions in "Rowkeys"
+
+`aggregation_group` Set aggregation group
+
+`partition_date_column` Set partition date column
+
+`partition_date_start` Set partition start date
+
+Cases can be found in file cube_def.csv, it does not support cube creation with lookup table
+
+Use `-c` to create, use `-F` to specify cube definition file, for instance
+
+`python kylin_client_tool.py -c -F cube_def.csv`
+
+## Build cube
+###Build cube with cube definition file
+Use `-b` to build, use `-F` to specify the cube definition file, if "partition date column" specified, use `-T` to specify the end date(year-month-day), if not specified, it will take current time as end date, for instance,
+
+`python kylin_client_tool.py -b -F cube_def.csv -T 2016-03-01`
+
+###Build cube with file of cube names
+Use `-f` to specify the file of cube names, each line of the file has one cube name, for instance,
+
+`python kylin_client_tool.py -b -f cube_names.csv -T 2016-03-01`
+
+###Build cube with cube names
+Use `-C` to specify cube names, split with commas, for instance,
+
+`python kylin_client_tool.py -b -C client_tool_test1,client_tool_test2 -T 2016-03-01`
+
+## Job management
+###Check jobs' statuses
+Use `-s` to check, use `-f` to specify file of cube names, use `-C` to specify cube names, if not specified, it will check all cubes' statuses. Use `-S` to specify jobs' statuses, R for `Running`, E for `Error`, F for `Finished`, D for `Discarded`, for instance,
+
+`python kylin_client_tool.py -s -C kylin_sales_cube -f cube_names.csv -S F`
+
+###Resume jobs
+Use `-r` to resume jobs, use `-f` to specify file of cube names, use `-C` to specify cube names, if not specified, it will reuse all jobs with error status, for instance,
+
+`python kylin_client_tool.py -r -C kylin_sales_cube -f cube_names.csv`
+
+###Cancel jobs
+Use `-k` to cancel jobs, use `-f` to specify file with cube names, use `-C` to specify cube names, if not specified, it will cancel all jobs with running or error statuses, for instance,
+
+`python kylin_client_tool.py -k -C kylin_sales_cube -f cube_names.csv`
+
+## Schedule cubes building
+### Build cube at set intervals
+On the basis of command of cube building, use `-B i` to specify the mode of build cube at set intervals, use `-O` to specify how many hours, for instance,
+
+`python kylin_client_tool.py -b -F cube_def.csv -B i -O 1`
+
+### Build cube at given time
+Use `-B t` to specify the mode of build cube at given time, use `-O` to specify the build time, split with commas,for instance,
+
+`python kylin_client_tool.py -b -F cube_def.csv -T 2016-03-04 -B t -O 2016,3,1,0,0,0`