You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by li...@apache.org on 2015/03/28 01:03:49 UTC
[01/37] incubator-kylin git commit: update website title to Apache
Kylin
Repository: incubator-kylin
Updated Branches:
refs/heads/master b1b3fad20 -> 23f0dae95
update website title to Apache Kylin
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/13d791a5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/13d791a5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/13d791a5
Branch: refs/heads/master
Commit: 13d791a581ac9ab34265711c40eb74cd4190ad92
Parents: d4ee587
Author: lukehan <lu...@apache.org>
Authored: Mon Mar 16 13:37:33 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Mon Mar 16 13:37:33 2015 -0700
----------------------------------------------------------------------
docs/website/_includes/head.cn.html | 4 ++--
docs/website/_includes/head.html | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/13d791a5/docs/website/_includes/head.cn.html
----------------------------------------------------------------------
diff --git a/docs/website/_includes/head.cn.html b/docs/website/_includes/head.cn.html
index ae98dee..7e4f3f7 100644
--- a/docs/website/_includes/head.cn.html
+++ b/docs/website/_includes/head.cn.html
@@ -3,9 +3,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Kylin.IO | {% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
+ <title>Apache Kylin | {% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
<meta name="description" content="{% if page.excerpt %}{{ page.excerpt | strip_html | strip_newlines | truncate: 160 }}{% else %}{{ site.description }}{% endif %}">
- <meta name="author" content="Kylin.IO">
+ <meta name="author" content="Apache Kylin">
<link rel="shortcut icon" href="fav.png" type="image/png">
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/13d791a5/docs/website/_includes/head.html
----------------------------------------------------------------------
diff --git a/docs/website/_includes/head.html b/docs/website/_includes/head.html
index ae98dee..7e4f3f7 100644
--- a/docs/website/_includes/head.html
+++ b/docs/website/_includes/head.html
@@ -3,9 +3,9 @@
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Kylin.IO | {% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
+ <title>Apache Kylin | {% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
<meta name="description" content="{% if page.excerpt %}{{ page.excerpt | strip_html | strip_newlines | truncate: 160 }}{% else %}{{ site.description }}{% endif %}">
- <meta name="author" content="Kylin.IO">
+ <meta name="author" content="Apache Kylin">
<link rel="shortcut icon" href="fav.png" type="image/png">
[35/37] incubator-kylin git commit: Merge pull request #450 from
IT-DONG/staging
Posted by li...@apache.org.
Merge pull request #450 from IT-DONG/staging
change "! -eq" syntax to "!=" since we compare string instead of a number
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/c573b074
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/c573b074
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/c573b074
Branch: refs/heads/master
Commit: c573b07448b6aea2c56dd5573ef400f971a031b8
Parents: cac1589 df0123b
Author: qianhao.zhou <qi...@ebay.com>
Authored: Thu Mar 26 10:15:45 2015 +0800
Committer: qianhao.zhou <qi...@ebay.com>
Committed: Thu Mar 26 10:15:45 2015 +0800
----------------------------------------------------------------------
job/pom.xml | 27 ++++++++++++---------------
script/download-tomcat.sh | 4 ++--
2 files changed, 14 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
[22/37] incubator-kylin git commit: Merge branch 'staging' of
https://github.com/KylinOLAP/Kylin into staging
Posted by li...@apache.org.
Merge branch 'staging' of https://github.com/KylinOLAP/Kylin into staging
Conflicts:
query/src/test/resources/query/sql/query78.sql
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/3655b338
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/3655b338
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/3655b338
Branch: refs/heads/master
Commit: 3655b338f9833a894fc873a393e294dcb7a3cf78
Parents: 222b144 3563a91
Author: Li, Yang <ya...@ebay.com>
Authored: Fri Mar 20 15:45:37 2015 +0800
Committer: Li, Yang <ya...@ebay.com>
Committed: Fri Mar 20 15:45:37 2015 +0800
----------------------------------------------------------------------
----------------------------------------------------------------------
[07/37] incubator-kylin git commit: KYLIN-650 add "what's next" part
Posted by li...@apache.org.
KYLIN-650 add "what's next" part
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/ae75f5f4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/ae75f5f4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/ae75f5f4
Branch: refs/heads/master
Commit: ae75f5f408c123b9b140f0334e185cd475159724
Parents: da16316
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 19:10:03 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 19:10:03 2015 -0700
----------------------------------------------------------------------
.../Kylin Cube Build and Job Monitoring Tutorial.md | 10 +++++++++-
docs/Tutorial/Kylin ODBC Driver Tutorial.md | 1 -
docs/Tutorial/Quick play with a sample cube.md | 8 +++++---
3 files changed, 14 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/ae75f5f4/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md b/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md
index 827cd94..452f24b 100644
--- a/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md
+++ b/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md
@@ -57,4 +57,12 @@ Click the icon button show in each step to see the details: `Parameters`, `Log`,
- 
\ No newline at end of file
+ 
+
+#### What's next
+
+After cube being built, you might need to:
+
+1. [Query the cube via web interface](Kylin Web Tutorial.md)
+2. [Query the cube via ODBC](Kylin ODBC Driver Tutorial.md)
+3. [Grant permission to cubes](Kylin Cube Permission Grant Tutorial.md)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/ae75f5f4/docs/Tutorial/Kylin ODBC Driver Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin ODBC Driver Tutorial.md b/docs/Tutorial/Kylin ODBC Driver Tutorial.md
index 7f84213..17653fd 100644
--- a/docs/Tutorial/Kylin ODBC Driver Tutorial.md
+++ b/docs/Tutorial/Kylin ODBC Driver Tutorial.md
@@ -32,6 +32,5 @@ Kylin ODBC Driver Tutorial
2. [Kylin and Excel Tutorial](Kylin and Excel Tutorial.md)
-
## Bug Report
Open github issue here
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/ae75f5f4/docs/Tutorial/Quick play with a sample cube.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Quick play with a sample cube.md b/docs/Tutorial/Quick play with a sample cube.md
index 37c44ff..9495b4a 100644
--- a/docs/Tutorial/Quick play with a sample cube.md
+++ b/docs/Tutorial/Quick play with a sample cube.md
@@ -10,9 +10,11 @@ Kylin provides a script for you to create a sample Cube; the script will also cr
select cal_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by cal_dt order by cal_dt
6. You can verify the query result and compare the response time with hive;
+
#### What's next
-After cube being created, you might need to:
+After cube being built, you might need to:
-1. [Build the cube so that it can be queried](Kylin Cube Build and Job Monitoring Tutorial.md)
-2. [Grant permission to cubes](Kylin Cube Permission Grant Tutorial.md)
\ No newline at end of file
+1. [Query the cube via web interface](Kylin Web Tutorial.md)
+2. [Query the cube via ODBC](Kylin ODBC Driver Tutorial.md)
+3. [Grant permission to cubes](Kylin Cube Permission Grant Tutorial.md)
\ No newline at end of file
[04/37] incubator-kylin git commit: KYLIN-650 moving wikis to code
repository
Posted by li...@apache.org.
KYLIN-650 moving wikis to code repository
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/592e96b5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/592e96b5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/592e96b5
Branch: refs/heads/master
Commit: 592e96b519171ae9906cd1774b2a3e88e9dc3d91
Parents: a540f40
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 18:56:04 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 18:56:21 2015 -0700
----------------------------------------------------------------------
README.md | 71 ++-----------
...equently Asked Questions on Installation.pdf | Bin 52261 -> 0 bytes
docs/Installation/Hadoop Environment.md | 21 ++++
.../How to generate install binaries.md | 6 ++
docs/Installation/Installation.md | 38 +++++++
.../Installation/Multiple Kylin REST servers.md | 9 ++
...f Hadoop CLI Installation (Dev Env Setup).md | 76 -------------
... Hadoop CLI Installation (Dev Env Setup).pdf | Bin 44909 -> 0 bytes
.../Installation/Off Hadoop CLI Installation.md | 67 ++++++++++++
docs/Installation/On Hadoop CLI installation.md | 106 +++----------------
.../Installation/On Hadoop CLI installation.pdf | Bin 704398 -> 0 bytes
...On Hadoop Kylin installation using Docker.md | 2 +-
...n Hadoop Kylin installation using Docker.pdf | Bin 40098 -> 0 bytes
docs/Installation/install_and_run.md | 83 ---------------
docs/Installation/off_cli_install_scene.png | Bin 0 -> 72567 bytes
docs/Installation/on_cli_install_scene.png | Bin 0 -> 53722 bytes
docs/README.md | 0
...n Cube Build and Job Monitoring Tutorial.pdf | Bin 1054988 -> 0 bytes
docs/Tutorial/Kylin Cube Creation Tutorial.md | 9 +-
docs/Tutorial/Kylin Cube Creation Tutorial.pdf | Bin 1488946 -> 0 bytes
.../Kylin Cube Permission Grant Tutorial.pdf | Bin 358047 -> 0 bytes
docs/Tutorial/Kylin ODBC Driver Tutorial.md | 7 ++
docs/Tutorial/Kylin ODBC Driver Tutorial.pdf | Bin 34429 -> 0 bytes
docs/Tutorial/Kylin Web Tutorial.md | 8 +-
docs/Tutorial/Kylin Web Tutorial.pdf | Bin 1714951 -> 0 bytes
docs/Tutorial/Kylin and Excel Tutorial.md | 3 +
docs/Tutorial/Kylin and Tableau Tutorial.pdf | Bin 1246553 -> 0 bytes
docs/Tutorial/Quick play with a sample cube.md | 10 ++
old_README.md | 94 ++++++++++++++++
29 files changed, 294 insertions(+), 316 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 44f6859..28c2df6 100644
--- a/README.md
+++ b/README.md
@@ -2,83 +2,24 @@ Kylin OLAP
===========
-Overview
+What is Kylin
------------
> Extreme OLAP Engine for Big Data
An open source distributed Analytical Engine from eBay to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop to support TB to PB size analysis.
+This [slice](docs/Apache Kylin 2014 Dec.pdf) might help you better understand what kylin is and how it works.
-Getting Started
----------------
-### Prerequisites ###
-
-#### Hadoop ####
-- Hadoop: 2.2.0.2.0.6.0-61 or above
-- Hive: 0.12.0.2.0.6.0-61 or above
-- HBase: 0.96.0.2.0.6.0-61-hadoop2
-
-
-_Tested with Hortonworks HDP 2.1.3 and Cloudera Quickstart VM 5.1._
-
-#### Misc ####
-- Command hadoop, hive, hbase is workable on your hadoop cluster
-- JDK Runtime: JDK7 (OpenJDK or Oracle JDK)
-- Maven
-- Git
-- Tomcat (CATALINA_HOME being set)
-- Npm
-
-#### Before Install ####
-- For the quick tutorial, we assume that your hadoop has priviledges disabled which means any user could run hadoop and hive command
-
-
-Installation
+Tutorial
------------
-It is very easy to install Kylin for exploration/development. There are 3 supported options:
-
-
-1. Sandbox (HDP or CDH sandbox)
-2. Dev Environment (IDE + Sandbox)
-3. Docker container (Docker)
-
-#### Sandbox (On-Hadoop-CLI installation) ####
-
-If you are free to install Kylin on your hadoop CLI machine or Hadoop sandbox, this is the most convenient scenario, for it puts everything in a single machine.
-
-
-
+Please follow this installation tutorial to start with Kylin: [Installation Tutorial](docs/Installation/Installation.md)
-For normal users, the application in the above picture means `Kylin Web`, which contains a web interface for cube building, querying and all sorts of management. Kylin Web launches a query engine for querying and a cube build engine for building cubes. These two engines interact with the components in Hadoop CLI, like hive and hbase.
-For a hands-on tutorial please visit [On-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-CLI-installation).
-#### Dev Environment (Off-Hadoop-CLI Installation) ####
-
-This is typically for development environment setup.
-
-Applications does not necessarily mean Kylin Web, it can also be any test cases or java applications running at your local IDE(query engine and cube build engine can be launched programmatically in your code). In this case, it is no longer reasonable to assume that all the Kylin components reside in the same machine as your Hadoop CLI. Fortunately, Kylin still works under such condition with proper settings on your CLI machine.
-
-
-
-For a hands-on tutorial please visit [Off-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/Off-Hadoop-CLI-Installation-(Dev-Env-Setup))
-
-#### Docker Container ####
-With help from [SequenceIQ](http://sequenceiq.com/), there's docker container for Kylin (along with Hadoop, HBase and Hive) available now:[sequenceiq/docker-kylin](https://github.com/sequenceiq/docker-kylin). The only thing you will need to do is to pull the container from the official Docker repository to be up and running in few minutes.
-
-Features:
-
-- Size - Light weight compared to downloading and setting up HDP or CDH sandbox.
-- Cluster support - Supports multi node installation.
-- Fully Automated - No manual steps. One command does it all
-
-For a hands-on tutorial please visit [Kylin Docker installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-Kylin-installation-using-Docker).
-
-
-
-### Resources ###
+Resources
+------------
* Web Site: <http://kylin.io>
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Frequently Asked Questions on Installation.pdf
----------------------------------------------------------------------
diff --git a/docs/Installation/Frequently Asked Questions on Installation.pdf b/docs/Installation/Frequently Asked Questions on Installation.pdf
deleted file mode 100644
index ff340ab..0000000
Binary files a/docs/Installation/Frequently Asked Questions on Installation.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Hadoop Environment.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Hadoop Environment.md b/docs/Installation/Hadoop Environment.md
new file mode 100644
index 0000000..2e72527
--- /dev/null
+++ b/docs/Installation/Hadoop Environment.md
@@ -0,0 +1,21 @@
+## Hadoop Environment
+
+Kylin requires you having access to a hadoop CLI, where you have full permissions to hdfs, hive, hbase and map-reduce. To make things easier we strongly recommend you starting with running Kylin on a hadoop sandbox, like <http://hortonworks.com/products/hortonworks-sandbox/>. In the following tutorial we'll go with **Hortonworks Sandbox 2.1** and **Cloudera QuickStart VM 5.1**.
+
+To avoid permission issue, we suggest you using `root` account. The password for **Hortonworks Sandbox 2.1** is `hadoop` , for **Cloudera QuickStart VM 5.1** is `cloudera`.
+
+We also suggest you using bridged mode instead of NAT mode in your virtual box settings. Bridged mode will assign your sandbox an independent IP so that you can avoid issues like https://github.com/KylinOLAP/Kylin/issues/12
+
+### Start Hadoop
+
+Please make sure Hive, HDFS and HBase are available on our CLI machine.
+If you don't know how, here's a simple tutorial for hortonworks sanbox:
+
+Use ambari helps to launch hadoop:
+
+ ambari-agent start
+ ambari-server start
+
+With both command successfully run you can go to ambari homepage at <http://your_sandbox_ip:8080> (user:admin,password:admin) to check everything's status. **By default hortonworks ambari disables Hbase, you'll need manually start the `Hbase` service at ambari homepage.**
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/How to generate install binaries.md
----------------------------------------------------------------------
diff --git a/docs/Installation/How to generate install binaries.md b/docs/Installation/How to generate install binaries.md
new file mode 100644
index 0000000..d7946e0
--- /dev/null
+++ b/docs/Installation/How to generate install binaries.md
@@ -0,0 +1,6 @@
+###Build from source
+> git clone https://github.com/KylinOLAP/Kylin.git
+> cd KylinOLAP/Kylin
+> sh script/package.sh
+
+In order to generate binary package, **maven** and **npm** are pre-requisites.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Installation.md b/docs/Installation/Installation.md
new file mode 100644
index 0000000..5b33a56
--- /dev/null
+++ b/docs/Installation/Installation.md
@@ -0,0 +1,38 @@
+Installation
+------------
+
+### Prerequisites on hadoop###
+
+- Hadoop: 2.2.0.2.0.6.0-61 or above
+- Hive: 0.12.0.2.0.6.0-61 or above
+- HBase: 0.96.0.2.0.6.0-61-hadoop2
+
+_Tested with Hortonworks HDP 2.1.3 and Cloudera Quickstart VM 5.1._
+
+
+It is very easy to install Kylin for exploration/development. There are 3 installation scenarios:
+
+#### On-Hadoop-CLI installation ####
+
+If you are free to install Kylin on your hadoop CLI machine or Hadoop sandbox, this is the most convenient scenario, for it puts everything in a single machine.
+
+For a hands-on tutorial please visit [On-Hadoop-CLI installation](On Hadoop CLI installation.md).
+
+#### Dev Environment (Off-Hadoop-CLI Installation) ####
+
+This is typically for development environment setup.
+
+For a hands-on tutorial please visit [Off Hadoop CLI Installation (Dev Env Setup)](Off Hadoop CLI Installation.md).
+#### Docker Container ####
+With help from [SequenceIQ](http://sequenceiq.com/), there's docker container for Kylin (along with Hadoop, HBase and Hive) available now:[sequenceiq/docker-kylin](https://github.com/sequenceiq/docker-kylin). The only thing you will need to do is to pull the container from the official Docker repository to be up and running in few minutes.
+
+Features:
+
+- Size - Light weight compared to downloading and setting up HDP or CDH sandbox.
+- Cluster support - Supports multi node installation.
+- Fully Automated - No manual steps. One command does it all
+
+For a hands-on tutorial please visit [Kylin Docker installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-Kylin-installation-using-Docker).
+
+
+
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Multiple Kylin REST servers.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Multiple Kylin REST servers.md b/docs/Installation/Multiple Kylin REST servers.md
new file mode 100644
index 0000000..2ee6832
--- /dev/null
+++ b/docs/Installation/Multiple Kylin REST servers.md
@@ -0,0 +1,9 @@
+### Multiple Kylin REST servers
+
+If you are running Kylin in a cluster or you have multiple Kylin REST server instances, please make sure you have the following property correctly configured in ${KYLIN_HOME}/conf/kylin.properties
+
+1. kylin.rest.servers
+ List of web servers in use, this enables one web server instance to sync up with other servers.
+
+2. kylin.server.mode
+ Make sure there is only one instance whose "kylin.server.mode" is set to "all" if there are multiple instances.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).md
----------------------------------------------------------------------
diff --git a/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).md b/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).md
deleted file mode 100644
index ef7b205..0000000
--- a/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).md
+++ /dev/null
@@ -1,76 +0,0 @@
-Off Hadoop CLI Installation (Dev Env Setup)
-===
-Off-Hadoop-CLI installation is usually for **development use**.
-
-Developers want to run kylin test cases or applications at their development machine. The scenario is depicted at https://github.com/KylinOLAP/Kylin#off-hadoop-cli-installation.
-
-
-By following this tutorial, you will be able to build kylin test cubes by running a specific test case, and you can further run other test cases against the cubes having been built.
-
-
-## Environment on the Hadoop CLI
-
-Off-Hadoop-CLI installation requires you having a hadoop CLI machine (or a hadoop sandbox) as well as your local develop machine. To make things easier we strongly recommend you starting with running Kylin on a hadoop sandbox, like <http://hortonworks.com/products/hortonworks-sandbox/>. In the following tutorial we'll go with **Hortonworks Sandbox 2.1**.
-
-### Start Hadoop
-
-In Hortonworks sandbox, ambari helps to launch hadoop:
-
- ambari-agent start
- ambari-server start
-
-With both command successfully run you can go to ambari home page at <http://yoursandboxip:8080> (user:admin,password:admin) to check everything's status. By default ambari disables Hbase, you'll need manually start the `Hbase` service.
-
-For other hadoop distribution, basically start the hadoop cluster, make sure HDFS, YARN, Hive, HBase are running.
-
-
-## Environment on the dev machine
-
-### Install maven
-
-The latest maven can be found at <http://maven.apache.org/download.cgi>, we create a symbolic so that `mvn` can be run anywhere.
-
- cd ~
- wget http://apache.proserve.nl/maven/maven-3/3.2.3/binaries/apache-maven-3.2.3-bin.tar.gz
- tar -xzvf apache-maven-3.2.3-bin.tar.gz
- ln -s /root/apache-maven-3.2.3/bin/mvn /usr/bin/mvn
-
-### Compile
-
-First clone the Kylin project to your local:
-
- git clone https://github.com/KylinOLAP/Kylin.git
-
-Install Kylin artifacts to the maven repo
-
- mvn clean install -DskipTests
-
-### Modify local configuration
-
-Local configuration must be modified to point to your hadoop sandbox (or CLI) machine. If you are using a Hortonworks sandbox, this section may be skipped.
-
-* In **examples/test_case_data/sandbox/kylin.properties**
- * Find `sandbox` and replace with your hadoop hosts
- * Find `kylin.job.remote.cli.username` and `kylin.job.remote.cli.password`, fill in the user name and password used to login hadoop cluster for hadoop command execution
-
-* In **examples/test_case_data/sandbox**
- * For each configuration xml file, find all occurrence of `sandbox` and replace with your hadoop hosts
-
-An alternative to the host replacement is updating your `hosts` file to resolve `sandbox` and `sandbox.hortonworks.com` to the IP of your sandbox machine.
-
-### Run unit tests
-
-Run a end-to-end cube building test
-
- mvn test -Dtest=org.apache.kylin.job.BuildCubeWithEngineTest -DfailIfNoTests=false
-
-Run other tests, the end-to-end cube building test is exclueded
-
- mvn test
-
-### Launch Kylin Web Server
-
-In your Eclipse IDE, launch `org.apache.kylin.rest.DebugTomcat` with specifying VM arguments "-Dspring.profiles.active=sandbox". (By default Kylin server will listen on 7070 port; If you want to use another port, please specify it as a parameter when run `DebugTomcat)
-
-Check Kylin Web available at http://localhost:7070 (user:ADMIN,password:KYLIN)
-
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).pdf
----------------------------------------------------------------------
diff --git a/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).pdf b/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).pdf
deleted file mode 100644
index 1cdf8da..0000000
Binary files a/docs/Installation/Off Hadoop CLI Installation (Dev Env Setup).pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/Off Hadoop CLI Installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Off Hadoop CLI Installation.md b/docs/Installation/Off Hadoop CLI Installation.md
new file mode 100644
index 0000000..d63bf13
--- /dev/null
+++ b/docs/Installation/Off Hadoop CLI Installation.md
@@ -0,0 +1,67 @@
+Off Hadoop CLI Installation (Dev Env Setup)
+===
+Off-Hadoop-CLI installation is usually for **development use**.
+
+Developers want to run kylin test cases or applications at their development machine. The scenario is depicted as:
+
+
+
+The "Applications" here can be any unit cases running on your dev machine's IDE. By following this tutorial, you will be able to build kylin test cubes by running a specific test case, and you can further run other test cases against the cubes having been built.
+
+
+## Environment on the Hadoop CLI
+
+### Environment
+
+Kylin Rquires a properly setup hadoop environment to run. Please take a look [this](Hadoop Environment.md).
+
+## Environment on the dev machine
+
+### Install maven
+
+The latest maven can be found at <http://maven.apache.org/download.cgi>, we create a symbolic so that `mvn` can be run anywhere.
+
+ cd ~
+ wget http://apache.proserve.nl/maven/maven-3/3.2.3/binaries/apache-maven-3.2.3-bin.tar.gz
+ tar -xzvf apache-maven-3.2.3-bin.tar.gz
+ ln -s /root/apache-maven-3.2.3/bin/mvn /usr/bin/mvn
+
+### Compile
+
+First clone the Kylin project to your local:
+
+ git clone https://github.com/KylinOLAP/Kylin.git
+
+Install Kylin artifacts to the maven repo
+
+ mvn clean install -DskipTests
+
+### Modify local configuration
+
+Local configuration must be modified to point to your hadoop sandbox (or CLI) machine. If you are using a Hortonworks sandbox, this section may be skipped.
+
+* In **examples/test_case_data/sandbox/kylin.properties**
+ * Find `sandbox` and replace with your hadoop hosts
+ * Find `kylin.job.remote.cli.username` and `kylin.job.remote.cli.password`, fill in the user name and password used to login hadoop cluster for hadoop command execution
+
+* In **examples/test_case_data/sandbox**
+ * For each configuration xml file, find all occurrence of `sandbox` and replace with your hadoop hosts
+
+An alternative to the host replacement is updating your `hosts` file to resolve `sandbox` and `sandbox.hortonworks.com` to the IP of your sandbox machine.
+
+### Run unit tests
+
+Run a end-to-end cube building test
+
+ mvn test -Dtest=org.apache.kylin.job.BuildCubeWithEngineTest -DfailIfNoTests=false
+
+Run other tests, the end-to-end cube building test is exclueded
+
+ mvn test
+
+### Launch Kylin Web Server
+
+In your Eclipse IDE, launch `org.apache.kylin.rest.DebugTomcat` with specifying VM arguments "-Dspring.profiles.active=sandbox". (By default Kylin server will listen on 7070 port; If you want to use another port, please specify it as a parameter when run `DebugTomcat)
+
+Check Kylin Web available at http://localhost:7070 (user:ADMIN,password:KYLIN)
+
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/On Hadoop CLI installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/On Hadoop CLI installation.md b/docs/Installation/On Hadoop CLI installation.md
index a718722..db2409b 100644
--- a/docs/Installation/On Hadoop CLI installation.md
+++ b/docs/Installation/On Hadoop CLI installation.md
@@ -1,102 +1,30 @@
On Hadoop CLI installation
===
-On-Hadoop-CLI installation is for demo use, or for those who want to host their own web site to provide Kylin service. The scenario is depicted at https://github.com/KylinOLAP/Kylin#on-hadoop-cli-installation.
+On-Hadoop-CLI installation is the most common way of installing Kylin. It can be used for demo use, or for those who want to host their own web site to provide Kylin service. The scenario is depicted as:
-Except for some prerequisite software installations, the core of Kylin installation is accomplished by running a single script. After running the script, you will be able to build sample cube and query the tables behind the cubes via a unified web interface.
-
-## Environment
-
-Running On-Hadoop-CLI installation requires you having access to a hadoop CLI, where you have full permissions to hdfs, hive, hbase and map-reduce. To make things easier we strongly recommend you starting with running Kylin on a hadoop sandbox, like <http://hortonworks.com/products/hortonworks-sandbox/>. In the following tutorial we'll go with **Hortonworks Sandbox 2.1** and **Cloudera QuickStart VM 5.1**.
-
-To avoid permission issue, we suggest you using `root` account. The password for **Hortonworks Sandbox 2.1** is `hadoop` , for **Cloudera QuickStart VM 5.1** is `cloudera`.
-
-We also suggest you using bridged mode instead of NAT mode in your virtual box settings. Bridged mode will assign your sandbox an independent IP so that you can avoid issues like https://github.com/KylinOLAP/Kylin/issues/12
-
-### Start Hadoop
-
-For Hortonworks, ambari helps to launch hadoop:
-
- ambari-agent start
- ambari-server start
-
-With both command successfully run you can go to ambari homepage at <http://your_sandbox_ip:8080> (user:admin,password:admin) to check everything's status. **By default hortonworks ambari disables Hbase, you'll need manually start the `Hbase` service at ambari homepage.**
-
-
-
-For Cloudera, you can skip this step since they're by default activated.
-
-### Install tomcat
-
-The latest tomcat can be found at <http://tomcat.apache.org/download-70.cgi>, we need the variable `CATALINA_HOME` exported.
+
- cd ~
- wget http://apache.cs.uu.nl/dist/tomcat/tomcat-7/v7.0.57/bin/apache-tomcat-7.0.57.tar.gz
- tar -xzvf apache-tomcat-7.0.57.tar.gz
- export CATALINA_HOME=~/apache-tomcat-7.0.57
+For normal use cases, the application in the above picture means Kylin Web, which contains a web interface for cube building, querying and all sorts of management. Kylin Web launches a query engine for querying and a cube build engine for building cubes. These two engines interact with the components in Hadoop CLI, like hive and hbase.
+Except for some prerequisite software installations, the core of Kylin installation is accomplished by running a single script. After running the script, you will be able to build sample cube and query the tables behind the cubes via a unified web interface.
-### Install maven:
-
-For Cloudera, you can skip this step since maven is by default installed.
-
-The latest maven can be found at <http://maven.apache.org/download.cgi>, we create a symbolic so that `mvn` can be run anywhere.
-
- cd ~
- wget http://apache.proserve.nl/maven/maven-3/3.2.3/binaries/apache-maven-3.2.3-bin.tar.gz
- tar -xzvf apache-maven-3.2.3-bin.tar.gz
- ln -s ~/apache-maven-3.2.3/bin/mvn /usr/bin/mvn
-
-
-### Install npm
-
-Npm comes with latest versions of nodejs, just donwload nodejs from <http://nodejs.org/download/>, `npm` is located at the `/bin` folder of `nodejs`, we append the folder to `PATH` so that `npm` can be run anywhere.
-
- cd ~
- wget http://nodejs.org/dist/v0.10.32/node-v0.10.32-linux-x64.tar.gz
- tar -xzvf node-v0.10.32-linux-x64.tar.gz
- export PATH=~/node-v0.10.32-linux-x64/bin:$PATH
-
-
-## Build cubes & Query tables
-
-First clone the Kylin project to your local:
-
- git clone https://github.com/KylinOLAP/Kylin.git
-
-Go into the folder and run deploy script:
-
- cd Kylin/
- ./deploy.sh
-
-If you meet any problems, please check [FAQ](https://github.com/KylinOLAP/Kylin/wiki/Frequently-Asked-Questions-on-Installation) first.
-This script will help to:
-
-1. Check your environment
-2. Build Kylin artifacts with Maven
-3. Generate some sample tables
-3. Create empty cubes for these tables
-4. Lauch a one-stop website to build cubes & query tables
+### Environment
-After successfully running the script, please navigate to <http://your_sandbox_ip:7070> to build your cube and query it. The username/password is ADMIN/KYLIN
+Kylin Rquires a properly setup hadoop environment to run. Please take a look [this](Hadoop Environment.md).
-### Build Cube in Cubes Tab
+### Install Kylin
-
-### Check Building Progress in Job Tab
-
-### Query Tables in Query Tab
-
+1. Download latest Kylin binaries at http://kylin.incubator.apache.org/download/
+2. export KYLIN_HOME pointing to the extracted Kylin folder
+3. Make sure the user has the privilege to run hadoop, hive and hbase cmd in shell. If you are not so sure, you can just run **bin/check-env.sh**, it will print out the detail information if you have some environment issues.
+4. To start Kylin, simply run **bin/kylin.sh start**
+5. To stop Kylin, simply run **bin/kylin.sh stop**
-## What if I closed my VM?
+> If you want to have multiple Kylin instances please refer to [this](Multiple Kylin REST servers.md)
-If you shut down the VM and restarted it, Kylin will not automatically start. Depending on whether you succeed in running the deploy.sh, you should:
+After Kylin started you can visit <http://your_sandbox_ip:7070/kylin>. The username/password is ADMIN/KYLIN. It's a clean Kylin homepage with nothing in there. To start with you can:
-### If running deploy.sh failed last time
-1. Kill it if any tomcat instance exist
-2. Start over again
+1. [Quick play with a sample cube](../Tutorial/Quick play with a sample cube.md)
+2. [Create and Build your own cube](../Tutorial/Kylin Cube Creation Tutorial.md)
-### If running deploy.sh succeeded last time
-1. Kill it if any tomcat instance exist
-2. Make sure Hbase is running
-3. run `export CATALINA_HOME=~/apache-tomcat-7.0.56`
-4. run `sudo -i "${CATALINA_HOME}/bin/startup.sh"`
\ No newline at end of file
+Here's also a overview introduction on the website [Kylin Web Tutorial](../Tutorial/Kylin Web Tutorial.md)
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/On Hadoop CLI installation.pdf
----------------------------------------------------------------------
diff --git a/docs/Installation/On Hadoop CLI installation.pdf b/docs/Installation/On Hadoop CLI installation.pdf
deleted file mode 100644
index 636555a..0000000
Binary files a/docs/Installation/On Hadoop CLI installation.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/On Hadoop Kylin installation using Docker.md
----------------------------------------------------------------------
diff --git a/docs/Installation/On Hadoop Kylin installation using Docker.md b/docs/Installation/On Hadoop Kylin installation using Docker.md
index 81b2d7f..694b738 100644
--- a/docs/Installation/On Hadoop Kylin installation using Docker.md
+++ b/docs/Installation/On Hadoop Kylin installation using Docker.md
@@ -36,5 +36,5 @@ Use `admin/admin` to login. Make sure HBase is running.
#Kylin Dashboard
http://<container_ip>:7070
```
-The default credentials to login are: `ADMIN:KYLIN`. The cluster is pre-populated with sample data and is ready to build cubes as shown [here](https://github.com/KylinOLAP/Kylin/wiki/Kylin-Cube-Creation-Tutorial).
+The default credentials to login are: `ADMIN:KYLIN`. The cluster is pre-populated with sample data and is ready to build cubes as shown [here](../Tutorial/Kylin Cube Build and Job Monitoring Tutorial.md).
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/On Hadoop Kylin installation using Docker.pdf
----------------------------------------------------------------------
diff --git a/docs/Installation/On Hadoop Kylin installation using Docker.pdf b/docs/Installation/On Hadoop Kylin installation using Docker.pdf
deleted file mode 100644
index a30fe15..0000000
Binary files a/docs/Installation/On Hadoop Kylin installation using Docker.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/install_and_run.md
----------------------------------------------------------------------
diff --git a/docs/Installation/install_and_run.md b/docs/Installation/install_and_run.md
deleted file mode 100644
index 16179fa..0000000
--- a/docs/Installation/install_and_run.md
+++ /dev/null
@@ -1,83 +0,0 @@
-##Install and Run
-
-###How to run
-
-1. Download the release version(according to the hadoop distribution)
-2. Setup a KYLIN_HOME pointing to the corresponding directory where you extract the release tar
-3. Make sure the user has the privilege to run hadoop, hive and hbase cmd in shell. If you are not so sure, you can just run **bin/check-env.sh**, it will print out the detail information if you have some environment issues.
-4. To start Kylin, simply run **bin/kylin.sh start**
-5. To stop Kylin, simply run **bin/kylin.sh stop**
-
-
-If you are running Kylin in a cluster or you have multiple Kylin instances, please make sure you have the following property correctly configured.
-
-1. kylin.rest.servers
-
- List of web servers in use, this enables one web server instance to sync up with other servers.
-
-
-2. kylin.server.mode
-
- Make sure there is only one instance whose "kylin.server.mode" is set to "all" if there are multiple instances.
-
-
-###The directory structure of the installation
-
-> ├── bin
-> │ ├── check-env.sh
-> │ ├── find-hive-dependency.sh
-> │ ├── health-check.sh
-> │ ├── kylin.sh
-> │ └── sample.sh
-> │
-> ├── conf
-> │ ├── kylin_job_conf.xml
-> │ └── kylin.properties
-> │
-> ├── sample_cube
-> │ ├── data
-> │ └── metadata
-> │
-> ├── tomcat
-> │ ├── webapps
-> │ │ └── kylin.war
-> │ …….
-> │
-> └── lib
-> ├── kylin-coprocessor-${version}.jar
-> ├── kylin-job-${version}.jar
-> └── kylin-jdbc-${version}.jar
-
-###Create sample cube
-Kylin provides a script for you to create a sample Cube; the script will also create three sample hive tables:
-
-1. Run ${KYLIN_HOME}/bin/sample.sh
-2. Logon Kylin web, select project "learn_kylin";
-3. Select the sample cube "kylin_sales_cube", click "Actions" -> "Build", pick up a date later than 2014-01-01 (to cover all 10000 sample records);
-4. Check the build progress in "Jobs" tab, until 100%;
-5. Execute SQLs in the "Query" tab, for example:
- select cal_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by cal_dt order by cal_dt
-6. You can verify the query result and compare the response time with hive;
-
-###Build from source
-> git clone https://github.com/KylinOLAP/Kylin.git
-> cd KylinOLAP/Kylin
-> sh script/package.sh
-
-In order to generate binary package, **maven** and **npm** are pre-requisites.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/off_cli_install_scene.png
----------------------------------------------------------------------
diff --git a/docs/Installation/off_cli_install_scene.png b/docs/Installation/off_cli_install_scene.png
new file mode 100644
index 0000000..26e9e4f
Binary files /dev/null and b/docs/Installation/off_cli_install_scene.png differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Installation/on_cli_install_scene.png
----------------------------------------------------------------------
diff --git a/docs/Installation/on_cli_install_scene.png b/docs/Installation/on_cli_install_scene.png
new file mode 100644
index 0000000..5b07d25
Binary files /dev/null and b/docs/Installation/on_cli_install_scene.png differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/README.md
----------------------------------------------------------------------
diff --git a/docs/README.md b/docs/README.md
deleted file mode 100644
index e69de29..0000000
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.pdf b/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.pdf
deleted file mode 100644
index b4253d4..0000000
Binary files a/docs/Tutorial/Kylin Cube Build and Job Monitoring Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Cube Creation Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Cube Creation Tutorial.md b/docs/Tutorial/Kylin Cube Creation Tutorial.md
index cc85075..294f665 100644
--- a/docs/Tutorial/Kylin Cube Creation Tutorial.md
+++ b/docs/Tutorial/Kylin Cube Creation Tutorial.md
@@ -119,4 +119,11 @@ Choose partition type, partition column and start date.
You can overview your cube and go back to previous step to modify it. Click the `Save` button to complete the cube creation.
-
\ No newline at end of file
+
+
+### IV. What's next
+
+After cube being created, you might need to:
+
+1. [Build the cube so that it can be queried](Kylin Cube Build and Job Monitoring Tutorial.md)
+2. [Grant permission to cubes](Kylin Cube Permission Grant Tutorial.md)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Cube Creation Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Cube Creation Tutorial.pdf b/docs/Tutorial/Kylin Cube Creation Tutorial.pdf
deleted file mode 100644
index cd6e934..0000000
Binary files a/docs/Tutorial/Kylin Cube Creation Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Cube Permission Grant Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Cube Permission Grant Tutorial.pdf b/docs/Tutorial/Kylin Cube Permission Grant Tutorial.pdf
deleted file mode 100644
index adfb6d7..0000000
Binary files a/docs/Tutorial/Kylin Cube Permission Grant Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin ODBC Driver Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin ODBC Driver Tutorial.md b/docs/Tutorial/Kylin ODBC Driver Tutorial.md
index 43439fc..7f84213 100644
--- a/docs/Tutorial/Kylin ODBC Driver Tutorial.md
+++ b/docs/Tutorial/Kylin ODBC Driver Tutorial.md
@@ -26,5 +26,12 @@ Kylin ODBC Driver Tutorial
3. Both drivers already be installed on Tableau Server, you properly should be able to publish to there without issues
+## Use ODBC compatible applications
+
+1. [Kylin and Tableau Tutorial](Kylin and Tableau Tutorial.md)
+2. [Kylin and Excel Tutorial](Kylin and Excel Tutorial.md)
+
+
+
## Bug Report
Open github issue here
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin ODBC Driver Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin ODBC Driver Tutorial.pdf b/docs/Tutorial/Kylin ODBC Driver Tutorial.pdf
deleted file mode 100644
index e2d4986..0000000
Binary files a/docs/Tutorial/Kylin ODBC Driver Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Web Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Web Tutorial.md b/docs/Tutorial/Kylin Web Tutorial.md
index 5328370..af49d9b 100644
--- a/docs/Tutorial/Kylin Web Tutorial.md
+++ b/docs/Tutorial/Kylin Web Tutorial.md
@@ -8,7 +8,7 @@ Kylin Web Tutorial
> Mac: Google Chrome, FireFox, Safari
## 1. Access & Login
-Host to access: http://your_sandbox_ip:9080
+Host to access: http://your_sandbox_ip:7070/kylin
Login with username/password: ADMIN/KYLIN
@@ -19,6 +19,9 @@ Although Kylin will using SQL as query interface and leverage Hive metadata, kyl
## 3. Kylin OLAP Cube
+
+> To make cubes availabe you'll have to create them first [Kylin Cube Creation Tutorial](Kylin Cube Creation Tutorial.md)
+
Kylin's OLAP Cubes are pre-calculation datasets from Star Schema Hive tables, Here's the web management interface for user to explorer, manage all cubes.Go to `Cubes` Menu, it will list all cubes available in system:
@@ -42,6 +45,9 @@ To explore more detail about the Cube
## 4. Write and Execute SQL on web
+
+>> To make queries availabe you'll have to create are build cubes first [Kylin Cube Creation Tutorial](Kylin Cube Creation Tutorial.md) and [Kylin Cube Build and Job Monitoring Tutorial](Kylin Cube Build and Job Monitoring Tutorial.md)
+
Kylin's web offer a simple query tool for user to run SQL to explorer existing cube, verify result and explorer the result set using #5's Pivot analysis and visualization
> **Query Limit**
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin Web Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin Web Tutorial.pdf b/docs/Tutorial/Kylin Web Tutorial.pdf
deleted file mode 100644
index 6f27a4b..0000000
Binary files a/docs/Tutorial/Kylin Web Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin and Excel Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin and Excel Tutorial.md b/docs/Tutorial/Kylin and Excel Tutorial.md
new file mode 100644
index 0000000..49961e0
--- /dev/null
+++ b/docs/Tutorial/Kylin and Excel Tutorial.md
@@ -0,0 +1,3 @@
+Kylin and Excel Tutorial
+===
+For now our ODBC driver has not yet finished supporting Excel, watch the mail list for updates.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Kylin and Tableau Tutorial.pdf
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin and Tableau Tutorial.pdf b/docs/Tutorial/Kylin and Tableau Tutorial.pdf
deleted file mode 100644
index 14edbe2..0000000
Binary files a/docs/Tutorial/Kylin and Tableau Tutorial.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/docs/Tutorial/Quick play with a sample cube.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Quick play with a sample cube.md b/docs/Tutorial/Quick play with a sample cube.md
new file mode 100644
index 0000000..cd767b3
--- /dev/null
+++ b/docs/Tutorial/Quick play with a sample cube.md
@@ -0,0 +1,10 @@
+### Quick start with sample cube
+Kylin provides a script for you to create a sample Cube; the script will also create three sample hive tables:
+
+1. Run ${KYLIN_HOME}/bin/sample.sh
+2. Logon Kylin web, select project "learn_kylin";
+3. Select the sample cube "kylin_sales_cube", click "Actions" -> "Build", pick up a date later than 2014-01-01 (to cover all 10000 sample records);
+4. Check the build progress in "Jobs" tab, until 100%;
+5. Execute SQLs in the "Query" tab, for example:
+ select cal_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by cal_dt order by cal_dt
+6. You can verify the query result and compare the response time with hive;
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/592e96b5/old_README.md
----------------------------------------------------------------------
diff --git a/old_README.md b/old_README.md
new file mode 100644
index 0000000..44f6859
--- /dev/null
+++ b/old_README.md
@@ -0,0 +1,94 @@
+Kylin OLAP
+===========
+
+
+Overview
+------------
+> Extreme OLAP Engine for Big Data
+
+An open source distributed Analytical Engine from eBay to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop to support TB to PB size analysis.
+
+
+Getting Started
+---------------
+
+### Prerequisites ###
+
+#### Hadoop ####
+- Hadoop: 2.2.0.2.0.6.0-61 or above
+- Hive: 0.12.0.2.0.6.0-61 or above
+- HBase: 0.96.0.2.0.6.0-61-hadoop2
+
+
+_Tested with Hortonworks HDP 2.1.3 and Cloudera Quickstart VM 5.1._
+
+#### Misc ####
+- Command hadoop, hive, hbase is workable on your hadoop cluster
+- JDK Runtime: JDK7 (OpenJDK or Oracle JDK)
+- Maven
+- Git
+- Tomcat (CATALINA_HOME being set)
+- Npm
+
+#### Before Install ####
+- For the quick tutorial, we assume that your hadoop has priviledges disabled which means any user could run hadoop and hive command
+
+
+Installation
+------------
+It is very easy to install Kylin for exploration/development. There are 3 supported options:
+
+
+1. Sandbox (HDP or CDH sandbox)
+2. Dev Environment (IDE + Sandbox)
+3. Docker container (Docker)
+
+#### Sandbox (On-Hadoop-CLI installation) ####
+
+If you are free to install Kylin on your hadoop CLI machine or Hadoop sandbox, this is the most convenient scenario, for it puts everything in a single machine.
+
+
+
+
+
+For normal users, the application in the above picture means `Kylin Web`, which contains a web interface for cube building, querying and all sorts of management. Kylin Web launches a query engine for querying and a cube build engine for building cubes. These two engines interact with the components in Hadoop CLI, like hive and hbase.
+
+For a hands-on tutorial please visit [On-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-CLI-installation).
+
+#### Dev Environment (Off-Hadoop-CLI Installation) ####
+
+This is typically for development environment setup.
+
+Applications does not necessarily mean Kylin Web, it can also be any test cases or java applications running at your local IDE(query engine and cube build engine can be launched programmatically in your code). In this case, it is no longer reasonable to assume that all the Kylin components reside in the same machine as your Hadoop CLI. Fortunately, Kylin still works under such condition with proper settings on your CLI machine.
+
+
+
+For a hands-on tutorial please visit [Off-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/Off-Hadoop-CLI-Installation-(Dev-Env-Setup))
+
+#### Docker Container ####
+With help from [SequenceIQ](http://sequenceiq.com/), there's docker container for Kylin (along with Hadoop, HBase and Hive) available now:[sequenceiq/docker-kylin](https://github.com/sequenceiq/docker-kylin). The only thing you will need to do is to pull the container from the official Docker repository to be up and running in few minutes.
+
+Features:
+
+- Size - Light weight compared to downloading and setting up HDP or CDH sandbox.
+- Cluster support - Supports multi node installation.
+- Fully Automated - No manual steps. One command does it all
+
+For a hands-on tutorial please visit [Kylin Docker installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-Kylin-installation-using-Docker).
+
+
+
+### Resources ###
+
+* Web Site: <http://kylin.io>
+
+* Google Group: [Kylin OLAP Group](https://groups.google.com/forum/#!forum/kylin-olap)
+
+* Developer Mail: <ky...@googlegroups.com>
+
+* How To Contribute: See [wiki](https://github.com/KylinOLAP/Kylin/wiki/How-to-Contribute)
+
+* Presentation: [Kylin Hadoop OLAP Engine v1.0](https://github.com/KylinOLAP/Kylin/blob/master/docs/Kylin_Hadoop_OLAP_Engine_v1.0.pdf?raw=true)
+
+* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
+
[34/37] incubator-kylin git commit: to prevent from generating *.RSA,
*.SF, *.DSA, the fix for kylin issue 621
Posted by li...@apache.org.
to prevent from generating *.RSA, *.SF, *.DSA, the fix for kylin issue 621
to prevent from generating *.RSA, *.SF, *.DSA, the fix for kylin issue 621
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/df0123bc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/df0123bc
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/df0123bc
Branch: refs/heads/master
Commit: df0123bc84263580334469dc420e96a7700b90de
Parents: 05b0b1a
Author: IT-DONG <wa...@sina.cn>
Authored: Wed Mar 25 18:42:09 2015 +0800
Committer: IT-DONG <wa...@sina.cn>
Committed: Wed Mar 25 18:42:09 2015 +0800
----------------------------------------------------------------------
job/pom.xml | 27 ++++++++++++---------------
1 file changed, 12 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/df0123bc/job/pom.xml
----------------------------------------------------------------------
diff --git a/job/pom.xml b/job/pom.xml
index 1fff5af..bfc73bb 100644
--- a/job/pom.xml
+++ b/job/pom.xml
@@ -261,24 +261,21 @@
</goals>
<configuration>
<minimizeJar>false</minimizeJar>
+ <shadedArtifactAttached>true</shadedArtifactAttached>
+ <shadedClassifierName>job</shadedClassifierName>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>META-INF/*.SF</exclude>
+ <exclude>META-INF/*.DSA</exclude>
+ <exclude>META-INF/*.RSA</exclude>
+ </excludes>
+ </filter>
+ </filters>
</configuration>
</execution>
</executions>
-
- <configuration>
- <shadedArtifactAttached>true</shadedArtifactAttached>
- <shadedClassifierName>job</shadedClassifierName>
- <filters>
- <filter>
- <excludes>
- <exclude>META-INF/*.SF</exclude>
- <exclude>META-INF/*.DSA</exclude>
- <exclude>META-INF/*.RSA</exclude>
- </excludes>
- </filter>
- </filters>
- </configuration>
-
</plugin>
</plugins>
</build>
[37/37] incubator-kylin git commit: refine how to contribute doc
Posted by li...@apache.org.
refine how to contribute doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/23f0dae9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/23f0dae9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/23f0dae9
Branch: refs/heads/master
Commit: 23f0dae956750b46b9b4de07aa1bc4164293b057
Parents: b876b7a
Author: Yang Li <li...@apache.org>
Authored: Sat Mar 28 01:02:51 2015 +0100
Committer: Yang Li <li...@apache.org>
Committed: Sat Mar 28 01:02:51 2015 +0100
----------------------------------------------------------------------
docs/MISC/How to Contribute.md | 43 ++++++++++++++++---------------------
1 file changed, 19 insertions(+), 24 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/23f0dae9/docs/MISC/How to Contribute.md
----------------------------------------------------------------------
diff --git a/docs/MISC/How to Contribute.md b/docs/MISC/How to Contribute.md
index e270606..90c9732 100644
--- a/docs/MISC/How to Contribute.md
+++ b/docs/MISC/How to Contribute.md
@@ -1,41 +1,36 @@
#### Setup Dev Env
-* subscribe our developers' mail list via <de...@kylin.incubator.apache.org>
-* Fork on [GitHub](https://github.com/KylinOLAP)
+* Subscribe our developers' mail list via <de...@kylin.incubator.apache.org>
+* Fork from [GitHub](https://github.com/apache/incubator-kylin) *staging* branch, it is the bug fix branch of master.
* ...
#### Making Changes
-* Raise an issue on GitHub, describe the feature/enhancement/bug
-* Discuss with others in google group or issue comments, make sure the proposed changes fit in with what others are doing and have planned for the project
-* Make changes in your fork
-* Write unit test if no existing cover your change
-* Push to GitHub under your fork
+* Discuss feature/enhancement/bug you plan to do in dev mail list, get consensus before you start.
+* Create a [JIRA](https://issues.apache.org/jira/browse/KYLIN) to describe your task
+* Make changes in your fork.
+* Write unit test to cover your change.
#### Contribute The Work
-* Raise a pull request on GitHub, include both **code** and **test**, link with **related issue**
-* Committer will review in terms of correctness, performance, design, coding style, test coverage
-* Discuss and revise if necessary
-* Finally committer merge code into main branch
+* Pull request to *staging* branch.
+* Committer review in terms of correctness, performance, design, coding style, test coverage.
+* Discuss and revise if necessary.
+* Committer merge code into *staging*.
+* After integration test, change merges into master.
#### Wish List
-Some potential work items
+Some work items for new joiners.
* Query Engine
- * Cache generated class, reduce delay into ms level and avoid full GC triggered by perm generation
- * [Issue #14](https://github.com/KylinOLAP/Kylin/issues/14) Derive meaningful cost in OLAP relational operator
- * [Issue #15](https://github.com/KylinOLAP/Kylin/issues/15) Implement multi-column distinct count
-* Metadata
- * [Issue #7](https://github.com/KylinOLAP/Kylin/issues/7) Merge multiple hbase tables
+ * Cache Calcite generated class, reduce delay into ms level and avoid full GC triggered by perm generation
+ * [KYLIN-491](https://issues.apache.org/jira/browse/KYLIN-491) Derive meaningful cost in OLAP relational operator
* Job Engine
- * [Issue #16](https://github.com/KylinOLAP/Kylin/issues/16) Tune HDFS & HBase parameters
- * [Issue #17](https://github.com/KylinOLAP/Kylin/issues/17) Increase HDFS block size 1GB or close
- * Shell command to support kill operation
- * Use DoubleDouble instead of BigDecimal during cube build
- * Drop quartz dependency, assess the cost/benefit first
- * Cardinality run as one step job, allows progress tracking
+ * [KYLIN-489](https://issues.apache.org/jira/browse/KYLIN-489) Tune HDFS & HBase parameters. Requires a hadoop cluster of 10+ nodes.
+ * Fork some build step (build dictionary, create hbase table) as child process, better resource cleanup and easier troubleshooting.
+ * Use DoubleDouble instead of BigDecimal during cube build. Expect better CPU performance.
* ODBC/JDBC
* Test Kylin remote JDBC with java report tools
- * Implement ODBC async mode, fetching from Kylin and feeding to client in parallel
+ * [KYLIN-602](https://issues.apache.org/jira/browse/KYLIN-602) ODBC driver support excel
+ * Implement ODBC async mode, streaming from Kylin and feeding to client
* Benchmark
* [Issue #18](https://github.com/KylinOLAP/Kylin/issues/18) Benchmark on standard dataset
[18/37] incubator-kylin git commit: KYLIN-650 typo
Posted by li...@apache.org.
KYLIN-650 typo
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/269eac01
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/269eac01
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/269eac01
Branch: refs/heads/master
Commit: 269eac01c35489727ba6ec509b5449d9d4c8dcde
Parents: 0be8930
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:53:26 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:53:26 2015 -0700
----------------------------------------------------------------------
docs/MISC/Kylin Document Convensions.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/269eac01/docs/MISC/Kylin Document Convensions.md
----------------------------------------------------------------------
diff --git a/docs/MISC/Kylin Document Convensions.md b/docs/MISC/Kylin Document Convensions.md
index 52fa6f7..a5b4fa0 100644
--- a/docs/MISC/Kylin Document Convensions.md
+++ b/docs/MISC/Kylin Document Convensions.md
@@ -1,4 +1,4 @@
-Kylin Document Convernsions
+Kylin Document Convensions
-----
1. Keep documentation in markdown files
[03/37] incubator-kylin git commit: change "! -eq" syntax to "!="
since we compare string instead of a number
Posted by li...@apache.org.
change "! -eq" syntax to "!=" since we compare string instead of a number
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/05b0b1a8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/05b0b1a8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/05b0b1a8
Branch: refs/heads/master
Commit: 05b0b1a833793856bacf53bec4ecb56b51a951f2
Parents: 0170668
Author: IT-DONG <wa...@sina.cn>
Authored: Wed Mar 18 18:54:35 2015 +0800
Committer: IT-DONG <wa...@sina.cn>
Committed: Wed Mar 18 18:54:35 2015 +0800
----------------------------------------------------------------------
script/download-tomcat.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/05b0b1a8/script/download-tomcat.sh
----------------------------------------------------------------------
diff --git a/script/download-tomcat.sh b/script/download-tomcat.sh
index 1751541..77be1bf 100755
--- a/script/download-tomcat.sh
+++ b/script/download-tomcat.sh
@@ -10,7 +10,7 @@ then
echo "not binary file found"
wget http://mirror.sdunix.com/apache/tomcat/tomcat-7/v7.0.59/bin/apache-tomcat-7.0.59.tar.gz || echo "download tomcat failed"
else
- if [ ! `md5sum apache-tomcat-7.0.59.tar.gz | awk '{print $1}'` -eq "ec570258976edf9a833cd88fd9220909" ]
+ if [ `md5sum apache-tomcat-7.0.59.tar.gz | awk '{print $1}'` != "ec570258976edf9a833cd88fd9220909" ]
then
echo "md5 check failed"
rm apache-tomcat-7.0.59.tar.gz
@@ -24,4 +24,4 @@ rm -rf tomcat/webapps/*
mv tomcat/conf/server.xml tomcat/conf/server.xml.bak
cp deploy/server.xml tomcat/conf/server.xml
-echo "server.xml overwritten..."
\ No newline at end of file
+echo "server.xml overwritten..."
[27/37] incubator-kylin git commit: Update all ‘kylin_metadata_qa’ to ‘kylin_metadata’
Posted by li...@apache.org.
Update all ‘kylin_metadata_qa’ to ‘kylin_metadata’
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/79f4223b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/79f4223b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/79f4223b
Branch: refs/heads/master
Commit: 79f4223b00a9e4513615a253641e6e57f85d2576
Parents: 41d2d31
Author: Shao Feng, Shi <sh...@ebay.com>
Authored: Mon Mar 23 10:35:59 2015 +0800
Committer: Shao Feng, Shi <sh...@ebay.com>
Committed: Mon Mar 23 10:35:59 2015 +0800
----------------------------------------------------------------------
docs/Operations/Kylin Metadata Store.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/79f4223b/docs/Operations/Kylin Metadata Store.md
----------------------------------------------------------------------
diff --git a/docs/Operations/Kylin Metadata Store.md b/docs/Operations/Kylin Metadata Store.md
index 33c1afe..d9db4c3 100644
--- a/docs/Operations/Kylin Metadata Store.md
+++ b/docs/Operations/Kylin Metadata Store.md
@@ -1,9 +1,9 @@
Kylin organizes all of its metadata(including cube descriptions and instances, projects, inverted index description and instances, jobs, tables and dictionaries) as a hierarchy file system. However, Kylin uses hbase to store it, rather than normal file system. If you check your kylin configuration file(kylin.properties) you will find such a line:
`# The metadata store in hbase`
-`kylin.metadata.url=kylin_metadata_qa@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure`
+`kylin.metadata.url=kylin_metadata@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure`
-This indicates that the metadata will be saved as a htable called `kylin_metadata_qa`. You can scan the htable in hbase shell to check it out.
+This indicates that the metadata will be saved as a htable called `kylin_metadata`. You can scan the htable in hbase shell to check it out.
# Backup Metadata Store
[15/37] incubator-kylin git commit: KYLIN-650 cleaning
Posted by li...@apache.org.
KYLIN-650 cleaning
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/af42057a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/af42057a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/af42057a
Branch: refs/heads/master
Commit: af42057ad2ff01a7ffca32b69091177af9ee80fa
Parents: 5acdcd2
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:27:00 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:27:00 2015 -0700
----------------------------------------------------------------------
README.md | 2 +
docs/JobEngine/Design.md | 2 +-
old_README.md | 94 -------------------------------------------
3 files changed, 3 insertions(+), 95 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/af42057a/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index f3d383f..fba7c0a 100644
--- a/README.md
+++ b/README.md
@@ -48,6 +48,8 @@ Advance Topics
1.[New meta data model structure](docs/TechInside/New meta data model structure.md)
+2.[Job Engine Design](docs/JobEngine/Design.md)
+
Get Help
------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/af42057a/docs/JobEngine/Design.md
----------------------------------------------------------------------
diff --git a/docs/JobEngine/Design.md b/docs/JobEngine/Design.md
index 9c1a94d..59b9ebe 100644
--- a/docs/JobEngine/Design.md
+++ b/docs/JobEngine/Design.md
@@ -4,7 +4,7 @@
### Executable
-
+
**Executable** is a top-level interface for all kinds of jobs or tasks.
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/af42057a/old_README.md
----------------------------------------------------------------------
diff --git a/old_README.md b/old_README.md
deleted file mode 100644
index 44f6859..0000000
--- a/old_README.md
+++ /dev/null
@@ -1,94 +0,0 @@
-Kylin OLAP
-===========
-
-
-Overview
-------------
-> Extreme OLAP Engine for Big Data
-
-An open source distributed Analytical Engine from eBay to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop to support TB to PB size analysis.
-
-
-Getting Started
----------------
-
-### Prerequisites ###
-
-#### Hadoop ####
-- Hadoop: 2.2.0.2.0.6.0-61 or above
-- Hive: 0.12.0.2.0.6.0-61 or above
-- HBase: 0.96.0.2.0.6.0-61-hadoop2
-
-
-_Tested with Hortonworks HDP 2.1.3 and Cloudera Quickstart VM 5.1._
-
-#### Misc ####
-- Command hadoop, hive, hbase is workable on your hadoop cluster
-- JDK Runtime: JDK7 (OpenJDK or Oracle JDK)
-- Maven
-- Git
-- Tomcat (CATALINA_HOME being set)
-- Npm
-
-#### Before Install ####
-- For the quick tutorial, we assume that your hadoop has priviledges disabled which means any user could run hadoop and hive command
-
-
-Installation
-------------
-It is very easy to install Kylin for exploration/development. There are 3 supported options:
-
-
-1. Sandbox (HDP or CDH sandbox)
-2. Dev Environment (IDE + Sandbox)
-3. Docker container (Docker)
-
-#### Sandbox (On-Hadoop-CLI installation) ####
-
-If you are free to install Kylin on your hadoop CLI machine or Hadoop sandbox, this is the most convenient scenario, for it puts everything in a single machine.
-
-
-
-
-
-For normal users, the application in the above picture means `Kylin Web`, which contains a web interface for cube building, querying and all sorts of management. Kylin Web launches a query engine for querying and a cube build engine for building cubes. These two engines interact with the components in Hadoop CLI, like hive and hbase.
-
-For a hands-on tutorial please visit [On-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-CLI-installation).
-
-#### Dev Environment (Off-Hadoop-CLI Installation) ####
-
-This is typically for development environment setup.
-
-Applications does not necessarily mean Kylin Web, it can also be any test cases or java applications running at your local IDE(query engine and cube build engine can be launched programmatically in your code). In this case, it is no longer reasonable to assume that all the Kylin components reside in the same machine as your Hadoop CLI. Fortunately, Kylin still works under such condition with proper settings on your CLI machine.
-
-
-
-For a hands-on tutorial please visit [Off-Hadoop-CLI installation](https://github.com/KylinOLAP/Kylin/wiki/Off-Hadoop-CLI-Installation-(Dev-Env-Setup))
-
-#### Docker Container ####
-With help from [SequenceIQ](http://sequenceiq.com/), there's docker container for Kylin (along with Hadoop, HBase and Hive) available now:[sequenceiq/docker-kylin](https://github.com/sequenceiq/docker-kylin). The only thing you will need to do is to pull the container from the official Docker repository to be up and running in few minutes.
-
-Features:
-
-- Size - Light weight compared to downloading and setting up HDP or CDH sandbox.
-- Cluster support - Supports multi node installation.
-- Fully Automated - No manual steps. One command does it all
-
-For a hands-on tutorial please visit [Kylin Docker installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-Kylin-installation-using-Docker).
-
-
-
-### Resources ###
-
-* Web Site: <http://kylin.io>
-
-* Google Group: [Kylin OLAP Group](https://groups.google.com/forum/#!forum/kylin-olap)
-
-* Developer Mail: <ky...@googlegroups.com>
-
-* How To Contribute: See [wiki](https://github.com/KylinOLAP/Kylin/wiki/How-to-Contribute)
-
-* Presentation: [Kylin Hadoop OLAP Engine v1.0](https://github.com/KylinOLAP/Kylin/blob/master/docs/Kylin_Hadoop_OLAP_Engine_v1.0.pdf?raw=true)
-
-* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
-
[33/37] incubator-kylin git commit: Merge branch 'staging'
Posted by li...@apache.org.
Merge branch 'staging'
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/92ff93e2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/92ff93e2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/92ff93e2
Branch: refs/heads/master
Commit: 92ff93e22cdf3b50a613430fa403a8046535ae6a
Parents: ced5962 cac1589
Author: lukehan <lu...@apache.org>
Authored: Tue Mar 24 18:19:17 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Tue Mar 24 18:19:17 2015 -0700
----------------------------------------------------------------------
docs/website/download/index.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
[12/37] incubator-kylin git commit: KYLIN-650 add operations sections
Posted by li...@apache.org.
KYLIN-650 add operations sections
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/284ef260
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/284ef260
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/284ef260
Branch: refs/heads/master
Commit: 284ef260140d0cae845dfb6075a1e679f9e9300c
Parents: 4bd5407
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:07:09 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:07:09 2015 -0700
----------------------------------------------------------------------
README.md | 14 ++++++++-
.../Advanced settings of Kylin environment.md | 28 ++++++++++++++++++
docs/Operations/Export Kylin HBase data.md | 19 ++++++++++++
docs/Operations/Kylin Metadata Store.md | 31 ++++++++++++++++++++
4 files changed, 91 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/284ef260/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 2489576..fa180b0 100644
--- a/README.md
+++ b/README.md
@@ -17,8 +17,20 @@ Tutorial
Please follow this installation tutorial to start with Kylin: [Installation Tutorial](docs/Installation/Installation.md)
-Advance Usage
+Advance Topics
-------
+####REST
+
+####Operations
+1.[Kylin Metadata Store](docs/Operations/Kylin Metadata Store.md)
+
+2.[Export Kylin HBase data](docs/Operations/Export Kylin HBase data.md)
+
+3.[Advanced settings of Kylin environment](docs/Operations/Advanced settings of Kylin environment.md)
+
+####Test
+
+####Technial Details
Get Help
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/284ef260/docs/Operations/Advanced settings of Kylin environment.md
----------------------------------------------------------------------
diff --git a/docs/Operations/Advanced settings of Kylin environment.md b/docs/Operations/Advanced settings of Kylin environment.md
new file mode 100644
index 0000000..eabfa43
--- /dev/null
+++ b/docs/Operations/Advanced settings of Kylin environment.md
@@ -0,0 +1,28 @@
+Advanced settings of Kylin environment
+----
+
+### A. Enabling LZO compression
+
+LZO compression can be leveraged to compress the output of MR jobs, as well as hbase table storage, reducing the storage overhead. By default we do not enable LZO compression in Kylin because hadoop sandbox venders tend to not include LZO in their distributions due to license(GPL) issues.
+
+To enable LZO in Kylin, there are three steps:
+
+#### Make sure LZO is working in your environment
+
+we have a simple tool to test whether LZO is well installed and configured in your environment(we only test it on the hadoop CLI that you deployed Kylin), Just run
+
+`hbase org.apache.hadoop.util.RunJar kylin-job-latest.jar com.kylinolap.job.tools.LZOSupportnessChecker`
+
+If the program prints "LZO supported by current env? true", you're good to go. Otherwise you'll need to first install LZO properly.
+
+#### Modify kylin_job_conf.xml
+
+You'll need to stop Kylin first by running `${KYLIN_HOME}/bin/kylin.sh stop`, and then modify /etc ${KYLIN_HOME}/conf/kylin_job_conf.xml by uncommenting some configuration entries related to LZO compression.
+
+#### export KYLIN_LD_LIBRARY_PATH to carry your native library paths
+
+Before starting Kylin again, you'll need to uncomment KYLIN_LD_LIBRARY_PATH in ${KYLIN_HOME}/bin/setenv.sh to carry your LZO native library paths. Here's an example for hdp 2.1:
+
+ export KYLIN_LD_LIBRARY_PATH=::/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native
+
+After exporting, you need to run `${KYLIN_HOME}/bin/kylin.sh start` to start Kylin again. Now Kylin will use LZO to compress MR outputs and hbase tables.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/284ef260/docs/Operations/Export Kylin HBase data.md
----------------------------------------------------------------------
diff --git a/docs/Operations/Export Kylin HBase data.md b/docs/Operations/Export Kylin HBase data.md
new file mode 100644
index 0000000..90c4778
--- /dev/null
+++ b/docs/Operations/Export Kylin HBase data.md
@@ -0,0 +1,19 @@
+# Export Kylin HBase data
+
+Kylin persists all data (meta data and cube) in HBase; You may want to export the data sometimes for whatever purposes (backup, migration, troubleshotting etc); This page describes the steps to do this and also there is a Java app for you to do this easily;
+
+Steps:
+
+1. Cleanup unused cubes to save storage space (be cautious on production!): run the following command in hbase CLI:
+```
+hbase org.apache.hadoop.util.RunJar /${KYLIN_HOME}/lib/kylin-job-(version).jar org.apache.kylin.job.hadoop.cube.StorageCleanupJob --delete true
+```
+2. List all HBase tables, iterate and then export each Kylin table to HDFS; See https://hbase.apache.org/book/ops_mgt.html#export
+
+3. Copy the export folder from HDFS to local file system, and then archive it;
+
+4. (optional) Download the archive from Hadoop CLI to local;
+
+5. Cleanup the export folder from CLI HDFS and local file system;
+
+Kylin provide the "ExportHBaseData.java" (currently only exist in "minicluster" branch) for you to do the step 2-5 in one run; Please ensure the correct path of "kylin.properties" has been set in the sys env; This Java uses the sandbox config by default;
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/284ef260/docs/Operations/Kylin Metadata Store.md
----------------------------------------------------------------------
diff --git a/docs/Operations/Kylin Metadata Store.md b/docs/Operations/Kylin Metadata Store.md
new file mode 100644
index 0000000..33c1afe
--- /dev/null
+++ b/docs/Operations/Kylin Metadata Store.md
@@ -0,0 +1,31 @@
+Kylin organizes all of its metadata(including cube descriptions and instances, projects, inverted index description and instances, jobs, tables and dictionaries) as a hierarchy file system. However, Kylin uses hbase to store it, rather than normal file system. If you check your kylin configuration file(kylin.properties) you will find such a line:
+
+`# The metadata store in hbase`
+`kylin.metadata.url=kylin_metadata_qa@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure`
+
+This indicates that the metadata will be saved as a htable called `kylin_metadata_qa`. You can scan the htable in hbase shell to check it out.
+
+# Backup Metadata Store
+
+Sometimes you need to backup the Kylin's Metadata Store from hbase to your disk file system.
+In such cases, assuming you're on the hadoop CLI(or sandbox) where you deployed Kylin, you can use:
+
+ mkdir ~/meta_dump
+
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar com.kylinolap.common.persistence.ResourceTool copy ${KYLIN_HOME}/conf/kylin.properties ~/meta_dump
+
+to dump your metadata to your local folder ~/meta_dump.
+
+# Restore Metadata Store
+
+In case you find your metadata store messed up, and you want to restore to a previous backup:
+
+first clean up the metadata store:
+
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar com.kylinolap.common.persistence.ResourceTool reset
+
+then upload the backup metadata in ~/meta_dump to Kylin's metadata store:
+
+ hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-x.x.x-SNAPSHOT-job.jar com.kylinolap.common.persistence.ResourceTool copy ~/meta_dump ${KYLIN_HOME}/conf/kylin.properties
+
+
[26/37] incubator-kylin git commit: Update kylin.properties for mini
cluster
Posted by li...@apache.org.
Update kylin.properties for mini cluster
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/41d2d319
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/41d2d319
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/41d2d319
Branch: refs/heads/master
Commit: 41d2d31918bf148ff1d51d4e47a6166f82e49ba5
Parents: c5ac4f6
Author: Shao Feng, Shi <sh...@ebay.com>
Authored: Mon Mar 23 10:30:47 2015 +0800
Committer: Shao Feng, Shi <sh...@ebay.com>
Committed: Mon Mar 23 10:30:47 2015 +0800
----------------------------------------------------------------------
.../java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java | 2 +-
examples/test_case_data/minicluster/b-kylin/kylin.properties | 2 +-
examples/test_case_data/minicluster/b-kylin/meta/kylin.properties | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/41d2d319/common/src/test/java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java
----------------------------------------------------------------------
diff --git a/common/src/test/java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java b/common/src/test/java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java
index 3846302..3a13738 100644
--- a/common/src/test/java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java
+++ b/common/src/test/java/org/apache/kylin/common/util/HBaseMiniclusterHelper.java
@@ -38,7 +38,7 @@ public class HBaseMiniclusterHelper {
public static final String SHARED_STORAGE_PREFIX = "KYLIN_";
public static final String CUBE_STORAGE_PREFIX = "KYLIN_";
public static final String II_STORAGE_PREFIX = "KYLIN_II";
- public static final String TEST_METADATA_TABLE = "kylin_metadata_qa";
+ public static final String TEST_METADATA_TABLE = "kylin_metadata";
private static final String hbaseTarLocation = "../examples/test_case_data/minicluster/hbase-export.tar.gz";
private static final String iiEndpointClassName = "org.apache.kylin.storage.hbase.coprocessor.endpoint.IIEndpoint";
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/41d2d319/examples/test_case_data/minicluster/b-kylin/kylin.properties
----------------------------------------------------------------------
diff --git a/examples/test_case_data/minicluster/b-kylin/kylin.properties b/examples/test_case_data/minicluster/b-kylin/kylin.properties
index 5d1b667..f1234a8 100644
--- a/examples/test_case_data/minicluster/b-kylin/kylin.properties
+++ b/examples/test_case_data/minicluster/b-kylin/kylin.properties
@@ -4,7 +4,7 @@
kylin.rest.servers=ADMIN:KYLIN@localhost
# The metadata store in hbase
-kylin.metadata.url=kylin_metadata_qa@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
+kylin.metadata.url=kylin_metadata@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
# The storage for final cube file in hbase
kylin.storage.url=hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/41d2d319/examples/test_case_data/minicluster/b-kylin/meta/kylin.properties
----------------------------------------------------------------------
diff --git a/examples/test_case_data/minicluster/b-kylin/meta/kylin.properties b/examples/test_case_data/minicluster/b-kylin/meta/kylin.properties
index 17e5c99..642f845 100644
--- a/examples/test_case_data/minicluster/b-kylin/meta/kylin.properties
+++ b/examples/test_case_data/minicluster/b-kylin/meta/kylin.properties
@@ -4,7 +4,7 @@
kylin.rest.servers=ADMIN:KYLIN@localhost
# The metadata store in hbase
-kylin.metadata.url=kylin_metadata_qa@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
+kylin.metadata.url=kylin_metadata@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
# The storage for final cube file in hbase
kylin.storage.url=hbase:sandbox.hortonworks.com:2181:/hbase-unsecure
[06/37] incubator-kylin git commit: KYLIN-650 updates
Posted by li...@apache.org.
KYLIN-650 updates
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/da16316a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/da16316a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/da16316a
Branch: refs/heads/master
Commit: da16316a54f6f04c91ad62d104bea6844c533a49
Parents: 592e96b
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 19:04:45 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 19:04:45 2015 -0700
----------------------------------------------------------------------
docs/Installation/Installation.md | 2 +-
docs/Installation/Off Hadoop CLI Installation.md | 2 +-
docs/Tutorial/Quick play with a sample cube.md | 10 +++++++++-
3 files changed, 11 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/da16316a/docs/Installation/Installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Installation.md b/docs/Installation/Installation.md
index 5b33a56..2fa345c 100644
--- a/docs/Installation/Installation.md
+++ b/docs/Installation/Installation.md
@@ -32,7 +32,7 @@ Features:
- Cluster support - Supports multi node installation.
- Fully Automated - No manual steps. One command does it all
-For a hands-on tutorial please visit [Kylin Docker installation](https://github.com/KylinOLAP/Kylin/wiki/On-Hadoop-Kylin-installation-using-Docker).
+For a hands-on tutorial please visit [Kylin Docker installation](On Hadoop Kylin installation using Docker.md).
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/da16316a/docs/Installation/Off Hadoop CLI Installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Off Hadoop CLI Installation.md b/docs/Installation/Off Hadoop CLI Installation.md
index d63bf13..f83ed5d 100644
--- a/docs/Installation/Off Hadoop CLI Installation.md
+++ b/docs/Installation/Off Hadoop CLI Installation.md
@@ -63,5 +63,5 @@ Run other tests, the end-to-end cube building test is exclueded
In your Eclipse IDE, launch `org.apache.kylin.rest.DebugTomcat` with specifying VM arguments "-Dspring.profiles.active=sandbox". (By default Kylin server will listen on 7070 port; If you want to use another port, please specify it as a parameter when run `DebugTomcat)
-Check Kylin Web available at http://localhost:7070 (user:ADMIN,password:KYLIN)
+Check Kylin Web available at http://localhost:7070/kylin (user:ADMIN,password:KYLIN)
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/da16316a/docs/Tutorial/Quick play with a sample cube.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Quick play with a sample cube.md b/docs/Tutorial/Quick play with a sample cube.md
index cd767b3..37c44ff 100644
--- a/docs/Tutorial/Quick play with a sample cube.md
+++ b/docs/Tutorial/Quick play with a sample cube.md
@@ -1,4 +1,5 @@
### Quick start with sample cube
+
Kylin provides a script for you to create a sample Cube; the script will also create three sample hive tables:
1. Run ${KYLIN_HOME}/bin/sample.sh
@@ -7,4 +8,11 @@ Kylin provides a script for you to create a sample Cube; the script will also cr
4. Check the build progress in "Jobs" tab, until 100%;
5. Execute SQLs in the "Query" tab, for example:
select cal_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by cal_dt order by cal_dt
-6. You can verify the query result and compare the response time with hive;
\ No newline at end of file
+6. You can verify the query result and compare the response time with hive;
+
+#### What's next
+
+After cube being created, you might need to:
+
+1. [Build the cube so that it can be queried](Kylin Cube Build and Job Monitoring Tutorial.md)
+2. [Grant permission to cubes](Kylin Cube Permission Grant Tutorial.md)
\ No newline at end of file
[11/37] incubator-kylin git commit: KYLIN-650 minor change
Posted by li...@apache.org.
KYLIN-650 minor change
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/4bd54078
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/4bd54078
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/4bd54078
Branch: refs/heads/master
Commit: 4bd54078af6dd7c60c4bb5eb8aab4a99e2a6d5f3
Parents: eff1393
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 22:27:23 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 22:27:23 2015 -0700
----------------------------------------------------------------------
README.md | 12 ++++--
...requently Asked Questions on Installation.md | 26 ------------
.../MISC/FAQ on Kylin Installation and Usage.md | 40 +++++++++++++++++++
docs/MISC/How to Contribute.md | 42 ++++++++++++++++++++
4 files changed, 90 insertions(+), 30 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/4bd54078/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 232758c..2489576 100644
--- a/README.md
+++ b/README.md
@@ -17,6 +17,10 @@ Tutorial
Please follow this installation tutorial to start with Kylin: [Installation Tutorial](docs/Installation/Installation.md)
+Advance Usage
+-------
+
+
Get Help
------------
@@ -26,13 +30,13 @@ The fastest way to get response from our developers is to send email to our mail
Resources
------------
-* Web Site: <http://kylin.io>
+* [FAQ](docs/MISC/FAQ on Kylin Installation and Usage.md)
-* Developer Mail: <de...@kylin.incubator.apache.org>
+* Web Site: <http://kylin.incubator.apache.org/>
-* How To Contribute: See [wiki](https://github.com/KylinOLAP/Kylin/wiki/How-to-Contribute)
+* Developer Mail: <de...@kylin.incubator.apache.org>
-* Presentation: [Kylin Hadoop OLAP Engine v1.0](https://github.com/KylinOLAP/Kylin/blob/master/docs/Kylin_Hadoop_OLAP_Engine_v1.0.pdf?raw=true)
+* How To Contribute: See [this](docs/MISC/How to Contribute.md)
* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/4bd54078/docs/Installation/Frequently Asked Questions on Installation.md
----------------------------------------------------------------------
diff --git a/docs/Installation/Frequently Asked Questions on Installation.md b/docs/Installation/Frequently Asked Questions on Installation.md
deleted file mode 100644
index c160403..0000000
--- a/docs/Installation/Frequently Asked Questions on Installation.md
+++ /dev/null
@@ -1,26 +0,0 @@
-Frequently Asked Questions on Installation
----
-* Some NPM error causes ERROR exit (中国大陆地区用户请特别注意此问题)?
-> Check out https://github.com/KylinOLAP/Kylin/issues/35
-
-* Can't get master address from ZooKeeper" when installing Kylin on Hortonworks Sandbox
-> Check out https://github.com/KylinOLAP/Kylin/issues/9.
-
-* Install scripted finished in my virtual machine, but cannot visit via http://localhost:9080
-> Check out https://github.com/KylinOLAP/Kylin/issues/12.
-
-* Map Reduce Job information can't display on sandbox deployment
-> Check out https://github.com/KylinOLAP/Kylin/issues/40
-
-* Install Kylin on CDH 5.2 or Hadoop 2.5.x
-> Check out discussion: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/kylin-olap/X0GZfsX1jLc/nzs6xAhNpLkJ
->
-```
-I was able to deploy Kylin with following option in POM.
-<hadoop2.version>2.5.0</hadoop2.version>
-<yarn.version>2.5.0</yarn.version>
-<hbase-hadoop2.version>0.98.6-hadoop2</hbase-hadoop2.version>
-<zookeeper.version>3.4.5</zookeeper.version>
-<hive.version>0.13.1</hive.version>
-My Cluster is running on Cloudera Distribution CDH 5.2.0.
-```
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/4bd54078/docs/MISC/FAQ on Kylin Installation and Usage.md
----------------------------------------------------------------------
diff --git a/docs/MISC/FAQ on Kylin Installation and Usage.md b/docs/MISC/FAQ on Kylin Installation and Usage.md
new file mode 100644
index 0000000..1c1291e
--- /dev/null
+++ b/docs/MISC/FAQ on Kylin Installation and Usage.md
@@ -0,0 +1,40 @@
+FAQ on Kylin Installation and Usage
+---
+#### Some NPM error causes ERROR exit (中国大陆地区用户请特别注意此问题)?
+Check out https://github.com/KylinOLAP/Kylin/issues/35
+
+#### Can't get master address from ZooKeeper" when installing Kylin on Hortonworks Sandbox
+Check out https://github.com/KylinOLAP/Kylin/issues/9.
+
+#### Install scripted finished in my virtual machine, but cannot visit via http://localhost:9080
+Check out https://github.com/KylinOLAP/Kylin/issues/12.
+
+#### Map Reduce Job information can't display on sandbox deployment
+Check out https://github.com/KylinOLAP/Kylin/issues/40
+
+#### Install Kylin on CDH 5.2 or Hadoop 2.5.x
+Check out discussion: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/kylin-olap/X0GZfsX1jLc/nzs6xAhNpLkJ
+```
+I was able to deploy Kylin with following option in POM.
+<hadoop2.version>2.5.0</hadoop2.version>
+<yarn.version>2.5.0</yarn.version>
+<hbase-hadoop2.version>0.98.6-hadoop2</hbase-hadoop2.version>
+<zookeeper.version>3.4.5</zookeeper.version>
+<hive.version>0.13.1</hive.version>
+My Cluster is running on Cloudera Distribution CDH 5.2.0.
+```
+
+#### Unable to load a big cube as HTable, with java.lang.OutOfMemoryError: unable to create new native thread
+HBase (as of writing) allocates one thread per region when bulk loading a HTable. Try reduce the number of regions of your cube by setting its "capacity" to "MEDIUM" or "LARGE". Also tweaks OS & JVM can allow more threads, for example see [this article](http://blog.egilh.com/2006/06/2811aspx.html).
+
+#### Failed to run BuildCubeWithEngineTest, saying failed to connect to hbase while hbase is active
+User may get this error when first time run hbase client, please check the error trace to see whether there is an error saying couldn't access a folder like "/hadoop/hbase/local/jars"; If that folder doesn't exist, create it.
+
+#### SUM(field) returns a negtive result while all the numbers in this field are > 0
+If a column is declared as integer in Hive, the SQL engine (calcite) will use column's type (integer) as the data type for "SUM(field)", while the aggregated value on this field may exceed the scope of integer; in that case the cast will cause a negtive value be returned; The workround is, alter that column's type to BIGINT in hive, and then sync the table schema to Kylin (the cube doesn't need rebuild); Keep in mind that, always declare as BIGINT in hive for an integer column which would be used as a measure in Kylin; See hive number types: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-NumericTypes
+
+#### Why Kylin need extract the distinct columns from Fact Table before building cube?
+Kylin uses dictionary to encode the values in each column, this greatly reduce the cube's storage size. To build the dictionary, Kylin need fetch the distinct values for each column.
+
+#### Why Kylin calculate the HIVE table cardinality?
+The cardinality of dimensions is an important measure of cube complexity. The higher the cardinality, the bigger the cube, and thus the longer to build and the slower to query. Cardinality > 1,000 is worth attention and > 1,000,000 should be avoided at best effort. For optimal cube performance, try reduce high cardinality by categorize values or derive features.
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/4bd54078/docs/MISC/How to Contribute.md
----------------------------------------------------------------------
diff --git a/docs/MISC/How to Contribute.md b/docs/MISC/How to Contribute.md
new file mode 100644
index 0000000..e270606
--- /dev/null
+++ b/docs/MISC/How to Contribute.md
@@ -0,0 +1,42 @@
+#### Setup Dev Env
+* subscribe our developers' mail list via <de...@kylin.incubator.apache.org>
+* Fork on [GitHub](https://github.com/KylinOLAP)
+* ...
+
+#### Making Changes
+* Raise an issue on GitHub, describe the feature/enhancement/bug
+* Discuss with others in google group or issue comments, make sure the proposed changes fit in with what others are doing and have planned for the project
+* Make changes in your fork
+* Write unit test if no existing cover your change
+* Push to GitHub under your fork
+
+
+#### Contribute The Work
+* Raise a pull request on GitHub, include both **code** and **test**, link with **related issue**
+* Committer will review in terms of correctness, performance, design, coding style, test coverage
+* Discuss and revise if necessary
+* Finally committer merge code into main branch
+
+
+#### Wish List
+Some potential work items
+* Query Engine
+ * Cache generated class, reduce delay into ms level and avoid full GC triggered by perm generation
+ * [Issue #14](https://github.com/KylinOLAP/Kylin/issues/14) Derive meaningful cost in OLAP relational operator
+ * [Issue #15](https://github.com/KylinOLAP/Kylin/issues/15) Implement multi-column distinct count
+* Metadata
+ * [Issue #7](https://github.com/KylinOLAP/Kylin/issues/7) Merge multiple hbase tables
+* Job Engine
+ * [Issue #16](https://github.com/KylinOLAP/Kylin/issues/16) Tune HDFS & HBase parameters
+ * [Issue #17](https://github.com/KylinOLAP/Kylin/issues/17) Increase HDFS block size 1GB or close
+ * Shell command to support kill operation
+ * Use DoubleDouble instead of BigDecimal during cube build
+ * Drop quartz dependency, assess the cost/benefit first
+ * Cardinality run as one step job, allows progress tracking
+* ODBC/JDBC
+ * Test Kylin remote JDBC with java report tools
+ * Implement ODBC async mode, fetching from Kylin and feeding to client in parallel
+* Benchmark
+ * [Issue #18](https://github.com/KylinOLAP/Kylin/issues/18) Benchmark on standard dataset
+
+
[20/37] incubator-kylin git commit: KYLIN-648, refine test sql
Posted by li...@apache.org.
KYLIN-648, refine test sql
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/3563a914
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/3563a914
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/3563a914
Branch: refs/heads/master
Commit: 3563a91469619dc1ca22816175e26e22557694ce
Parents: c66a74b
Author: Li, Yang <ya...@ebay.com>
Authored: Fri Mar 20 15:31:37 2015 +0800
Committer: Li, Yang <ya...@ebay.com>
Committed: Fri Mar 20 15:31:37 2015 +0800
----------------------------------------------------------------------
query/src/test/resources/query/sql/query78.sql | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/3563a914/query/src/test/resources/query/sql/query78.sql
----------------------------------------------------------------------
diff --git a/query/src/test/resources/query/sql/query78.sql b/query/src/test/resources/query/sql/query78.sql
index f1d8fb2..0da9611 100644
--- a/query/src/test/resources/query/sql/query78.sql
+++ b/query/src/test/resources/query/sql/query78.sql
@@ -1,4 +1,4 @@
-select sum(PRICE), LSTG_FORMAT_NAME
+select sum(PRICE) as GMV, LSTG_FORMAT_NAME
from test_kylin_fact
where (LSTG_FORMAT_NAME in ('ABIN')) or (LSTG_FORMAT_NAME>='FP-GTC' and LSTG_FORMAT_NAME<='Others')
group by LSTG_FORMAT_NAME
[32/37] incubator-kylin git commit: Merge branch 'master' of
https://github.com/KylinOLAP/Kylin
Posted by li...@apache.org.
Merge branch 'master' of https://github.com/KylinOLAP/Kylin
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/ced5962b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/ced5962b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/ced5962b
Branch: refs/heads/master
Commit: ced5962bdad0bdef5552e2cc5eb755709aea9940
Parents: b6a85cf b1b3fad
Author: lukehan <lu...@apache.org>
Authored: Tue Mar 24 18:19:09 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Tue Mar 24 18:19:09 2015 -0700
----------------------------------------------------------------------
----------------------------------------------------------------------
[13/37] incubator-kylin git commit: KYLIN-650 category resting pages
to different topics
Posted by li...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/Test/Run Kylin test case with HBase Mini Cluster.md
----------------------------------------------------------------------
diff --git a/docs/Test/Run Kylin test case with HBase Mini Cluster.md b/docs/Test/Run Kylin test case with HBase Mini Cluster.md
new file mode 100644
index 0000000..6088eb9
--- /dev/null
+++ b/docs/Test/Run Kylin test case with HBase Mini Cluster.md
@@ -0,0 +1,15 @@
+## Run test case with HBase Mini Cluster
+
+Kylin is moving as many as possible unit test cases from sandbox to HBase mini cluster, so that user can run tests easily in local without a hadoop sandbox; Two maven profiles are created in the root pom.xml, "default" and "sandbox"; The default profile will startup a HBase Mini Cluster to prepare the test data and run the unit tests (the test cases that are not supported by Mini cluster have been added in the "exclude" list); If you want to keep using Sandbox to run test, just run:
+mvn test -P sandbox
+
+
+### When use the "default" profile, Kylin will:
+
+* Startup a HBase minicluster and update KylinConfig with the dynamic HBase configurations;
+* Create Kylin metadata tables and import six example cube tables;
+* Import the hbase data from a tar ball from local: examples/test_case_data/minicluster/hbase-export.tar.gz (the hbase-export.tar.gz will be updated on complete of running BuildCubeWithEngineTest）
+* After all test cases be completed, shutdown minicluster and cleanup KylinConfig cache;
+
+### To ensure Mini cluster can run successfully, you need:
+* Make sure JAVA_HOME is properly set;
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/Tutorial/How to use kylin remote jdbc driver.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/How to use kylin remote jdbc driver.md b/docs/Tutorial/How to use kylin remote jdbc driver.md
new file mode 100644
index 0000000..45de447
--- /dev/null
+++ b/docs/Tutorial/How to use kylin remote jdbc driver.md
@@ -0,0 +1,83 @@
+### Authentication
+Build on kylin authentication restful service. Supported parameters:
+* user : username
+* password : password
+* ssl: true/false. Default be false; If true, all the services call will use https.
+
+### Connection URL format:
+```
+jdbc:kylin://<hostname>:<port>/<kylin_project_name>
+```
+* If "ssl" = true, the "port" should be Kylin server's HTTPS port;
+* If "port" is not specified, the driver will use default port: HTTP 80, HTTPS 443;
+* The "kylin_project_name" must be specified and user need ensure it exists in Kylin server;
+
+### 1. Query with Statement
+```
+ Driver driver = (Driver) Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
+
+ Properties info = new Properties();
+ info.put("user", "ADMIN");
+ info.put("password", "KYLIN");
+ Connection conn = driver.connect("jdbc:kylin://localhost:7070/kylin_project_name", info);
+ Statement state = conn.createStatement();
+ ResultSet resultSet = state.executeQuery("select * from test_table");
+
+ while (resultSet.next()) {
+ assertEquals("foo", resultSet.getString(1));
+ assertEquals("bar", resultSet.getString(2));
+ assertEquals("tool", resultSet.getString(3));
+ }
+```
+
+### 2. Query with PreparedStatement
+Supported prepared statement parameters:
+* setString
+* setInt
+* setShort
+* setLong
+* setFloat
+* setDouble
+* setBoolean
+* setByte
+* setDate
+* setTime
+* setTimestamp
+
+```
+ Driver driver = (Driver) Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
+ Properties info = new Properties();
+ info.put("user", "ADMIN");
+ info.put("password", "KYLIN");
+ Connection conn = driver.connect("jdbc:kylin://localhost:7070/kylin_project_name", info);
+ PreparedStatement state = conn.prepareStatement("select * from test_table where id=?");
+ state.setInt(1, 10);
+ ResultSet resultSet = state.executeQuery();
+
+ while (resultSet.next()) {
+ assertEquals("foo", resultSet.getString(1));
+ assertEquals("bar", resultSet.getString(2));
+ assertEquals("tool", resultSet.getString(3));
+ }
+```
+
+### 3. Get query result metadata
+Kylin jdbc driver supports metadata list methods:
+List catalog, schema, table and column with sql pattern filters(such as %).
+
+```
+ Driver driver = (Driver) Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
+ Properties info = new Properties();
+ info.put("user", "ADMIN");
+ info.put("password", "KYLIN");
+ Connection conn = driver.connect("jdbc:kylin://localhost:7070/kylin_project_name", info);
+ Statement state = conn.createStatement();
+ ResultSet resultSet = state.executeQuery("select * from test_table");
+
+ ResultSet tables = conn.getMetaData().getTables(null, null, "dummy", null);
+ while (tables.next()) {
+ for (int i = 0; i < 10; i++) {
+ assertEquals("dummy", tables.getString(i + 1));
+ }
+ }
+```
[31/37] incubator-kylin git commit: update binary package with commit
6f1d30fcab48fd02e18f91386b41c12bfd188178
Posted by li...@apache.org.
update binary package with commit 6f1d30fcab48fd02e18f91386b41c12bfd188178
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/cac1589c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/cac1589c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/cac1589c
Branch: refs/heads/master
Commit: cac1589c639e53d3d3b65f64dccb043a57de463a
Parents: f9e3a6a
Author: lukehan <lu...@apache.org>
Authored: Tue Mar 24 18:17:25 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Tue Mar 24 18:17:25 2015 -0700
----------------------------------------------------------------------
docs/website/download/index.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/cac1589c/docs/website/download/index.md
----------------------------------------------------------------------
diff --git a/docs/website/download/index.md b/docs/website/download/index.md
index 9718e8b..a2cc3e5 100644
--- a/docs/website/download/index.md
+++ b/docs/website/download/index.md
@@ -15,8 +15,8 @@ title: Community
<p class="big_text">Binary Package (v0.7.x)</p>
<p><a href="http://kylin.incubator.apache.org/download/kylin-0.7.1-SNAPSHOT.tgz" target="_blank">v0.7.1-SNAPSHOT (Dev Build)</a>
- <br/>Updated Date - March 16, 2015 <br/>
- <br/>Git Commit: <a href="https://github.com/KylinOLAP/Kylin/tree/4f6869e1f1f6b2ed22e6d3b638a4048905d27e49" target="_blank">4f6869e</a>
+ <br/>Updated Date - March 24, 2015 <br/>
+ <br/>Git Commit: <a href="https://github.com/KylinOLAP/Kylin/tree/6f1d30fcab48fd02e18f91386b41c12bfd188178" target="_blank">6f1d30f</a>
</p>
</div>
[25/37] incubator-kylin git commit: Update kylin.properties for mini
cluster
Posted by li...@apache.org.
Update kylin.properties for mini cluster
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/c5ac4f61
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/c5ac4f61
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/c5ac4f61
Branch: refs/heads/master
Commit: c5ac4f61c272dc52727344669ccf23c3dd070b88
Parents: bed423f
Author: Shao Feng, Shi <sh...@ebay.com>
Authored: Mon Mar 23 10:22:09 2015 +0800
Committer: Shao Feng, Shi <sh...@ebay.com>
Committed: Mon Mar 23 10:22:09 2015 +0800
----------------------------------------------------------------------
examples/test_case_data/minicluster/kylin.properties | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/c5ac4f61/examples/test_case_data/minicluster/kylin.properties
----------------------------------------------------------------------
diff --git a/examples/test_case_data/minicluster/kylin.properties b/examples/test_case_data/minicluster/kylin.properties
index 373186c..736de65 100644
--- a/examples/test_case_data/minicluster/kylin.properties
+++ b/examples/test_case_data/minicluster/kylin.properties
@@ -4,7 +4,7 @@
kylin.rest.servers=ADMIN:KYLIN@localhost
# The metadata store in hbase
-kylin.metadata.url=kylin_metadata_qa@hbase:minicluster:/hbase-unsecure
+kylin.metadata.url=kylin_metadata@hbase:minicluster:/hbase-unsecure
# The storage for final cube file in hbase
kylin.storage.url=hbase:minicluster:/hbase-unsecure
[29/37] incubator-kylin git commit: Merge branch 'staging' of
https://github.com/KylinOLAP/Kylin into staging
Posted by li...@apache.org.
Merge branch 'staging' of https://github.com/KylinOLAP/Kylin into staging
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/f9e3a6a0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/f9e3a6a0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/f9e3a6a0
Branch: refs/heads/master
Commit: f9e3a6a07e0ed5e5abb03a95bdd2112383fadf9a
Parents: 3625d53 6f1d30f
Author: lukehan <lu...@apache.org>
Authored: Tue Mar 24 18:09:16 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Tue Mar 24 18:09:16 2015 -0700
----------------------------------------------------------------------
README.md | 88 +-
bin/kylin.sh | 16 +-
bin/setenv.sh | 36 +
.../common/util/HBaseMiniclusterHelper.java | 2 +-
.../java/org/apache/kylin/cube/CubeManager.java | 50 +-
.../org/apache/kylin/cube/model/CubeDesc.java | 4 +
...requently Asked Questions on Installation.md | 26 -
...equently Asked Questions on Installation.pdf | Bin 52261 -> 0 bytes
docs/Installation/Hadoop Environment.md | 21 +
.../How to generate install binaries.md | 6 +
docs/Installation/Installation.md | 38 +
.../Installation/Multiple Kylin REST servers.md | 9 +
...f Hadoop CLI Installation (Dev Env Setup).md | 76 -
... Hadoop CLI Installation (Dev Env Setup).pdf | Bin 44909 -> 0 bytes
.../Installation/Off Hadoop CLI Installation.md | 67 +
docs/Installation/On Hadoop CLI installation.md | 106 +-
.../Installation/On Hadoop CLI installation.pdf | Bin 704398 -> 0 bytes
...On Hadoop Kylin installation using Docker.md | 2 +-
...n Hadoop Kylin installation using Docker.pdf | Bin 40098 -> 0 bytes
docs/Installation/install_and_run.md | 83 -
docs/Installation/off_cli_install_scene.png | Bin 0 -> 72567 bytes
docs/Installation/on_cli_install_scene.png | Bin 0 -> 53722 bytes
docs/JobEngine/Design.md | 2 +-
.../MISC/FAQ on Kylin Installation and Usage.md | 43 +
docs/MISC/How to Contribute.md | 42 +
docs/MISC/Kylin Document Conventions.md | 9 +
.../Advanced settings of Kylin environment.md | 28 +
docs/Operations/Export Kylin HBase data.md | 19 +
docs/Operations/Kylin Metadata Store.md | 31 +
docs/README.md | 0
docs/REST/Build Cube with Restful API.md | 51 +
...w to consume Kylin REST API in javascript.md | 42 +
docs/REST/Kylin Restful API List.md | 2020 ++++++++++++++++++
.../TechInside/New meta data model structure.md | 18 +
...n Kylin test case with HBase Mini Cluster.md | 15 +
.../How to use kylin remote jdbc driver.md | 87 +
...in Cube Build and Job Monitoring Tutorial.md | 10 +-
...n Cube Build and Job Monitoring Tutorial.pdf | Bin 1054988 -> 0 bytes
docs/Tutorial/Kylin Cube Creation Tutorial.md | 9 +-
docs/Tutorial/Kylin Cube Creation Tutorial.pdf | Bin 1488946 -> 0 bytes
.../Kylin Cube Permission Grant Tutorial.pdf | Bin 358047 -> 0 bytes
docs/Tutorial/Kylin ODBC Driver Tutorial.md | 6 +
docs/Tutorial/Kylin ODBC Driver Tutorial.pdf | Bin 34429 -> 0 bytes
docs/Tutorial/Kylin Web Tutorial.md | 8 +-
docs/Tutorial/Kylin Web Tutorial.pdf | Bin 1714951 -> 0 bytes
docs/Tutorial/Kylin and Excel Tutorial.md | 3 +
docs/Tutorial/Kylin and Tableau Tutorial.md | 6 +-
docs/Tutorial/Kylin and Tableau Tutorial.pdf | Bin 1246553 -> 0 bytes
docs/Tutorial/Quick play with a sample cube.md | 20 +
.../minicluster/b-kylin/kylin.properties | 2 +-
.../minicluster/b-kylin/meta/kylin.properties | 2 +-
.../test_case_data/minicluster/kylin.properties | 2 +-
.../org/apache/kylin/jdbc/stub/KylinClient.java | 8 +-
.../org/apache/kylin/job/JoinedFlatTable.java | 46 +-
.../kylin/job/hadoop/AbstractHadoopJob.java | 3 +-
.../hadoop/hive/CubeJoinedFlatTableDesc.java | 29 -
.../job/hadoop/hive/IIJoinedFlatTableDesc.java | 20 -
.../job/hadoop/hive/IJoinedFlatTableDesc.java | 5 -
.../kylin/job/tools/CubeMigrationCLI.java | 4 +-
.../job/hadoop/cube/BaseCuboidJobTest.java | 3 +
.../job/hadoop/cube/MergeCuboidJobTest.java | 4 +-
.../kylin/job/hadoop/cube/NDCuboidJobTest.java | 4 +-
.../cube/RangeKeyDistributionJobTest.java | 4 +-
.../kylin/metadata/project/ProjectL2Cache.java | 27 +-
pom.xml | 4 +
query/src/test/resources/query/sql/query78.sql | 4 +
.../apache/kylin/rest/service/JobService.java | 110 +-
.../kylin/storage/hbase/CubeStorageEngine.java | 15 +-
webapp/app/js/controllers/cubes.js | 2 +-
webapp/app/partials/query/query_detail.html | 2 +-
70 files changed, 2871 insertions(+), 528 deletions(-)
----------------------------------------------------------------------
[23/37] incubator-kylin git commit: KYLIN-650 add Kylin vs. Druid
Posted by li...@apache.org.
KYLIN-650 add Kylin vs. Druid
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/8934297a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/8934297a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/8934297a
Branch: refs/heads/master
Commit: 8934297a436cb22e9a5c40c34a01217a880ede14
Parents: 3655b33
Author: honma <ho...@ebay.com>
Authored: Sat Mar 21 10:32:12 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Sat Mar 21 10:32:12 2015 -0700
----------------------------------------------------------------------
docs/MISC/FAQ on Kylin Installation and Usage.md | 3 +++
1 file changed, 3 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/8934297a/docs/MISC/FAQ on Kylin Installation and Usage.md
----------------------------------------------------------------------
diff --git a/docs/MISC/FAQ on Kylin Installation and Usage.md b/docs/MISC/FAQ on Kylin Installation and Usage.md
index 1c1291e..a99e9f0 100644
--- a/docs/MISC/FAQ on Kylin Installation and Usage.md
+++ b/docs/MISC/FAQ on Kylin Installation and Usage.md
@@ -38,3 +38,6 @@ Kylin uses dictionary to encode the values in each column, this greatly reduce t
#### Why Kylin calculate the HIVE table cardinality?
The cardinality of dimensions is an important measure of cube complexity. The higher the cardinality, the bigger the cube, and thus the longer to build and the slower to query. Cardinality > 1,000 is worth attention and > 1,000,000 should be avoided at best effort. For optimal cube performance, try reduce high cardinality by categorize values or derive features.
+
+#### What is the difference between Kylin and Druid
+take a look at http://mail-archives.apache.org/mod_mbox/incubator-kylin-dev/201503.mbox/browser
[28/37] incubator-kylin git commit: amend JDBC doc
Posted by li...@apache.org.
amend JDBC doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/6f1d30fc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/6f1d30fc
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/6f1d30fc
Branch: refs/heads/master
Commit: 6f1d30fcab48fd02e18f91386b41c12bfd188178
Parents: 79f4223
Author: Li, Yang <ya...@ebay.com>
Authored: Mon Mar 23 12:03:04 2015 +0800
Committer: Li, Yang <ya...@ebay.com>
Committed: Mon Mar 23 12:03:04 2015 +0800
----------------------------------------------------------------------
docs/Tutorial/How to use kylin remote jdbc driver.md | 4 ++++
1 file changed, 4 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6f1d30fc/docs/Tutorial/How to use kylin remote jdbc driver.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/How to use kylin remote jdbc driver.md b/docs/Tutorial/How to use kylin remote jdbc driver.md
index 45de447..1d0584f 100644
--- a/docs/Tutorial/How to use kylin remote jdbc driver.md
+++ b/docs/Tutorial/How to use kylin remote jdbc driver.md
@@ -1,3 +1,7 @@
+### Get JDBC jar
+
+Kylin JDBC jar file is under lib/ of binary package, named `kylin-jdbc-verion.jar`.
+
### Authentication
Build on kylin authentication restful service. Supported parameters:
* user : username
[21/37] incubator-kylin git commit: KYLIN-648, refine test sql
Posted by li...@apache.org.
KYLIN-648, refine test sql
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/222b1440
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/222b1440
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/222b1440
Branch: refs/heads/master
Commit: 222b1440a83a1990c56b2441b0fb3a905204510c
Parents: c66a74b
Author: Li, Yang <ya...@ebay.com>
Authored: Fri Mar 20 15:31:37 2015 +0800
Committer: Li, Yang <ya...@ebay.com>
Committed: Fri Mar 20 15:44:28 2015 +0800
----------------------------------------------------------------------
query/src/test/resources/query/sql/query78.sql | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/222b1440/query/src/test/resources/query/sql/query78.sql
----------------------------------------------------------------------
diff --git a/query/src/test/resources/query/sql/query78.sql b/query/src/test/resources/query/sql/query78.sql
index f1d8fb2..d055b8e 100644
--- a/query/src/test/resources/query/sql/query78.sql
+++ b/query/src/test/resources/query/sql/query78.sql
@@ -1,4 +1,4 @@
-select sum(PRICE), LSTG_FORMAT_NAME
+select sum(PRICE) as GMV, LSTG_FORMAT_NAME as FORMAT_NAME
from test_kylin_fact
where (LSTG_FORMAT_NAME in ('ABIN')) or (LSTG_FORMAT_NAME>='FP-GTC' and LSTG_FORMAT_NAME<='Others')
group by LSTG_FORMAT_NAME
[36/37] incubator-kylin git commit: Merge branch 'master' into staging
Posted by li...@apache.org.
Merge branch 'master' into staging
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/b876b7ad
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/b876b7ad
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/b876b7ad
Branch: refs/heads/master
Commit: b876b7ad5dfdf5479b49f4c6200efd4c3fd1228e
Parents: c573b07 92ff93e
Author: liyang@apache.org <ya...@D-SHC-00801746.corp.ebay.com>
Authored: Fri Mar 27 03:36:25 2015 +0100
Committer: liyang@apache.org <ya...@D-SHC-00801746.corp.ebay.com>
Committed: Fri Mar 27 03:36:25 2015 +0100
----------------------------------------------------------------------
bin/kylin.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
[14/37] incubator-kylin git commit: KYLIN-650 category resting pages
to different topics
Posted by li...@apache.org.
KYLIN-650 category resting pages to different topics
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/5acdcd23
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/5acdcd23
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/5acdcd23
Branch: refs/heads/master
Commit: 5acdcd23098dda31338ebdb687bc7fc86076b160
Parents: 284ef26
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:23:11 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:23:11 2015 -0700
----------------------------------------------------------------------
README.md | 15 +
docs/REST/Build Cube with Restful API.md | 51 +
...w to consume Kylin REST API in javascript.md | 42 +
docs/REST/Kylin Restful API List.md | 2020 ++++++++++++++++++
.../TechInside/New meta data model structure.md | 18 +
...n Kylin test case with HBase Mini Cluster.md | 15 +
.../How to use kylin remote jdbc driver.md | 83 +
7 files changed, 2244 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index fa180b0..f3d383f 100644
--- a/README.md
+++ b/README.md
@@ -19,8 +19,19 @@ Please follow this installation tutorial to start with Kylin: [Installation Tuto
Advance Topics
-------
+####Connectivity
+1.[How to use kylin remote jdbc driver](docs/Tutorial/How to use kylin remote jdbc driver.md)
+
+2.[Kylin ODBC Driver Tutorial](docs/Tutorial/Kylin ODBC Driver Tutorial.md)
+
####REST
+1.[Kylin Restful API List](docs/REST/Kylin Restful API List.md)
+
+2.[Build Cube with Restful API](docs/REST/Build Cube with Restful API.md)
+
+3.[How to consume Kylin REST API in javascript](docs/REST/How to consume Kylin REST API in javascript.md)
+
####Operations
1.[Kylin Metadata Store](docs/Operations/Kylin Metadata Store.md)
@@ -30,8 +41,12 @@ Advance Topics
####Test
+1.[Run Kylin test case with HBase Mini Cluster](docs/Test/Run Kylin test case with HBase Mini Cluster.md)
+
+
####Technial Details
+1.[New meta data model structure](docs/TechInside/New meta data model structure.md)
Get Help
------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/REST/Build Cube with Restful API.md
----------------------------------------------------------------------
diff --git a/docs/REST/Build Cube with Restful API.md b/docs/REST/Build Cube with Restful API.md
new file mode 100644
index 0000000..4eb5381
--- /dev/null
+++ b/docs/REST/Build Cube with Restful API.md
@@ -0,0 +1,51 @@
+
+### 1. Authentication
+* Currently, Kylin uses [basic authentication](http://en.wikipedia.org/wiki/Basic_access_authentication).
+* Add `Authorization` header to first request for authentication
+* Or you can do a specific request by `POST http://localhost:7070/kylin/api/user/authentication`
+* Once authenticated, client can go subsequent requests with cookies.
+* Example
+```
+ POST http://localhost:7070/kylin/api/user/authentication
+
+ Authorization:Basic xxxxJD124xxxGFxxxSDF
+ Content-Type: application/json;charset=UTF-8
+```
+
+### 2. Get details of cube.
+* `GET http://localhost:7070/kylin/api/cubes?cubeName={cube_name}&limit=15&offset=0`
+* Client can find cube segment date ranges in returned cube detail.
+* Example
+```
+ GET http://localhost:7070/kylin/api/cubes?cubeName=test_kylin_cube_with_slr&limit=15&offset=0
+
+ Authorization:Basic xxxxJD124xxxGFxxxSDF
+ Content-Type: application/json;charset=UTF-8
+```
+
+### 3. Then submit a build job of the cube.
+* `PUT http://localhost:7070/kylin/api/cubes/{cube_name}/rebuild`
+* For put request body detail please refer to [service doc](https://github.com/KylinOLAP/Kylin/wiki/Restful-Service-Doc).
+ * `startTime` and `endTime` should be utc timestamp.
+ * `buildType` can be `BUILD` or `MERGE`. `BUILD` is for building a new segment or refreshing an existing segment. `MERGE` is for merging multiple existing segments into one bigger segment.
+* This method will return a newly created job instance, in which the uuid is the identity of job to track job status.
+* Example
+```
+ PUT http://localhost:7070/kylin/api/cubes/test_kylin_cube_with_slr/rebuild
+
+ Authorization:Basic xxxxJD124xxxGFxxxSDF
+ Content-Type: application/json;charset=UTF-8
+
+ {
+ "startTime": 0,
+ "endTime": 1388563200000,
+ "buildType": "BUILD"
+ }
+```
+
+### 4. Track job status.
+* `GET http://localhost:7070/kylin/api/jobs/{job_uuid}`
+* Returned `job_status` represents current status of job.
+
+### 5. If the job failed, you can resume the job.
+* `PUT http://localhost:7070/kylin/api/jobs/{job_uuid}/resume`
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/REST/How to consume Kylin REST API in javascript.md
----------------------------------------------------------------------
diff --git a/docs/REST/How to consume Kylin REST API in javascript.md b/docs/REST/How to consume Kylin REST API in javascript.md
new file mode 100644
index 0000000..a0ea1b6
--- /dev/null
+++ b/docs/REST/How to consume Kylin REST API in javascript.md
@@ -0,0 +1,42 @@
+Kylin securicty authorization is based on basic access authorization, so when you want to use API in your javascript,you need to involve the authorization info in http headers.
+
+
+## Here is a demo show how to use Kylin Query API.
+```
+$.ajaxSetup({
+ headers: { 'Authorization': "Basic eWFu**********X***ZA==", 'Content-Type': 'application/json;charset=utf-8' } // use your own authorization code here
+ });
+ var request = $.ajax({
+ url: "http://hostname/kylin/api/query",
+ type: "POST",
+ data: '{"sql":"select count(*) from SUMMARY;","offset":0,"limit":50000,"partial":true,"project":"test"}',
+ dataType: "json"
+ });
+ request.done(function( msg ) {
+ alert(msg);
+ });
+ request.fail(function( jqXHR, textStatus ) {
+ alert( "Request failed: " + textStatus );
+ });
+
+```
+
+
+For what is basic access authorization ,click here http://en.wikipedia.org/wiki/Basic_access_authentication
+
+## Keypoint:
+1. add basic access authorization info in heders.
+2. use right ajax type and right data synax
+
+How to generate your authorization code (download and import "jquery.base64.js" from https://github.com/yckart/jquery.base64.js)
+
+```
+var authorizationCode = $.base64('encode', 'NT_USERNAME' + ":" + 'NT_PASSWORD');
+
+$.ajaxSetup({
+ headers: {
+ 'Authorization': "Basic " + authorizationCode,
+ 'Content-Type': 'application/json;charset=utf-8'
+ }
+});
+```
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/REST/Kylin Restful API List.md
----------------------------------------------------------------------
diff --git a/docs/REST/Kylin Restful API List.md b/docs/REST/Kylin Restful API List.md
new file mode 100644
index 0000000..f1398e9
--- /dev/null
+++ b/docs/REST/Kylin Restful API List.md
@@ -0,0 +1,2020 @@
+## Kylin Restful API List
+
+* QUERY
+ * Login
+ * Query
+ * List queryable tables
+* CUBE
+ * List cubes
+ * Rebuild cube
+ * Disable cube
+ * Purge cube
+ * Enable cube
+* JOB
+ * Resume job
+ * Discard job
+ * Get job step log
+* Metadata
+ * Get Hive Table
+ * Get Hive Table (Extend Info)
+ * Get Hive Tables
+ * Load Hive Table
+* Wipe cache
+
+***
+
+### Login
+`POST /user/authentication`
+
+for example:
+```sh
+curl -c /path/to/cookiefile.txt -X POST -H "Authorization: Basic XXXXXXXXX" -H 'Content-Type: application/json' http://<host>:<port>/kylin/api/user/authentication
+```
+
+If login successfully, the JSESSIONID will be saved into the cookie file; In the subsequent http requests, attach the cookie, for example:
+```sh
+curl -b /path/to/cookiefile.txt -X PUT -H 'Content-Type: application/json' -d '{"startTime":'1423526400000', "endTime":'1423526400', "buildType":"BUILD"}' http://<host>:<port>/kylin/api/cubes/your_cube/rebuild
+```
+
+#### Header
+Authorization data encoded by basic auth. Header sample:
+Authorization:Basic {data}
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "userDetails":{
+ "password":null,
+ "username":"sample",
+ "authorities":[
+ {
+ "authority":"ROLE_ANALYST"
+ },
+ {
+ "authority":"ROLE_MODELER"
+ }
+ ],
+ "accountNonExpired":true,
+ "accountNonLocked":true,
+ "credentialsNonExpired":true,
+ "enabled":true
+ }
+}
+```
+
+***
+
+### Query
+`POST /query`
+#### Request Body
+* sql - `required` `string` The sql query string.
+* offset - `optional` `int` Query offset. If offset is set in sql, curIndex will be ignored.
+* limit - `optional` `int` Query limit. If limit is set in sql, perPage will be ignored.
+* acceptPartial - `optional` `bool` Flag to switch partial query.
+* project - `optional` `string` Project to perform query. Default value is 'DEFAULT'
+```sh
+{
+ "sql":"select * from TEST_KYLIN_FACT",
+ "offset":0,
+ "limit":50000,
+ "acceptPartial":true,
+ "project":"DEFAULT"
+}
+```
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "columnMetas":[
+ {
+ "isNullable":1,
+ "displaySize":0,
+ "label":"CAL_DT",
+ "name":"CAL_DT",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":0,
+ "scale":0,
+ "columnType":91,
+ "columnTypeName":"DATE",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":10,
+ "label":"LEAF_CATEG_ID",
+ "name":"LEAF_CATEG_ID",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":10,
+ "scale":0,
+ "columnType":4,
+ "columnTypeName":"INTEGER",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":10,
+ "label":"LSTG_SITE_ID",
+ "name":"LSTG_SITE_ID",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":10,
+ "scale":0,
+ "columnType":4,
+ "columnTypeName":"INTEGER",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":5,
+ "label":"SLR_SEGMENT_CD",
+ "name":"SLR_SEGMENT_CD",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":5,
+ "scale":0,
+ "columnType":5,
+ "columnTypeName":"SMALLINT",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":256,
+ "label":"LSTG_FORMAT_NAME",
+ "name":"LSTG_FORMAT_NAME",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":256,
+ "scale":0,
+ "columnType":12,
+ "columnTypeName":"VARCHAR(256)",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":19,
+ "label":"SELLER_ID",
+ "name":"SELLER_ID",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":19,
+ "scale":0,
+ "columnType":-5,
+ "columnTypeName":"BIGINT",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":19,
+ "label":"PRICE",
+ "name":"PRICE",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":19,
+ "scale":4,
+ "columnType":3,
+ "columnTypeName":"DECIMAL(19, 4)",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":19,
+ "label":"MIN_PRICE_",
+ "name":"MIN_PRICE_",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":19,
+ "scale":4,
+ "columnType":3,
+ "columnTypeName":"DECIMAL(19, 4)",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":19,
+ "label":"MAX_PRICE_",
+ "name":"MAX_PRICE_",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":19,
+ "scale":4,
+ "columnType":3,
+ "columnTypeName":"DECIMAL(19, 4)",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ },
+ {
+ "isNullable":1,
+ "displaySize":19,
+ "label":"COUNT__",
+ "name":"COUNT__",
+ "schemaName":null,
+ "catelogName":null,
+ "tableName":null,
+ "precision":19,
+ "scale":0,
+ "columnType":-5,
+ "columnTypeName":"BIGINT",
+ "readOnly":true,
+ "writable":false,
+ "caseSensitive":true,
+ "searchable":false,
+ "currency":false,
+ "signed":true,
+ "autoIncrement":false,
+ "definitelyWritable":false
+ }
+ ],
+ "results":[
+ [
+ "2013-08-07",
+ "32996",
+ "15",
+ "15",
+ "Auction",
+ "10000000",
+ "49.048952730908745",
+ "49.048952730908745",
+ "49.048952730908745",
+ "1"
+ ],
+ [
+ "2013-08-07",
+ "43398",
+ "0",
+ "14",
+ "ABIN",
+ "10000633",
+ "85.78317064220418",
+ "85.78317064220418",
+ "85.78317064220418",
+ "1"
+ ],
+ [
+ "2013-08-07",
+ "170083",
+ "3",
+ "12",
+ "FP-non GTC",
+ "10000688",
+ "49.4518822857833",
+ "49.4518822857833",
+ "49.4518822857833",
+ "1"
+ ],
+ [
+ "2013-08-07",
+ "73506",
+ "0",
+ "14",
+ "Others",
+ "10000858",
+ "35.7556514660872",
+ "35.7556514660872",
+ "35.7556514660872",
+ "1"
+ ],
+ [
+ "2013-08-07",
+ "26262",
+ "0",
+ "5",
+ "FP-GTC",
+ "10000914",
+ "69.63202173718798",
+ "69.63202173718798",
+ "69.63202173718798",
+ "1"
+ ],
+ [
+ "2013-08-07",
+ "1161",
+ "3",
+ "12",
+ "Others",
+ "10000959",
+ "76.72392839984632",
+ "76.72392839984632",
+ "76.72392839984632",
+ "1"
+ ]
+ ],
+ "cube":"test_kylin_cube_with_slr_desc",
+ "affectedRowCount":0,
+ "isException":false,
+ "exceptionMessage":null,
+ "duration":3451,
+ "partial":false
+}
+```
+
+***
+
+### List queryable tables
+`GET /tables_and_columns`
+#### Request Parameters
+* project - `required` `string` The project to load tables
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+[
+ {
+ "columns":[
+ {
+ "table_NAME":"TEST_CAL_DT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"CAL_DT",
+ "data_TYPE":91,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":1,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"DATE"
+ },
+ {
+ "table_NAME":"TEST_CAL_DT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"WEEK_BEG_DT",
+ "data_TYPE":91,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":2,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"DATE"
+ }
+ ],
+ "table_NAME":"TEST_CAL_DT",
+ "table_SCHEM":"EDW",
+ "ref_GENERATION":null,
+ "self_REFERENCING_COL_NAME":null,
+ "type_SCHEM":null,
+ "table_TYPE":"TABLE",
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_CAT":null,
+ "type_NAME":null
+ },
+ {
+ "columns":[
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"LEAF_CATEG_ID",
+ "data_TYPE":4,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":1,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"INTEGER"
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SITE_ID",
+ "data_TYPE":4,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":2,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"INTEGER"
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"META_CATEG_NAME",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":3,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"CATEG_LVL2_NAME",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":4,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"CATEG_LVL3_NAME",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":5,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"USER_DEFINED_FIELD1",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":6,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"USER_DEFINED_FIELD3",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":7,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"UPD_DATE",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":8,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "column_NAME":"UPD_USER",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":9,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ }
+ ],
+ "table_NAME":"TEST_CATEGORY_GROUPINGS",
+ "table_SCHEM":"EDW",
+ "ref_GENERATION":null,
+ "self_REFERENCING_COL_NAME":null,
+ "type_SCHEM":null,
+ "table_TYPE":"TABLE",
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_CAT":null,
+ "type_NAME":null
+ },
+ {
+ "columns":[
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"CAL_DT",
+ "data_TYPE":91,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":1,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"DATE"
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"LEAF_CATEG_ID",
+ "data_TYPE":4,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":2,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"INTEGER"
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"LSTG_SITE_ID",
+ "data_TYPE":4,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":3,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"INTEGER"
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SLR_SEGMENT_CD",
+ "data_TYPE":5,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":4,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"SMALLINT"
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"LSTG_FORMAT_NAME",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":5,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SELLER_ID",
+ "data_TYPE":-5,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":6,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"BIGINT"
+ },
+ {
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "column_NAME":"PRICE",
+ "data_TYPE":3,
+ "nullable":1,
+ "column_SIZE":19,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":4,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":19,
+ "ordinal_POSITION":7,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"DECIMAL(19, 4)"
+ }
+ ],
+ "table_NAME":"TEST_KYLIN_FACT",
+ "table_SCHEM":"EDW",
+ "ref_GENERATION":null,
+ "self_REFERENCING_COL_NAME":null,
+ "type_SCHEM":null,
+ "table_TYPE":"TABLE",
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_CAT":null,
+ "type_NAME":null
+ },
+ {
+ "columns":[
+ {
+ "table_NAME":"TEST_SELLER_TYPE_DIM",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SELLER_TYPE_CD",
+ "data_TYPE":5,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":1,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"SMALLINT"
+ },
+ {
+ "table_NAME":"TEST_SELLER_TYPE_DIM",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SELLER_TYPE_DESC",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":2,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ }
+ ],
+ "table_NAME":"TEST_SELLER_TYPE_DIM",
+ "table_SCHEM":"EDW",
+ "ref_GENERATION":null,
+ "self_REFERENCING_COL_NAME":null,
+ "type_SCHEM":null,
+ "table_TYPE":"TABLE",
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_CAT":null,
+ "type_NAME":null
+ },
+ {
+ "columns":[
+ {
+ "table_NAME":"TEST_SITES",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SITE_ID",
+ "data_TYPE":4,
+ "nullable":1,
+ "column_SIZE":-1,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":-1,
+ "ordinal_POSITION":1,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"INTEGER"
+ },
+ {
+ "table_NAME":"TEST_SITES",
+ "table_SCHEM":"EDW",
+ "column_NAME":"SITE_NAME",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":2,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ },
+ {
+ "table_NAME":"TEST_SITES",
+ "table_SCHEM":"EDW",
+ "column_NAME":"CRE_USER",
+ "data_TYPE":12,
+ "nullable":1,
+ "column_SIZE":256,
+ "buffer_LENGTH":-1,
+ "decimal_DIGITS":0,
+ "num_PREC_RADIX":10,
+ "column_DEF":null,
+ "sql_DATA_TYPE":-1,
+ "sql_DATETIME_SUB":-1,
+ "char_OCTET_LENGTH":256,
+ "ordinal_POSITION":3,
+ "is_NULLABLE":"YES",
+ "scope_CATLOG":null,
+ "scope_SCHEMA":null,
+ "scope_TABLE":null,
+ "source_DATA_TYPE":-1,
+ "iS_AUTOINCREMENT":null,
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_NAME":"VARCHAR(256) CHARACTER SET \"UTF-16LE\" COLLATE \"UTF-16LE$en_US$primary\""
+ }
+ ],
+ "table_NAME":"TEST_SITES",
+ "table_SCHEM":"EDW",
+ "ref_GENERATION":null,
+ "self_REFERENCING_COL_NAME":null,
+ "type_SCHEM":null,
+ "table_TYPE":"TABLE",
+ "table_CAT":"defaultCatalog",
+ "remarks":null,
+ "type_CAT":null,
+ "type_NAME":null
+ }
+]
+```
+
+
+***
+
+### List cubes
+`GET /cubes`
+#### Request Parameters
+* cubeName - `optional` `string` Cube name to find.
+* offset - `required` `int` Offset used by pagination
+* limit - `required` `int ` Cubes per page.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+[
+ {
+ "uuid":"1eaca32a-a33e-4b69-83dd-0bb8b1f8c53b",
+ "last_modified":1407831634847,
+ "name":"test_kylin_cube_with_slr_empty",
+ "owner":null,
+ "version":null,
+ "descriptor":"test_kylin_cube_with_slr_desc",
+ "cost":50,
+ "status":"DISABLED",
+ "segments":[
+ ],
+ "create_time":null,
+ "source_records_count":0,
+ "source_records_size":0,
+ "size_kb":0
+ }
+]
+```
+
+### Rebuild cube
+`PUT /cubes/{cubeName}/rebuild`
+#### Path Variable
+* cubeName - `required` `string` Cube name.
+
+#### Request Body
+* startTime - `required` `long` Start timestamp of data to build, e.g. 1388563200000 for 2014-1-1
+* endTime - `required` `long` End timestamp of data to build
+* buildType - `required` `string` Build type: 'BUILD' OR 'MERGE'
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "uuid":"c143e0e4-ac5f-434d-acf3-46b0d15e3dc6",
+ "last_modified":1407908916705,
+ "name":"test_kylin_cube_with_slr_empty - 19700101000000_20140731160000 - BUILD - PDT 2014-08-12 22:48:36",
+ "type":"BUILD",
+ "duration":0,
+ "related_cube":"test_kylin_cube_with_slr_empty",
+ "related_segment":"19700101000000_20140731160000",
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "mr_waiting":0,
+ "steps":[
+ {
+ "interruptCmd":null,
+ "name":"Create Intermediate Flat Hive Table",
+ "sequence_id":0,
+ "exec_cmd":"hive -e \"DROP TABLE IF EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6;\nCREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\n(\nCAL_DT date\n,LEAF_CATEG_ID int\n,LSTG_SITE_ID int\n,META_CATEG_NAME string\n,CATEG_LVL2_NAME string\n,CATEG_LVL3_NAME string\n,LSTG_FORMAT_NAME string\n,SLR_SEGMENT_CD smallint\n,SELLER_ID bigint\n,PRICE decimal\n)\nROW FORMAT DELIMITED FIELDS TERMINATED BY '\\177'\nSTORED AS SEQUENCEFILE\nLOCATION '/tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6';\nSET mapreduce.job.split.metainfo.maxsize=-1;\nSET mapred.compress.map.output=true;\nSET mapred.map.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compress=true;\nSET ma
pred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compression.type=BLOCK;\nSET mapreduce.job.max.split.locations=2000;\nSET hive.exec.compress.output=true;\nSET hive.auto.convert.join.noconditionaltask = true;\nSET hive.auto.convert.join.noconditionaltask.size = 300000000;\nINSERT OVERWRITE TABLE kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\nSELECT\nTEST_KYLIN_FACT.CAL_DT\n,TEST_KYLIN_FACT.LEAF_CATEG_ID\n,TEST_KYLIN_FACT.LSTG_SITE_ID\n,TEST_CATEGORY_GROUPINGS.META_CATEG_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL2_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL3_NAME\n,TEST_KYLIN_FACT.LSTG_FORMAT_NAME\n,TEST_KYLIN_FACT.SLR_SEGMENT_CD\n,TEST_KYLIN_FACT.SELLER_ID\n,TEST_KYLIN_FACT.PRICE\nFROM TEST_KYLIN_FACT\nINNER JOIN TEST_CAL_DT\nON TEST_KYLIN_FACT.CAL_DT = TEST_CAL_DT.CAL_DT\nINNER JOIN TEST_CATEGORY_GROUPINGS\nON TEST_KYLIN_FACT.LEAF_CATEG_ID = TEST_CATEGORY_GROUPINGS.LEAF_CATEG_ID AN
D TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_CATEGORY_GROUPINGS.SITE_ID\nINNER JOIN TEST_SITES\nON TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_SITES.SITE_ID\nINNER JOIN TEST_SELLER_TYPE_DIM\nON TEST_KYLIN_FACT.SLR_SEGMENT_CD = TEST_SELLER_TYPE_DIM.SELLER_TYPE_CD\nWHERE (test_kylin_fact.cal_dt < '2014-07-31 16:00:00')\n;\n\"",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"SHELL_CMD_HADOOP",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Extract Fact Table Distinct Columns",
+ "sequence_id":1,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_test_kylin_cube_with_slr_empty_Step_1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_FACTDISTINCT",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Dimension Dictionary",
+ "sequence_id":2,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_DICTIONARY",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Base Cuboid Data",
+ "sequence_id":3,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -jobname Kylin_Base_Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_3 -level 0",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_BASECUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 8-Dimension",
+ "sequence_id":4,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_4 -level 1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 7-Dimension",
+ "sequence_id":5,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_5 -level 2",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 6-Dimension",
+ "sequence_id":6,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_6 -level 3",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 5-Dimension",
+ "sequence_id":7,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_7 -level 4",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 4-Dimension",
+ "sequence_id":8,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/4d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_8 -level 5",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Calculate HTable Region Splits",
+ "sequence_id":9,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats -jobname Kylin_Region_Splits_Calculator_test_kylin_cube_with_slr_empty_Step_9 -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_RANGEKEYDISTRIBUTION",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Create HTable",
+ "sequence_id":10,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats/part-r-00000 -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADDOP_NO_MR_CREATEHTABLE",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Convert Cuboid Data to HFile",
+ "sequence_id":11,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -jobname Kylin_HFile_Generator_test_kylin_cube_with_slr_empty_Step_11",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_CONVERTHFILE",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Load HFile to HBase Table",
+ "sequence_id":12,
+ "exec_cmd":" -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile/ -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_BULKLOAD",
+ "info":null,
+ "run_async":false
+ }
+ ],
+ "job_status":"PENDING",
+ "progress":0.0
+}
+```
+
+
+### Disable Cube
+`PUT /cubes/{cubeName}/disable`
+#### Path variable
+* cubeName - `required` `string` Cube name.
+
+#### Response Sample
+(Same as "Enable Cube")
+
+
+### Purge Cube
+`PUT /cubes/{cubeName}/purge`
+#### Path variable
+* cubeName - `required` `string` Cube name.
+
+#### Response Sample
+(Same as "Enable Cube")
+
+
+### Enable Cube
+`PUT /cubes/{cubeName}/enable`
+#### Path variable
+* cubeName - `required` `string` Cube name.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "uuid":"1eaca32a-a33e-4b69-83dd-0bb8b1f8c53b",
+ "last_modified":1407909046305,
+ "name":"test_kylin_cube_with_slr_ready",
+ "owner":null,
+ "version":null,
+ "descriptor":"test_kylin_cube_with_slr_desc",
+ "cost":50,
+ "status":"ACTIVE",
+ "segments":[
+ {
+ "name":"19700101000000_20140531160000",
+ "storage_location_identifier":"KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_READY-19700101000000_20140531160000_BF043D2D-9A4A-45E9-AA59-5A17D3F34A50",
+ "date_range_start":0,
+ "date_range_end":1401552000000,
+ "status":"READY",
+ "size_kb":4758,
+ "source_records":6000,
+ "source_records_size":620356,
+ "last_build_time":1407832663227,
+ "last_build_job_id":"2c7a2b63-b052-4a51-8b09-0c24b5792cda",
+ "binary_signature":null,
+ "dictionaries":{
+ "TEST_CATEGORY_GROUPINGS/CATEG_LVL2_NAME":"/dict/TEST_CATEGORY_GROUPINGS/CATEG_LVL2_NAME/16d8185c-ee6b-4f8c-a919-756d9809f937.dict",
+ "TEST_KYLIN_FACT/LSTG_SITE_ID":"/dict/TEST_SITES/SITE_ID/0bec6bb3-1b0d-469c-8289-b8c4ca5d5001.dict",
+ "TEST_KYLIN_FACT/SLR_SEGMENT_CD":"/dict/TEST_SELLER_TYPE_DIM/SELLER_TYPE_CD/0c5d77ec-316b-47e0-ba9a-0616be890ad6.dict",
+ "TEST_KYLIN_FACT/CAL_DT":"/dict/PREDEFINED/date(yyyy-mm-dd)/64ac4f82-f2af-476e-85b9-f0805001014e.dict",
+ "TEST_CATEGORY_GROUPINGS/CATEG_LVL3_NAME":"/dict/TEST_CATEGORY_GROUPINGS/CATEG_LVL3_NAME/270fbfb0-281c-4602-8413-2970a7439c47.dict",
+ "TEST_KYLIN_FACT/LEAF_CATEG_ID":"/dict/TEST_CATEGORY_GROUPINGS/LEAF_CATEG_ID/2602386c-debb-4968-8d2f-b52b8215e385.dict",
+ "TEST_CATEGORY_GROUPINGS/META_CATEG_NAME":"/dict/TEST_CATEGORY_GROUPINGS/META_CATEG_NAME/0410d2c4-4686-40bc-ba14-170042a2de94.dict"
+ },
+ "snapshots":{
+ "TEST_CAL_DT":"/table_snapshot/TEST_CAL_DT.csv/8f7cfc8a-020d-4019-b419-3c6deb0ffaa0.snapshot",
+ "TEST_SELLER_TYPE_DIM":"/table_snapshot/TEST_SELLER_TYPE_DIM.csv/c60fd05e-ac94-4016-9255-96521b273b81.snapshot",
+ "TEST_CATEGORY_GROUPINGS":"/table_snapshot/TEST_CATEGORY_GROUPINGS.csv/363f4a59-b725-4459-826d-3188bde6a971.snapshot",
+ "TEST_SITES":"/table_snapshot/TEST_SITES.csv/78e0aecc-3ec6-4406-b86e-bac4b10ea63b.snapshot"
+ }
+ }
+ ],
+ "create_time":null,
+ "source_records_count":6000,
+ "source_records_size":0,
+ "size_kb":4758
+}
+```
+
+### Resume Job
+`PUT /jobs/{jobId}/resume`
+
+#### Path variable
+* jobId- `required` `string` Job id.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "uuid":"c143e0e4-ac5f-434d-acf3-46b0d15e3dc6",
+ "last_modified":1407908916705,
+ "name":"test_kylin_cube_with_slr_empty - 19700101000000_20140731160000 - BUILD - PDT 2014-08-12 22:48:36",
+ "type":"BUILD",
+ "duration":0,
+ "related_cube":"test_kylin_cube_with_slr_empty",
+ "related_segment":"19700101000000_20140731160000",
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "mr_waiting":0,
+ "steps":[
+ {
+ "interruptCmd":null,
+ "name":"Create Intermediate Flat Hive Table",
+ "sequence_id":0,
+ "exec_cmd":"hive -e \"DROP TABLE IF EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6;\nCREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\n(\nCAL_DT date\n,LEAF_CATEG_ID int\n,LSTG_SITE_ID int\n,META_CATEG_NAME string\n,CATEG_LVL2_NAME string\n,CATEG_LVL3_NAME string\n,LSTG_FORMAT_NAME string\n,SLR_SEGMENT_CD smallint\n,SELLER_ID bigint\n,PRICE decimal\n)\nROW FORMAT DELIMITED FIELDS TERMINATED BY '\\177'\nSTORED AS SEQUENCEFILE\nLOCATION '/tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6';\nSET mapreduce.job.split.metainfo.maxsize=-1;\nSET mapred.compress.map.output=true;\nSET mapred.map.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compress=true;\nSET ma
pred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compression.type=BLOCK;\nSET mapreduce.job.max.split.locations=2000;\nSET hive.exec.compress.output=true;\nSET hive.auto.convert.join.noconditionaltask = true;\nSET hive.auto.convert.join.noconditionaltask.size = 300000000;\nINSERT OVERWRITE TABLE kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\nSELECT\nTEST_KYLIN_FACT.CAL_DT\n,TEST_KYLIN_FACT.LEAF_CATEG_ID\n,TEST_KYLIN_FACT.LSTG_SITE_ID\n,TEST_CATEGORY_GROUPINGS.META_CATEG_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL2_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL3_NAME\n,TEST_KYLIN_FACT.LSTG_FORMAT_NAME\n,TEST_KYLIN_FACT.SLR_SEGMENT_CD\n,TEST_KYLIN_FACT.SELLER_ID\n,TEST_KYLIN_FACT.PRICE\nFROM TEST_KYLIN_FACT\nINNER JOIN TEST_CAL_DT\nON TEST_KYLIN_FACT.CAL_DT = TEST_CAL_DT.CAL_DT\nINNER JOIN TEST_CATEGORY_GROUPINGS\nON TEST_KYLIN_FACT.LEAF_CATEG_ID = TEST_CATEGORY_GROUPINGS.LEAF_CATEG_ID AN
D TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_CATEGORY_GROUPINGS.SITE_ID\nINNER JOIN TEST_SITES\nON TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_SITES.SITE_ID\nINNER JOIN TEST_SELLER_TYPE_DIM\nON TEST_KYLIN_FACT.SLR_SEGMENT_CD = TEST_SELLER_TYPE_DIM.SELLER_TYPE_CD\nWHERE (test_kylin_fact.cal_dt < '2014-07-31 16:00:00')\n;\n\"",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"SHELL_CMD_HADOOP",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Extract Fact Table Distinct Columns",
+ "sequence_id":1,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_test_kylin_cube_with_slr_empty_Step_1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_FACTDISTINCT",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Dimension Dictionary",
+ "sequence_id":2,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_DICTIONARY",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Base Cuboid Data",
+ "sequence_id":3,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -jobname Kylin_Base_Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_3 -level 0",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_BASECUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 8-Dimension",
+ "sequence_id":4,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_4 -level 1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 7-Dimension",
+ "sequence_id":5,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_5 -level 2",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 6-Dimension",
+ "sequence_id":6,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_6 -level 3",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 5-Dimension",
+ "sequence_id":7,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_7 -level 4",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 4-Dimension",
+ "sequence_id":8,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/4d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_8 -level 5",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Calculate HTable Region Splits",
+ "sequence_id":9,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats -jobname Kylin_Region_Splits_Calculator_test_kylin_cube_with_slr_empty_Step_9 -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_RANGEKEYDISTRIBUTION",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Create HTable",
+ "sequence_id":10,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats/part-r-00000 -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADDOP_NO_MR_CREATEHTABLE",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Convert Cuboid Data to HFile",
+ "sequence_id":11,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -jobname Kylin_HFile_Generator_test_kylin_cube_with_slr_empty_Step_11",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_CONVERTHFILE",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Load HFile to HBase Table",
+ "sequence_id":12,
+ "exec_cmd":" -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile/ -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_BULKLOAD",
+ "info":null,
+ "run_async":false
+ }
+ ],
+ "job_status":"PENDING",
+ "progress":0.0
+}
+```
+
+### Discard Job
+`PUT /jobs/{jobId}/cancel`
+#### Path variable
+* jobId- `required` `string` Job id.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "uuid":"c143e0e4-ac5f-434d-acf3-46b0d15e3dc6",
+ "last_modified":1407908916705,
+ "name":"test_kylin_cube_with_slr_empty - 19700101000000_20140731160000 - BUILD - PDT 2014-08-12 22:48:36",
+ "type":"BUILD",
+ "duration":0,
+ "related_cube":"test_kylin_cube_with_slr_empty",
+ "related_segment":"19700101000000_20140731160000",
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "mr_waiting":0,
+ "steps":[
+ {
+ "interruptCmd":null,
+ "name":"Create Intermediate Flat Hive Table",
+ "sequence_id":0,
+ "exec_cmd":"hive -e \"DROP TABLE IF EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6;\nCREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\n(\nCAL_DT date\n,LEAF_CATEG_ID int\n,LSTG_SITE_ID int\n,META_CATEG_NAME string\n,CATEG_LVL2_NAME string\n,CATEG_LVL3_NAME string\n,LSTG_FORMAT_NAME string\n,SLR_SEGMENT_CD smallint\n,SELLER_ID bigint\n,PRICE decimal\n)\nROW FORMAT DELIMITED FIELDS TERMINATED BY '\\177'\nSTORED AS SEQUENCEFILE\nLOCATION '/tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6';\nSET mapreduce.job.split.metainfo.maxsize=-1;\nSET mapred.compress.map.output=true;\nSET mapred.map.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compress=true;\nSET ma
pred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;\nSET mapred.output.compression.type=BLOCK;\nSET mapreduce.job.max.split.locations=2000;\nSET hive.exec.compress.output=true;\nSET hive.auto.convert.join.noconditionaltask = true;\nSET hive.auto.convert.join.noconditionaltask.size = 300000000;\nINSERT OVERWRITE TABLE kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6\nSELECT\nTEST_KYLIN_FACT.CAL_DT\n,TEST_KYLIN_FACT.LEAF_CATEG_ID\n,TEST_KYLIN_FACT.LSTG_SITE_ID\n,TEST_CATEGORY_GROUPINGS.META_CATEG_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL2_NAME\n,TEST_CATEGORY_GROUPINGS.CATEG_LVL3_NAME\n,TEST_KYLIN_FACT.LSTG_FORMAT_NAME\n,TEST_KYLIN_FACT.SLR_SEGMENT_CD\n,TEST_KYLIN_FACT.SELLER_ID\n,TEST_KYLIN_FACT.PRICE\nFROM TEST_KYLIN_FACT\nINNER JOIN TEST_CAL_DT\nON TEST_KYLIN_FACT.CAL_DT = TEST_CAL_DT.CAL_DT\nINNER JOIN TEST_CATEGORY_GROUPINGS\nON TEST_KYLIN_FACT.LEAF_CATEG_ID = TEST_CATEGORY_GROUPINGS.LEAF_CATEG_ID AN
D TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_CATEGORY_GROUPINGS.SITE_ID\nINNER JOIN TEST_SITES\nON TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_SITES.SITE_ID\nINNER JOIN TEST_SELLER_TYPE_DIM\nON TEST_KYLIN_FACT.SLR_SEGMENT_CD = TEST_SELLER_TYPE_DIM.SELLER_TYPE_CD\nWHERE (test_kylin_fact.cal_dt < '2014-07-31 16:00:00')\n;\n\"",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"SHELL_CMD_HADOOP",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Extract Fact Table Distinct Columns",
+ "sequence_id":1,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_test_kylin_cube_with_slr_empty_Step_1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_FACTDISTINCT",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Dimension Dictionary",
+ "sequence_id":2,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/fact_distinct_columns",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_DICTIONARY",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build Base Cuboid Data",
+ "sequence_id":3,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/kylin_intermediate_test_kylin_cube_with_slr_desc_19700101000000_20140731160000_c143e0e4_ac5f_434d_acf3_46b0d15e3dc6 -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -jobname Kylin_Base_Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_3 -level 0",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_BASECUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 8-Dimension",
+ "sequence_id":4,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/base_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_4 -level 1",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 7-Dimension",
+ "sequence_id":5,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/8d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_5 -level 2",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 6-Dimension",
+ "sequence_id":6,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/7d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_6 -level 3",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 5-Dimension",
+ "sequence_id":7,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/6d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_7 -level 4",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Build N-Dimension Cuboid Data : 4-Dimension",
+ "sequence_id":8,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -segmentname 19700101000000_20140731160000 -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/5d_cuboid -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/4d_cuboid -jobname Kylin_ND-Cuboid_Builder_test_kylin_cube_with_slr_empty_Step_8 -level 5",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NDCUBOID",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Calculate HTable Region Splits",
+ "sequence_id":9,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats -jobname Kylin_Region_Splits_Calculator_test_kylin_cube_with_slr_empty_Step_9 -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_RANGEKEYDISTRIBUTION",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Create HTable",
+ "sequence_id":10,
+ "exec_cmd":" -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/rowkey_stats/part-r-00000 -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADDOP_NO_MR_CREATEHTABLE",
+ "info":null,
+ "run_async":false
+ },
+ {
+ "interruptCmd":null,
+ "name":"Convert Cuboid Data to HFile",
+ "sequence_id":11,
+ "exec_cmd":" -conf C:/kylin/Kylin/server/src/main/resources/hadoop_job_conf_medium.xml -cubename test_kylin_cube_with_slr_empty -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/cuboid/* -output /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -jobname Kylin_HFile_Generator_test_kylin_cube_with_slr_empty_Step_11",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_CONVERTHFILE",
+ "info":null,
+ "run_async":true
+ },
+ {
+ "interruptCmd":null,
+ "name":"Load HFile to HBase Table",
+ "sequence_id":12,
+ "exec_cmd":" -input /tmp/kylin-c143e0e4-ac5f-434d-acf3-46b0d15e3dc6/test_kylin_cube_with_slr_empty/hfile/ -htablename KYLIN-CUBE-TEST_KYLIN_CUBE_WITH_SLR_EMPTY-19700101000000_20140731160000_11BB4326-5975-4358-804C-70D53642E03A -cubename test_kylin_cube_with_slr_empty",
+ "interrupt_cmd":null,
+ "exec_start_time":0,
+ "exec_end_time":0,
+ "exec_wait_time":0,
+ "step_status":"PENDING",
+ "cmd_type":"JAVA_CMD_HADOOP_NO_MR_BULKLOAD",
+ "info":null,
+ "run_async":false
+ }
+ ],
+ "job_status":"PENDING",
+ "progress":0.0
+}
+```
+
+### Get job step output
+`GET /{jobId}/steps/{stepId}/output`
+#### Path Variable
+* jobId - `required` `string` Job id.
+* stepId - `required` `string` Step id; the step id is composed by jobId with step sequence id; for example, the jobId is "fb479e54-837f-49a2-b457-651fc50be110", its 3rd step id is "fb479e54-837f-49a2-b457-651fc50be110-3",
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "cmd_output":"log string"
+}
+```
+
+
+### Get Hive Table
+`GET /tables/{tableName}`
+#### Request Parameters
+* tableName - `required` `string` table name to find.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ uuid: "69cc92c0-fc42-4bb9-893f-bd1141c91dbe",
+ name: "SAMPLE_07",
+ columns: [{
+ id: "1",
+ name: "CODE",
+ datatype: "string"
+ }, {
+ id: "2",
+ name: "DESCRIPTION",
+ datatype: "string"
+ }, {
+ id: "3",
+ name: "TOTAL_EMP",
+ datatype: "int"
+ }, {
+ id: "4",
+ name: "SALARY",
+ datatype: "int"
+ }],
+ database: "DEFAULT",
+ last_modified: 1419330476755
+}
+```
+
+### Get Hive Table (Extend Info)
+`GET /tables/{tableName}/exd-map`
+#### Request Parameters
+* tableName - `optional` `string` table name to find.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "minFileSize": "46055",
+ "totalNumberFiles": "1",
+ "location": "hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/sample_07",
+ "lastAccessTime": "1418374103365",
+ "lastUpdateTime": "1398176493340",
+ "columns": "struct columns { string code, string description, i32 total_emp, i32 salary}",
+ "partitionColumns": "",
+ "EXD_STATUS": "true",
+ "maxFileSize": "46055",
+ "inputformat": "org.apache.hadoop.mapred.TextInputFormat",
+ "partitioned": "false",
+ "tableName": "sample_07",
+ "owner": "hue",
+ "totalFileSize": "46055",
+ "outputformat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
+}
+```
+
+### Get Hive Tables
+`GET /tables`
+#### Request Parameters
+* project- `required` `string` will list all tables in the project.
+* ext- `optional` `boolean` set true to get extend info of table.
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+[
+ {
+ uuid: "53856c96-fe4d-459e-a9dc-c339b1bc3310",
+ name: "SAMPLE_08",
+ columns: [{
+ id: "1",
+ name: "CODE",
+ datatype: "string"
+ }, {
+ id: "2",
+ name: "DESCRIPTION",
+ datatype: "string"
+ }, {
+ id: "3",
+ name: "TOTAL_EMP",
+ datatype: "int"
+ }, {
+ id: "4",
+ name: "SALARY",
+ datatype: "int"
+ }],
+ database: "DEFAULT",
+ cardinality: {},
+ last_modified: 0,
+ exd: {
+ minFileSize: "46069",
+ totalNumberFiles: "1",
+ location: "hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/sample_08",
+ lastAccessTime: "1398176495945",
+ lastUpdateTime: "1398176495981",
+ columns: "struct columns { string code, string description, i32 total_emp, i32 salary}",
+ partitionColumns: "",
+ EXD_STATUS: "true",
+ maxFileSize: "46069",
+ inputformat: "org.apache.hadoop.mapred.TextInputFormat",
+ partitioned: "false",
+ tableName: "sample_08",
+ owner: "hue",
+ totalFileSize: "46069",
+ outputformat: "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
+ }
+}, {
+ uuid: "82d72d9c-705c-4706-aaf5-358ac2661cf9",
+ name: "TEST_KYLIN_FACT",
+ columns: [{
+ id: "1",
+ name: "TRANS_ID",
+ datatype: "bigint"
+ }, {
+ id: "2",
+ name: "CAL_DT",
+ datatype: "date"
+ }, {
+ id: "3",
+ name: "LSTG_FORMAT_NAME",
+ datatype: "string"
+ }, {
+ id: "4",
+ name: "LEAF_CATEG_ID",
+ datatype: "int"
+ }, {
+ id: "5",
+ name: "LSTG_SITE_ID",
+ datatype: "int"
+ }, {
+ id: "6",
+ name: "SLR_SEGMENT_CD",
+ datatype: "smallint"
+ }, {
+ id: "7",
+ name: "PRICE",
+ datatype: "decimal(38,16)"
+ }, {
+ id: "8",
+ name: "ITEM_COUNT",
+ datatype: "bigint"
+ }, {
+ id: "9",
+ name: "SELLER_ID",
+ datatype: "bigint"
+ }],
+ database: "DEFAULT",
+ cardinality: {},
+ last_modified: 0,
+ exd: {
+ minFileSize: "607328",
+ totalNumberFiles: "1",
+ location: "hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/test_kylin_fact",
+ lastAccessTime: "1418899958520",
+ lastUpdateTime: "1414051037179",
+ columns: "struct columns { i64 trans_id, date cal_dt, string lstg_format_name, i32 leaf_categ_id, i32 lstg_site_id, i16 slr_segment_cd, decimal(38,16) price, i64 item_count, i64 seller_id}",
+ partitionColumns: "",
+ EXD_STATUS: "true",
+ maxFileSize: "607328",
+ inputformat: "org.apache.hadoop.mapred.TextInputFormat",
+ partitioned: "false",
+ tableName: "test_kylin_fact",
+ owner: "root",
+ totalFileSize: "607328",
+ outputformat: "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
+ }
+ }
+]
+```
+
+### Load Hive Tables
+`POST /tables/{tables}/{project}`
+
+* tables- `required` `string` tables you want to load from hive,split with comma.
+* project- `required` `String` specif.
+
+
+#### Response Sample
+```sh
+Status: 200OK
+```
+```sh
+{
+ "result.loaded": ["DEFAULT.SAMPLE_07"],
+ "result.unloaded": ["sapmle_08"]
+}
+```
+
+
+### Wipe cache
+`GET /cache/{type}/{name}/{action}`
+#### Path variable
+* type - `required` `string` 'METADATA' or 'CUBE'
+* name - `required` `string` Cache key.
+* action - `required` `string` 'create', 'update' or 'drop'
+
+#### Response Sample
+```sh
+Status: 200OK
+```
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/5acdcd23/docs/TechInside/New meta data model structure.md
----------------------------------------------------------------------
diff --git a/docs/TechInside/New meta data model structure.md b/docs/TechInside/New meta data model structure.md
new file mode 100644
index 0000000..74fc74b
--- /dev/null
+++ b/docs/TechInside/New meta data model structure.md
@@ -0,0 +1,18 @@
+#New meta data model structure
+Kylin is doing a round code refactor which will introduce the following two changes on meta data:
+
+* Abstract a "model_desc" layer from "cube_desc"
+
+Before define a cube, user will need firstly define a model ("model_desc"); The model defines which is the fact table, which are lookup tables and how they be joined;
+
+With the model be defined, when user define a cube ("cube_desc"), he/she only need to specify the table/column name for a dimension, as the join conditions have already been defined;
+
+This abstraction is to extend the meta data to fulfill non-cube queries (coming soon);
+
+* Support data tables from multiple hive databases;
+
+User has the case that tables are from multiple hive database, and the table name might be the same; To support this case Kylin will use the database name + table name as the unique name for tables; And user need to specify the database name (if it is not "default") in SQL when query Kylin.
+
+Here is a sample; the fact table "test_kylin_fact" is from default hive database, you don't need to specify the db name; while lookup table is from "edw", you need use "edw.test_cal_dt" in the query:
+
+ select test_cal_dt.Week_Beg_Dt, sum(price) as c1, count(1) as c2 from test_kylin_fact inner JOIN edw.test_cal_dt as test_cal_dt ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt where test_kylin_fact.lstg_format_name='ABIN'
\ No newline at end of file
[30/37] incubator-kylin git commit: merge conflict
Posted by li...@apache.org.
merge conflict
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/b6a85cfc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/b6a85cfc
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/b6a85cfc
Branch: refs/heads/master
Commit: b6a85cfcb29eebe9cc3e7c0b065fabec5c935d9f
Parents: a74b911 f9e3a6a
Author: lukehan <lu...@apache.org>
Authored: Tue Mar 24 18:14:58 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Tue Mar 24 18:14:58 2015 -0700
----------------------------------------------------------------------
README.md | 88 +-
bin/kylin.sh | 16 +-
bin/setenv.sh | 36 +
.../common/util/HBaseMiniclusterHelper.java | 2 +-
.../java/org/apache/kylin/cube/CubeManager.java | 50 +-
.../org/apache/kylin/cube/model/CubeDesc.java | 4 +
...requently Asked Questions on Installation.md | 26 -
...equently Asked Questions on Installation.pdf | Bin 52261 -> 0 bytes
docs/Installation/Hadoop Environment.md | 21 +
.../How to generate install binaries.md | 6 +
docs/Installation/Installation.md | 38 +
.../Installation/Multiple Kylin REST servers.md | 9 +
...f Hadoop CLI Installation (Dev Env Setup).md | 76 -
... Hadoop CLI Installation (Dev Env Setup).pdf | Bin 44909 -> 0 bytes
.../Installation/Off Hadoop CLI Installation.md | 67 +
docs/Installation/On Hadoop CLI installation.md | 106 +-
.../Installation/On Hadoop CLI installation.pdf | Bin 704398 -> 0 bytes
...On Hadoop Kylin installation using Docker.md | 2 +-
...n Hadoop Kylin installation using Docker.pdf | Bin 40098 -> 0 bytes
docs/Installation/install_and_run.md | 83 -
docs/Installation/off_cli_install_scene.png | Bin 0 -> 72567 bytes
docs/Installation/on_cli_install_scene.png | Bin 0 -> 53722 bytes
docs/JobEngine/Design.md | 2 +-
.../MISC/FAQ on Kylin Installation and Usage.md | 43 +
docs/MISC/How to Contribute.md | 42 +
docs/MISC/Kylin Document Conventions.md | 9 +
.../Advanced settings of Kylin environment.md | 28 +
docs/Operations/Export Kylin HBase data.md | 19 +
docs/Operations/Kylin Metadata Store.md | 31 +
docs/README.md | 0
docs/REST/Build Cube with Restful API.md | 51 +
...w to consume Kylin REST API in javascript.md | 42 +
docs/REST/Kylin Restful API List.md | 2020 ++++++++++++++++++
.../TechInside/New meta data model structure.md | 18 +
...n Kylin test case with HBase Mini Cluster.md | 15 +
.../How to use kylin remote jdbc driver.md | 87 +
...in Cube Build and Job Monitoring Tutorial.md | 10 +-
...n Cube Build and Job Monitoring Tutorial.pdf | Bin 1054988 -> 0 bytes
docs/Tutorial/Kylin Cube Creation Tutorial.md | 9 +-
docs/Tutorial/Kylin Cube Creation Tutorial.pdf | Bin 1488946 -> 0 bytes
.../Kylin Cube Permission Grant Tutorial.pdf | Bin 358047 -> 0 bytes
docs/Tutorial/Kylin ODBC Driver Tutorial.md | 6 +
docs/Tutorial/Kylin ODBC Driver Tutorial.pdf | Bin 34429 -> 0 bytes
docs/Tutorial/Kylin Web Tutorial.md | 8 +-
docs/Tutorial/Kylin Web Tutorial.pdf | Bin 1714951 -> 0 bytes
docs/Tutorial/Kylin and Excel Tutorial.md | 3 +
docs/Tutorial/Kylin and Tableau Tutorial.md | 6 +-
docs/Tutorial/Kylin and Tableau Tutorial.pdf | Bin 1246553 -> 0 bytes
docs/Tutorial/Quick play with a sample cube.md | 20 +
docs/website/_includes/head.cn.html | 10 +-
docs/website/_includes/head.html | 9 +-
.../minicluster/b-kylin/kylin.properties | 2 +-
.../minicluster/b-kylin/meta/kylin.properties | 2 +-
.../test_case_data/minicluster/kylin.properties | 2 +-
.../org/apache/kylin/jdbc/stub/KylinClient.java | 8 +-
.../org/apache/kylin/job/JoinedFlatTable.java | 46 +-
.../kylin/job/hadoop/AbstractHadoopJob.java | 3 +-
.../hadoop/hive/CubeJoinedFlatTableDesc.java | 29 -
.../job/hadoop/hive/IIJoinedFlatTableDesc.java | 20 -
.../job/hadoop/hive/IJoinedFlatTableDesc.java | 5 -
.../kylin/job/tools/CubeMigrationCLI.java | 4 +-
.../job/hadoop/cube/BaseCuboidJobTest.java | 3 +
.../job/hadoop/cube/MergeCuboidJobTest.java | 4 +-
.../kylin/job/hadoop/cube/NDCuboidJobTest.java | 4 +-
.../cube/RangeKeyDistributionJobTest.java | 4 +-
.../kylin/metadata/project/ProjectL2Cache.java | 27 +-
pom.xml | 4 +
query/src/test/resources/query/sql/query78.sql | 4 +
.../apache/kylin/rest/service/JobService.java | 110 +-
.../kylin/storage/hbase/CubeStorageEngine.java | 15 +-
webapp/app/js/controllers/cubes.js | 2 +-
webapp/app/partials/query/query_detail.html | 2 +-
72 files changed, 2886 insertions(+), 532 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/b6a85cfc/bin/kylin.sh
----------------------------------------------------------------------
diff --cc bin/kylin.sh
index f8db9e5,232feef..bffe37f
--- a/bin/kylin.sh
+++ b/bin/kylin.sh
@@@ -35,9 -34,9 +34,10 @@@ the
export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$HBASE_CLASSPATH_PREFIX
export HBASE_CLASSPATH=$hive_dependency:${HBASE_CLASSPATH}
+ export JAVA_OPTS="-Xms2048M -Xmx2048M"
- hbase ${JAVA_OPTS} -Djava.util.logging.config.file=${tomcat_root}/conf/logging.properties \
+ hbase ${KYLIN_EXTRA_START_OPTS} \
+ -Djava.util.logging.config.file=${tomcat_root}/conf/logging.properties \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true \
-Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true \
[02/37] incubator-kylin git commit: update ga tracking settings
Posted by li...@apache.org.
update ga tracking settings
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/3625d536
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/3625d536
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/3625d536
Branch: refs/heads/master
Commit: 3625d536764f6ac6c0121a592b37e6378bd2f9f4
Parents: 13d791a
Author: lukehan <lu...@apache.org>
Authored: Mon Mar 16 13:49:00 2015 -0700
Committer: lukehan <lu...@apache.org>
Committed: Mon Mar 16 13:49:00 2015 -0700
----------------------------------------------------------------------
docs/website/_includes/head.cn.html | 6 ++++++
docs/website/_includes/head.html | 5 +++++
2 files changed, 11 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/3625d536/docs/website/_includes/head.cn.html
----------------------------------------------------------------------
diff --git a/docs/website/_includes/head.cn.html b/docs/website/_includes/head.cn.html
index 7e4f3f7..bb8a7e8 100644
--- a/docs/website/_includes/head.cn.html
+++ b/docs/website/_includes/head.cn.html
@@ -33,8 +33,14 @@
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ //oringal tracker for kylin.io
ga('create', 'UA-55534813-1', 'auto');
+ //new tracker for kylin.incubator.apache.org
+ ga('create', 'UA-55534813-2', 'auto', {'name':'incubator'});
+
ga('send', 'pageview');
+ ga('incubator.send', 'pageview');
</script>
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/3625d536/docs/website/_includes/head.html
----------------------------------------------------------------------
diff --git a/docs/website/_includes/head.html b/docs/website/_includes/head.html
index 7e4f3f7..7c005ee 100644
--- a/docs/website/_includes/head.html
+++ b/docs/website/_includes/head.html
@@ -33,8 +33,13 @@
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+ //oringal tracker for kylin.io
ga('create', 'UA-55534813-1', 'auto');
+ //new tracker for kylin.incubator.apache.org
+ ga('create', 'UA-55534813-2', 'auto', {'name':'incubator'});
+
ga('send', 'pageview');
+ ga('incubator.send', 'pageview');
</script>
[09/37] incubator-kylin git commit: KYLIN-650 remove reference to wiki
Posted by li...@apache.org.
KYLIN-650 remove reference to wiki
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/7faa216c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/7faa216c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/7faa216c
Branch: refs/heads/master
Commit: 7faa216c056d3076414ce5b350bf6c432b162bbb
Parents: f22582d
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 19:20:55 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 19:20:55 2015 -0700
----------------------------------------------------------------------
docs/Tutorial/Kylin and Tableau Tutorial.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/7faa216c/docs/Tutorial/Kylin and Tableau Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin and Tableau Tutorial.md b/docs/Tutorial/Kylin and Tableau Tutorial.md
index a2b28b2..dad90a8 100644
--- a/docs/Tutorial/Kylin and Tableau Tutorial.md
+++ b/docs/Tutorial/Kylin and Tableau Tutorial.md
@@ -25,7 +25,7 @@ Enter your Sever location and credentials: server host, port, username and passw
-Click "Connect" to get the list of projects that you have permission to access. See details about permission in [Kylin Cube Permission Grant Tutorial](https://github.com/KylinOLAP/Kylin/wiki/Kylin-Cube-Permission-Grant-Tutorial). Then choose the project you want to connect in the drop down list.
+Click "Connect" to get the list of projects that you have permission to access. See details about permission in [Kylin Cube Permission Grant Tutorial](Kylin Cube Permission Grant Tutorial.md). Then choose the project you want to connect in the drop down list.
[24/37] incubator-kylin git commit: KYLIN-650 fix link
Posted by li...@apache.org.
KYLIN-650 fix link
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/bed423f8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/bed423f8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/bed423f8
Branch: refs/heads/master
Commit: bed423f8c7c27f471c9e9f3c26d4780599dcf712
Parents: 8934297
Author: honma <ho...@ebay.com>
Authored: Sat Mar 21 10:34:03 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Sat Mar 21 10:34:03 2015 -0700
----------------------------------------------------------------------
docs/MISC/FAQ on Kylin Installation and Usage.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/bed423f8/docs/MISC/FAQ on Kylin Installation and Usage.md
----------------------------------------------------------------------
diff --git a/docs/MISC/FAQ on Kylin Installation and Usage.md b/docs/MISC/FAQ on Kylin Installation and Usage.md
index a99e9f0..a568614 100644
--- a/docs/MISC/FAQ on Kylin Installation and Usage.md
+++ b/docs/MISC/FAQ on Kylin Installation and Usage.md
@@ -40,4 +40,4 @@ Kylin uses dictionary to encode the values in each column, this greatly reduce t
The cardinality of dimensions is an important measure of cube complexity. The higher the cardinality, the bigger the cube, and thus the longer to build and the slower to query. Cardinality > 1,000 is worth attention and > 1,000,000 should be avoided at best effort. For optimal cube performance, try reduce high cardinality by categorize values or derive features.
#### What is the difference between Kylin and Druid
-take a look at http://mail-archives.apache.org/mod_mbox/incubator-kylin-dev/201503.mbox/browser
+take a look at http://mail-archives.apache.org/mod_mbox/incubator-kylin-dev/201503.mbox/%3Ctencent_0DAD681A15F3B2F2379CADC9%40qq.com%3E
[10/37] incubator-kylin git commit: KYLIN-650 add maillist
Posted by li...@apache.org.
KYLIN-650 add maillist
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/eff13931
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/eff13931
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/eff13931
Branch: refs/heads/master
Commit: eff1393143a8795b7b08c1ba05d564c914cb09f1
Parents: 7faa216
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 19:30:17 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 19:30:17 2015 -0700
----------------------------------------------------------------------
README.md | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/eff13931/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 28c2df6..232758c 100644
--- a/README.md
+++ b/README.md
@@ -17,15 +17,18 @@ Tutorial
Please follow this installation tutorial to start with Kylin: [Installation Tutorial](docs/Installation/Installation.md)
+Get Help
+------------
+
+The fastest way to get response from our developers is to send email to our mail list <de...@kylin.incubator.apache.org>, and remember to subscribe our mail list via <de...@kylin.incubator.apache.org>
+
Resources
------------
* Web Site: <http://kylin.io>
-* Google Group: [Kylin OLAP Group](https://groups.google.com/forum/#!forum/kylin-olap)
-
-* Developer Mail: <ky...@googlegroups.com>
+* Developer Mail: <de...@kylin.incubator.apache.org>
* How To Contribute: See [wiki](https://github.com/KylinOLAP/Kylin/wiki/How-to-Contribute)
[19/37] incubator-kylin git commit: KYLIN-650 typo
Posted by li...@apache.org.
KYLIN-650 typo
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/c66a74b3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/c66a74b3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/c66a74b3
Branch: refs/heads/master
Commit: c66a74b37dc03f6115bc69c5de64b747d893719f
Parents: 269eac0
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:55:15 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:55:15 2015 -0700
----------------------------------------------------------------------
README.md | 2 +-
docs/MISC/Kylin Document Convensions.md | 9 ---------
docs/MISC/Kylin Document Conventions.md | 9 +++++++++
3 files changed, 10 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/c66a74b3/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 39bb3e1..144e01b 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,6 @@ Resources
* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
-* [Kylin Document Convensions](docs/MISC/Kylin Document Convensions.md)
+* [Kylin Document Conventions](docs/MISC/Kylin Document Conventions.md)
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/c66a74b3/docs/MISC/Kylin Document Convensions.md
----------------------------------------------------------------------
diff --git a/docs/MISC/Kylin Document Convensions.md b/docs/MISC/Kylin Document Convensions.md
deleted file mode 100644
index a5b4fa0..0000000
--- a/docs/MISC/Kylin Document Convensions.md
+++ /dev/null
@@ -1,9 +0,0 @@
-Kylin Document Convensions
------
-
-1. Keep documentation in markdown files
-2. In markdown files, try to use relative URI for document/image reference.
-3. Never link to github wiki pages
-4. Try not to simply list all docuemnts in one page. Instead, try to organize different documents into a "story", so that readers are presented deeper materials as they proceed.
-5. Always keep documents updated
-6. ...
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/c66a74b3/docs/MISC/Kylin Document Conventions.md
----------------------------------------------------------------------
diff --git a/docs/MISC/Kylin Document Conventions.md b/docs/MISC/Kylin Document Conventions.md
new file mode 100644
index 0000000..5e42852
--- /dev/null
+++ b/docs/MISC/Kylin Document Conventions.md
@@ -0,0 +1,9 @@
+Kylin Document Conventions
+-----
+
+1. Keep documentation in markdown files
+2. In markdown files, try to use relative URI for document/image reference.
+3. Never link to github wiki pages
+4. Try not to simply list all docuemnts in one page. Instead, try to organize different documents into a "story", so that readers are presented deeper materials as they proceed.
+5. Always keep documents updated
+6. ...
\ No newline at end of file
[05/37] incubator-kylin git commit: add a separate setenv.sh for jvm
settings, debug settings and native libarary
Posted by li...@apache.org.
add a separate setenv.sh for jvm settings, debug settings and native libarary
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/a540f402
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/a540f402
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/a540f402
Branch: refs/heads/master
Commit: a540f4024134f2198c52c0a8dfede10da2a08f39
Parents: 014ef83
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 16:10:11 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 18:56:21 2015 -0700
----------------------------------------------------------------------
bin/kylin.sh | 16 ++++++++--------
bin/setenv.sh | 36 ++++++++++++++++++++++++++++++++++++
2 files changed, 44 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/a540f402/bin/kylin.sh
----------------------------------------------------------------------
diff --git a/bin/kylin.sh b/bin/kylin.sh
index a6983df..232feef 100644
--- a/bin/kylin.sh
+++ b/bin/kylin.sh
@@ -7,12 +7,6 @@ then
tomcat_root=${dir}/../tomcat
export tomcat_root
- #if [ ! -z "$KYLIN_LD_LIBRARY_PATH" ]
- #then
- # echo "KYLIN_LD_LIBRARY_PATH is set to $KYLIN_LD_LIBRARY_PATH"
- #else
- # exit 1
- #fi
#The location of all hadoop/hbase configurations are difficult to get.
#Plus, some of the system properties are secretly set in hadoop/hbase shell command.
@@ -23,7 +17,6 @@ then
#In this way we no longer need to explicitly configure hadoop/hbase related classpath for tomcat,
#hbase command will do all the dirty tasks for us:
- #-Djava.library.path=${KYLIN_LD_LIBRARY_PATH} \
useSandbox=`cat ${KYLIN_HOME}/conf/kylin.properties | grep 'kylin.sandbox' | awk -F '=' '{print $2}'`
@@ -31,12 +24,19 @@ then
if [ "$useSandbox" = "true" ]
then spring_profile="sandbox"
fi
+
+ #retrive $hive_dependency
source ${dir}/find-hive-dependency.sh
+ #retrive $KYLIN_EXTRA_START_OPTS
+ if [ -f "${dir}/setenv.sh" ]
+ then source ${dir}/setenv.sh
+ fi
export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$HBASE_CLASSPATH_PREFIX
export HBASE_CLASSPATH=$hive_dependency:${HBASE_CLASSPATH}
- hbase -Djava.util.logging.config.file=${tomcat_root}/conf/logging.properties \
+ hbase ${KYLIN_EXTRA_START_OPTS} \
+ -Djava.util.logging.config.file=${tomcat_root}/conf/logging.properties \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true \
-Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true \
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/a540f402/bin/setenv.sh
----------------------------------------------------------------------
diff --git a/bin/setenv.sh b/bin/setenv.sh
new file mode 100755
index 0000000..53f614d
--- /dev/null
+++ b/bin/setenv.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+
+# uncomment following to for it to take effect
+# export KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+DisableExplicitGC"
+
+# uncomment following to for it to take effect
+# export KYLIN_DEBUG_SETTINGS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -javaagent:${KYLIN_HOME}/lib/CustomAgent.jar -Dcom.ebay.lnp.rmiagent.port=8080"
+
+# uncomment following to for it to take effect
+# export KYLIN_LD_LIBRARY_SETTINGS="-Djava.library.path=/apache/hadoop/lib/native/Linux-amd64-64"
+
+export KYLIN_EXTRA_START_OPTS=""
+
+if [ ! -z "${KYLIN_JVM_SETTINGS}" ]
+then
+ echo "KYLIN_JVM_SETTINGS is ${KYLIN_JVM_SETTINGS}"
+ KYLIN_EXTRA_START_OPTS="${KYLIN_JVM_SETTINGS} ${KYLIN_EXTRA_START_OPTS}"
+else
+ echo "KYLIN_JVM_SETTINGS is not set, using default jvm settings"
+fi
+
+if [ ! -z "${KYLIN_DEBUG_SETTINGS}" ]
+then
+ echo "KYLIN_DEBUG_SETTINGS is ${KYLIN_DEBUG_SETTINGS}"
+ KYLIN_EXTRA_START_OPTS="${KYLIN_DEBUG_SETTINGS} ${KYLIN_EXTRA_START_OPTS}"
+else
+ echo "KYLIN_DEBUG_SETTINGS is not set, will not enable remote debuging"
+fi
+
+if [ ! -z "${KYLIN_LD_LIBRARY_SETTINGS}" ]
+then
+ echo "KYLIN_LD_LIBRARY_SETTINGS is ${KYLIN_LD_LIBRARY_SETTINGS}"
+ KYLIN_EXTRA_START_OPTS="${KYLIN_LD_LIBRARY_SETTINGS} ${KYLIN_EXTRA_START_OPTS}"
+else
+ echo "KYLIN_LD_LIBRARY_SETTINGS is not set, lzo compression at MR and hbase might not work"
+fi
[16/37] incubator-kylin git commit: KYLIN-650 Kylin Document
Convensions
Posted by li...@apache.org.
KYLIN-650 Kylin Document Convensions
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/a8757d57
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/a8757d57
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/a8757d57
Branch: refs/heads/master
Commit: a8757d576083d69ee996ae75c63bfae813aedbb4
Parents: af42057
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:36:55 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:36:55 2015 -0700
----------------------------------------------------------------------
README.md | 7 +++++--
docs/MISC/Kylin Document Convensions.md | 8 ++++++++
2 files changed, 13 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/a8757d57/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index fba7c0a..39bb3e1 100644
--- a/README.md
+++ b/README.md
@@ -17,7 +17,7 @@ Tutorial
Please follow this installation tutorial to start with Kylin: [Installation Tutorial](docs/Installation/Installation.md)
-Advance Topics
+Advanced Topics
-------
####Connectivity
1.[How to use kylin remote jdbc driver](docs/Tutorial/How to use kylin remote jdbc driver.md)
@@ -67,5 +67,8 @@ Resources
* How To Contribute: See [this](docs/MISC/How to Contribute.md)
-* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
+* Apache Proposal: [Apache Kylin](https://wiki.apache.org/incubator/KylinProposal)
+
+* [Kylin Document Convensions](docs/MISC/Kylin Document Convensions.md)
+
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/a8757d57/docs/MISC/Kylin Document Convensions.md
----------------------------------------------------------------------
diff --git a/docs/MISC/Kylin Document Convensions.md b/docs/MISC/Kylin Document Convensions.md
new file mode 100644
index 0000000..0d45dd9
--- /dev/null
+++ b/docs/MISC/Kylin Document Convensions.md
@@ -0,0 +1,8 @@
+Kylin Document Convernsions
+-----
+
+1. Keep documentation in markdown files
+2. In markdown files, try to use relative URI for document/image reference.
+3. Never link to github wiki pages
+4. Try not to simply list all docuemnts in one page. Instead, try to organize different documents into a "story", so that readers are presented deeper materials as they proceed.
+5. ...
\ No newline at end of file
[08/37] incubator-kylin git commit: KYLIN-650 reiterate port issue in
ODBC tutorial
Posted by li...@apache.org.
KYLIN-650 reiterate port issue in ODBC tutorial
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/f22582da
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/f22582da
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/f22582da
Branch: refs/heads/master
Commit: f22582da3b7f77b0bab405b2b5d3396322626813
Parents: ae75f5f
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 19:16:12 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 19:16:12 2015 -0700
----------------------------------------------------------------------
docs/Tutorial/Kylin and Tableau Tutorial.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/f22582da/docs/Tutorial/Kylin and Tableau Tutorial.md
----------------------------------------------------------------------
diff --git a/docs/Tutorial/Kylin and Tableau Tutorial.md b/docs/Tutorial/Kylin and Tableau Tutorial.md
index 8630dab..a2b28b2 100644
--- a/docs/Tutorial/Kylin and Tableau Tutorial.md
+++ b/docs/Tutorial/Kylin and Tableau Tutorial.md
@@ -10,11 +10,13 @@ Kylin and Tableau Tutorial
> More detail information or any issue, please contact Kylin Team: `kylinolap@gmail.com`
### Step 1. Install ODBC Driver
-Refer to wiki page [Kylin ODBC Driver Tutorial](https://github.com/KylinOLAP/Kylin/wiki/Kylin-ODBC-Driver-Tutorial).
+Refer to wiki page [Kylin ODBC Driver Tutorial](Kylin ODBC Driver Tutorial.md).
### Step 2. Connect to Kylin Server
> We recommended to use Connect Using Driver instead of Using DSN since Tableau team will not manage your DSN on Tableau Servers.
+**The snapshots are a little bit outdated because at that time Kylin ues port 9080 as default port, for new comers please use the new default port 7070**
+
Connect Using Driver: Select "Other Database(ODBC)" in the left panel and choose KylinODBCDriver in the pop-up window.
[17/37] incubator-kylin git commit: KYLIN-650 add more convensions
Posted by li...@apache.org.
KYLIN-650 add more convensions
Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/0be89306
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/0be89306
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/0be89306
Branch: refs/heads/master
Commit: 0be893069d7e4fae7dcaf4b85ce839f930ce93d7
Parents: a8757d5
Author: honma <ho...@ebay.com>
Authored: Thu Mar 19 23:38:56 2015 -0700
Committer: honma <ho...@ebay.com>
Committed: Thu Mar 19 23:38:56 2015 -0700
----------------------------------------------------------------------
docs/MISC/Kylin Document Convensions.md | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/0be89306/docs/MISC/Kylin Document Convensions.md
----------------------------------------------------------------------
diff --git a/docs/MISC/Kylin Document Convensions.md b/docs/MISC/Kylin Document Convensions.md
index 0d45dd9..52fa6f7 100644
--- a/docs/MISC/Kylin Document Convensions.md
+++ b/docs/MISC/Kylin Document Convensions.md
@@ -5,4 +5,5 @@ Kylin Document Convernsions
2. In markdown files, try to use relative URI for document/image reference.
3. Never link to github wiki pages
4. Try not to simply list all docuemnts in one page. Instead, try to organize different documents into a "story", so that readers are presented deeper materials as they proceed.
-5. ...
\ No newline at end of file
+5. Always keep documents updated
+6. ...
\ No newline at end of file