You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kylin.apache.org by sh...@apache.org on 2018/06/21 00:35:23 UTC
[kylin] 01/02: Add superset and redash,
move jdbc datasource to tutorial
This is an automated email from the ASF dual-hosted git repository.
shaofengshi pushed a commit to branch document
in repository https://gitbox.apache.org/repos/asf/kylin.git
commit 94d03cd3e935eab10c8237eeb2d160e516a7a531
Author: shaofengshi <sh...@apache.org>
AuthorDate: Thu Jun 21 08:34:49 2018 +0800
Add superset and redash, move jdbc datasource to tutorial
---
website/_data/docs23.yml | 5 +-
website/_docs23/index.md | 3 ++
.../setup_jdbc_datasource.md} | 56 +++++++++++++++++++---
3 files changed, 53 insertions(+), 11 deletions(-)
diff --git a/website/_data/docs23.yml b/website/_data/docs23.yml
index bb8a44b..40012c0 100644
--- a/website/_data/docs23.yml
+++ b/website/_data/docs23.yml
@@ -49,6 +49,7 @@
- tutorial/setup_systemcube
- tutorial/use_cube_planner
- tutorial/use_dashboard
+ - tutorial/setup_jdbc_datasource
- title: Integration
docs:
@@ -78,7 +79,3 @@
- howto/howto_update_coprocessor
- howto/howto_install_ranger_kylin_plugin
- howto/howto_enable_zookeeper_acl
- - howto/howto_setup_systemcube
- - howto/howto_use_cube_planner
- - howto/howto_use_dashboard
- - howto/howto_setup_jdbc_datasource
diff --git a/website/_docs23/index.md b/website/_docs23/index.md
index 3b60dd7..390990a 100644
--- a/website/_docs23/index.md
+++ b/website/_docs23/index.md
@@ -42,6 +42,7 @@ Tutorial
10. [Setup System Cube](tutorial/setup_systemcube.html)
11. [Optimize with Cube Planner](tutorial/use_cube_planner.html)
12. [Use System Dashboard](tutorial/use_dashboard.html)
+13. [Setup JDBC Data Source](tutorial/setup_jdbc_datasource.html)
Connectivity and APIs
@@ -59,6 +60,8 @@ Connectivity and APIs
11. [Connect from Apache Spark](tutorial/spark.html)
12. [Connect from Hue](tutorial/hue.html)
13. [Connect from Qlik Sense](tutorial/Qlik.html)
+14. [Connect from Apache Superset](/blog/2018/01/01/kylin-and-superset/)
+15. [Connect from Redash](/blog/2018/05/08/redash-kylin-plugin-strikingly/)
Operations
diff --git a/website/_docs23/howto/howto_setup_jdbc_datasource.md b/website/_docs23/tutorial/setup_jdbc_datasource.md
similarity index 55%
rename from website/_docs23/howto/howto_setup_jdbc_datasource.md
rename to website/_docs23/tutorial/setup_jdbc_datasource.md
index 7653360..881d76f 100644
--- a/website/_docs23/howto/howto_setup_jdbc_datasource.md
+++ b/website/_docs23/tutorial/setup_jdbc_datasource.md
@@ -2,7 +2,7 @@
layout: docs23
title: Setup JDBC Data Source
categories: howto
-permalink: /docs23/howto/howto_setup_jdbc_datasource.html
+permalink: /docs23/tutorial/setup_jdbc_datasource.html
---
> Available since Apache Kylin v2.3.x
@@ -19,24 +19,66 @@ Kylin uses Apache Sqoop to load data from relational databases to HDFS. Download
2. Prepare JDBC driver
- Copy Kylin JDBC driver from `$KYLIN_HOME/lib` to `$SQOOP_HOME/lib`.
+You need to download the JDBC Driver of your database to the Kylin server. The JDBC driver jar should be added to `$KYLIN_HOME/ext` and `$SQOOP_HOME/lib` folder.
3. Configure Kylin
-In `$KYLIN_HOME/conf/kylin.properties`, add the following configurations (assumming your MySQL is in the same host):
+In `$KYLIN_HOME/conf/kylin.properties`, add the following configurations.
+
+MySQL sample:
```
kylin.source.default=8
-kylin.source.jdbc.connection-url=jdbc:mysql://localhost:3306/employees
+kylin.source.jdbc.connection-url=jdbc:mysql://hostname:3306/employees
kylin.source.jdbc.driver=com.mysql.jdbc.Driver
kylin.source.jdbc.dialect=mysql
-kylin.source.jdbc.user=your_mysql_username
-kylin.source.jdbc.pass=your_mysql_password
+kylin.source.jdbc.user=your_username
+kylin.source.jdbc.pass=your_password
kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin
kylin.source.jdbc.filed-delimiter=|
```
-Please note, when configure these parameters in `conf/kylin.properties`, all your projects are using the JDBC as data source. If you need access different types of data source, you need configure these parameters at project level, this is the recommended way.
+SQL Server sample:
+
+```
+kylin.source.default=8
+kylin.source.jdbc.connection-url=jdbc:sqlserver://hostname:1433;database=sample
+kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
+kylin.source.jdbc.dialect=mssql
+kylin.source.jdbc.user=your_username
+kylin.source.jdbc.pass=your_password
+kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin
+kylin.source.jdbc.filed-delimiter=|
+```
+
+Amazon Redshift sample:
+
+```
+kylin.source.default=8
+kylin.source.jdbc.connection-url=jdbc:redshift://hostname:5439/sample
+kylin.source.jdbc.driver=com.amazon.redshift.jdbc.Driver
+kylin.source.jdbc.dialect=default
+kylin.source.jdbc.user=user
+kylin.source.jdbc.pass=pass
+kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin
+kylin.source.default=8
+kylin.source.jdbc.filed-delimiter=|
+```
+
+There is another parameter specifing how many splits should be divided. Sqoop would run a mapper for each split.
+
+```
+kylin.source.jdbc.sqoop-mapper-num=4
+```
+
+To make each mapper gets even input, the splitting column is selected by the following rules:
+ * ShardBy column, if exists;
+ * Partition date column, if exists;
+ * High cardinality column, if exists;
+ * Numeric column, if exists;
+ * A column at first glance.
+
+Please note, when configure these parameters in `conf/kylin.properties`, all your projects are using the JDBC as data source. If you need access different types of data source, you need configure these parameters at project level, this is the recommended way (since Kylin v2.4.0).
## Load tables from JDBC data source